Flink常用命令和安裝部署配置

bin/flink run ./flink.jar
bin/flink run ./flink.jar --input file:///home/user/input.txt --output file:///home/user/output.txt
bin/flink run -p 16 ./flink.jar #parallelism 16
bin/flink run -q ./flink.jar # log output disabled
bin/flink run -d ./flink.jar #detached mode
bin/flink run -m localhost:8081 ./flink.jar #specific JobManager
bin/flink run -c org.apache.flink.examples.java.wordcount.WordCount ./flink #a specific class
bin/flink run -m yarn-cluster -yn 2 -yjm 1024 -ytm 1024 ./flink.jar --input hdfs:///user/input.txt --output hdfs:///user/output.txt
- -yjm, --yarnjobManagerMemory Memory for JobManager Container with optional unit (MB)
- -yn, --yarncontainer Number of YARN container to allocate (=Number of Task Managers)
- -ynm, --yarnname Set a custom name for the application on YARN
- -ytm, --yarntaskManagerMemory Memory/TaskManager Container with optional unit (MB)
bin/flink list
bin/flink list -s
bin/flink list -r
bin/flink list -a
bin/flink list -m yarn-cluster -yid -r
bin/flink cancel
bin/flink cancel -s [targetDirectory] #with a savepoint
bin/flink stop
bin/flink modify -p #streaming jobs only
例如： bin/flink run -m yarn-cluster -yn 1 -yjm 1024 -ytm 1024 ./flink-kafka-hdfs-input-topic.jar --kafka.topic.name “RFID_INFO_TOPIC”

HDFS相關配置
- fs.hdfs.hadoopconf: /etc/SERVICE-HADOOP-ID
JAVA_HOME配置
- env.java.home: /usr/local/jdk1.8.0_144
HA配置
- high-availability: zookeeper
- high-availability.zookeeper.quorum: node001:2181,node002:2181,node003:2181
- high-availability.zookeeper.path.root: /flink
- high-availability.cluster-id: /cluster_one
- high-availability.storageDir: hdfs:///flink/ha/
checkPoint配置
- state.backend: filesystem #‘jobmanager’, ‘filesystem’, ‘rocksdb’
- state.checkpoints.dir: hdfs:///flink-checkpoints
- state.savepoints.dir: hdfs:///flink-savepoints
- state.checkpoints.num-retained: 20
其他相關配置
- jobmanager.rpc.port: 6123
- jobmanager.heap.size: 10240m
- taskmanager.heap.size: 10240m
- taskmanager.numberOfTaskSlots: 6
- parallelism.default: 6
- web.address: node001
- rest.port: 8081
- taskmanager.tmp.dirs: /etc/your/dir/task00,/etc/your/dir/task01,/etc/your/dir/task02 #可以多個路徑，多線程併發執行；
- jobmanager.web.tmpdir: /your/dir
- env.log.dir: /your/dir #默認$FLINK_HOME/log，必須絕對路徑

發表評論

所有評論

還沒有人評論，想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.