例子
Flink集羣
mvn package exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount -Dexec.args="--runner=FlinkRunner --flinkMaster=<flink master> --filesToStage=target/word-count-beam-bundled-0.1.jar --inputFile=/path/to/quickstart/pom.xml --output=/tmp/counts" -Pflink-runner
管道參數說明
| Field | Description | Default Value |
|---|---|---|
runner | The pipeline runner to use. This option allows you to determine the pipeline runner at runtime. | Set to FlinkRunnerto run using Flink. |
streaming | Whether streaming mode is enabled or disabled; true if enabled. Set to true if running pipelines with unbounded PCollections. | false |
flinkMaster | The url of the Flink JobManager on which to execute pipelines. This can either be the address of a cluster JobManager, in the form "host:port" or one of the special Strings "[local]" or "[auto]". "[local]" will start a local Flink Cluster in the JVM while "[auto]" will let the system decide where to execute the pipeline based on the environment. | [auto] |
filesToStage | Jar Files to send to all workers and put on the classpath. Here you have to put the fat jar that contains your program along with all dependencies. | empty |
parallelism | The degree of parallelism to be used when distributing operations onto workers. | 1 |
checkpointingInterval | The interval between consecutive checkpoints (i.e. snapshots of the current pipeline state used for fault tolerance). | -1L, i.e. disabled |
numberOfExecutionRetries | Sets the number of times that failed tasks are re-executed. A value of 0 effectively disables fault tolerance. A value of -1 indicates that the system default value (as defined in the configuration) should be used. | -1 |
executionRetryDelay | Sets the delay between executions. A value of -1 indicates that the default value should be used. | -1 |
stateBackend | Sets the state backend to use in streaming mode. The default is to read this setting from the Flink config. | empty, i.e. read from Flink config |