flink基礎篇-開發flink，必備的環境搭建

一、安裝

1.1、環境

1. 系統：mac
2. java 版本：java -version

java version "1.8.0_202"
Java(TM) SE Runtime Environment (build 1.8.0_202-b08)
Java HotSpot(TM) 64-Bit Server VM (build 25.202-b08, mixed mode)

3. flink 版本：1.10.0

1.2、啓動

1. 執行命令 ./apache-flink-1.10.0/deps/bin/start-cluster.sh
2. 訪問：http://127.0.0.1:8081/#/overview
   

二、寫一個demo

2.1 在idea上建一個工程，選擇這個archetype

2.2 coding

import org.apache.flink.api.common.functions.FlatMapFunction;
import org.apache.flink.api.java.DataSet;
import org.apache.flink.api.java.ExecutionEnvironment;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.util.Collector;

public class WordCountExample {


    public static void main(String[] args) throws Exception {
        //構建環境
        final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
        //通過字符串構建數據集
        DataSet<String> text = env.fromElements(
                "this is the first example about flink");
        //分割字符串、按照key進行分組、統計相同的key個數
        DataSet<Tuple2<String, Integer>> wordCounts = text
                .flatMap(new LineSplitter())
                .groupBy(0)
                .sum(1);
        //打印
        wordCounts.print();
    }
    //分割字符串的方法
    public static class LineSplitter implements FlatMapFunction<String, Tuple2<String, Integer>> {
        @Override
        public void flatMap(String line, Collector<Tuple2<String, Integer>> out) {
            for (String word : line.split(" ")) {
                out.collect(new Tuple2<String, Integer>(word, 1));
            }
        }
    }
}

3. 執行

3.1 在idea上運行

出現報錯

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/flink/api/common/functions/FlatMapFunction

在pom去掉關於flink的依賴的scope，改爲如下

		<dependency>
			<groupId>org.apache.flink</groupId>
			<artifactId>flink-java</artifactId>
			<version>${flink.version}</version>
		</dependency>
		<dependency>
			<groupId>org.apache.flink</groupId>
			<artifactId>flink-streaming-java_${scala.binary.version}</artifactId>
			<version>${flink.version}</version>
		</dependency>

在執行一次，截取一部分日誌：

3.2 命令提交給Apache Flink Dashboard

1、給程序的jar，copy到./apache-flink-1.10.0/deps/examples目錄下
2、然後提交執行命令

./bin/flink run -c com.learn.fink.WordCountExample ../examples/flink_example-1.0.0-SNAPSHOT.jar 

3、執行結果

3.3 在Apache Flink Dashboard上執行

點擊Add New，可以上傳jar，然後點擊submit

可以看到結果爲