flink(1)下載和啓動

## 下載和啓動

1. 安裝flink需要java 8.x的環境，因此我們首先需要檢查java 8是否已安裝，可以通過java -version檢查。

2. 下載安裝包

   可以在https://flink.apache.org/downloads.html 下載對應的版本，此次我們下載 Apache Flink 1.7.2 for Scala 2.11 (asc, sha512)。

   下載後解壓即可。

   $ cd ~/Downloads        # Go to download directory
   $ tar xzf flink-*.tgz   # Unpack the downloaded archive
   $ cd flink-1.7.2
   

3. 啓動flink

   $ ./bin/start-cluster.sh  # Start Flink
   

   默認是單機版，啓動後我們可以訪問http://localhost:8081，如果頁面訪問正常，我們可以看到有一個可用的TaskManager實例。

   我們也可以通過$ tail log/flink-*-standalonesession-*.log 檢查啓動日誌，確認是否啓動成功。

4. 運行

   我們使用如下java代碼來測試flink功能，這個任務會從socket中讀取文本信息，每隔5秒統計一次每個詞出現的次數。

   public class SocketWindowWordCount {
   
       public static void main(String[] args) throws Exception {
   
           // the port to connect to
           final int port;
           try {
               final ParameterTool params = ParameterTool.fromArgs(args);
               port = params.getInt("port");
           } catch (Exception e) {
               System.err.println("No port specified. Please run 'SocketWindowWordCount --port <port>'");
               return;
           }
   
           // get the execution environment
           final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
   
           // get input data by connecting to the socket
           DataStream<String> text = env.socketTextStream("localhost", port, "\n");
   
           // parse the data, group it, window it, and aggregate the counts
           DataStream<WordWithCount> windowCounts = text
               .flatMap(new FlatMapFunction<String, WordWithCount>() {
                   @Override
                   public void flatMap(String value, Collector<WordWithCount> out) {
                       for (String word : value.split("\\s")) {
                           out.collect(new WordWithCount(word, 1L));
                       }
                   }
               })
               .keyBy("word")
               .timeWindow(Time.seconds(5), Time.seconds(1))
               .reduce(new ReduceFunction<WordWithCount>() {
                   @Override
                   public WordWithCount reduce(WordWithCount a, WordWithCount b) {
                       return new WordWithCount(a.word, a.count + b.count);
                   }
               });
   
           // print the results with a single thread, rather than in parallel
           windowCounts.print().setParallelism(1);
   
           env.execute("Socket Window WordCount");
       }
   
       // Data type for words with count
       public static class WordWithCount {
   
           public String word;
           public long count;
   
           public WordWithCount() {}
   
           public WordWithCount(String word, long count) {
               this.word = word;
               this.count = count;
           }
   
           @Override
           public String toString() {
               return word + " : " + count;
           }
       }
   }
   

   由於需要監聽端口，我們需要啓動一個tcp服務。`

   $ nc -l 9000
   

   第二步，提交flink程序。

   $ ./bin/flink run examples/streaming/SocketWindowWordCount.jar --port 9000

   提交的程序會連接到9000端口並監聽輸入。

   $ nc -l 9000
   lorem ipsum
   ipsum ipsum ipsum
   bye
   

   代碼中我們使用print直接輸出，因此我們可以通過日誌文件查看結果：

   $ tail -f log/flink-*-taskexecutor-*.out
   lorem : 1
   bye : 1
   ipsum : 4
   

   5. 停止flink

      $ ./bin/stop-cluster.sh