Flink 下載安裝&運行
Flink 專欄文章
1- 異軍突起的流式計算框架Flink
2- Flink下載安裝&運行
下載安裝
Flink 的安裝有兩種方式
- 官網下載可運行包
- 下載源碼本地打包編譯
環境準備
- JDK 1.8 以上
- Git （無特殊版本要求）
- Maven 3.6 （理論其它版本不影響）
官網下載可運行包
-
訪問下載界面https://flink.apache.org/downloads.html
這裏下載apache-flink-1.10.1.tar.gz
![下載FLink]()
-
解壓下載包到指定目錄
-
以單機方式運行Flink
在解壓後Flink binary根目錄，找到\deps\bin目錄，執行start-cluster.bat, Linux 系統執行start-cluster.sh
訪問本地的8081端口，就可以看到FLink的Web界面了
下載源文件編譯打包
下載源文件打包相對於直接下載官方可運行包稍微麻煩一點點，和本地環境有很大的關係，主要原理是通過git克隆flink源碼，執行mvn install 命令
git clone https://github.com/apache/flink //克隆Flink master代碼
mvn clean package -DskipTests //刪除已有的包
mvn clean package -DskipTests -Dfast //進行打包
Flink UI 界面詳解
Flink 界面如下圖所示：
Overview： Flink 整體概覽Jobs: Flink 提交的任務含正在運行的任務和已經完成的任務Task Managers： 負責具體的任務執行和對應任務在每個節點上的資源申請和管理Job Manager: 負責整個 Flink 集羣任務的調度以及資源的管理Submit New Job: 提交一個新的任務，將jar包上傳到Flink
Flink Hello World
DMEO 案例中是用來計算每個單詞出現的次數，並通過maven打包上傳到Flink 就可以實現計算(本地也可以通過run main 函數查看具體計算效果)
package com.flink.tutorial.helloword;
import org.apache.flink.api.common.functions.FlatMapFunction;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.streaming.api.datastream.DataStream;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.util.Collector;
/**
* @author: cc
* @date: 2020/6/1 16:49
* @description: 計算單詞個數DEMO
*/
public class WordCount {
public static void main(String[] args) throws Exception {
System.out.println("Flink word count begin");
StreamExecutionEnvironment environment = StreamExecutionEnvironment.getExecutionEnvironment();
DataStream<String> dataStream = environment.fromElements(WordCountData.WORDS);
dataStream.flatMap(new Tokenizer())
.keyBy(0)
.sum(1)
.print();
environment.execute("Collect Word count");
System.out.println("Flink word count end");
}
public static final class Tokenizer implements FlatMapFunction<String, Tuple2<String, Integer>> {
@Override
public void flatMap(String value, Collector<Tuple2<String, Integer>> out) {
String[] words = value.split(" ");
for (String word : words) {
out.collect(new Tuple2<>(word, 1));
}
}
}
}
WordCountData 源碼(只是一個輸入，不用太在意內容)
package com.flink.tutorial.helloword;
public class WordCountData {
public static final String[] WORDS = new String[] {
"To be, or not to be,--that is the question:--",
"Whether 'tis nobler in the mind to suffer",
"The slings and arrows of outrageous fortune",
"Or to take arms against a sea of troubles,",
"And by opposing end them?--To die,--to sleep,--",
"No more; and by a sleep to say we end",
"The heartache, and the thousand natural shocks",
"That flesh is heir to,--'tis a consummation",
"Devoutly to be wish'd. To die,--to sleep;--",
"To sleep! perchance to dream:--ay, there's the rub;",
"For in that sleep of death what dreams may come,",
"When we have shuffled off this mortal coil,",
"Must give us pause: there's the respect",
"That makes calamity of so long life;",
"For who would bear the whips and scorns of time,",
"The oppressor's wrong, the proud man's contumely,",
"The pangs of despis'd love, the law's delay,",
"The insolence of office, and the spurns",
"That patient merit of the unworthy takes,",
"When he himself might his quietus make",
"With a bare bodkin? who would these fardels bear,",
"To grunt and sweat under a weary life,",
"But that the dread of something after death,--",
"The undiscover'd country, from whose bourn",
"No traveller returns,--puzzles the will,",
"And makes us rather bear those ills we have",
"Than fly to others that we know not of?",
"Thus conscience does make cowards of us all;",
"And thus the native hue of resolution",
"Is sicklied o'er with the pale cast of thought;",
"And enterprises of great pith and moment,",
"With this regard, their currents turn awry,",
"And lose the name of action.--Soft you now!",
"The fair Ophelia!--Nymph, in thy orisons",
"Be all my sins remember'd."
};
}