課程地址
先來分析一下文件flink-1.10.0-bin-scala_2.11.tgz
解壓之後是這樣的：
conf文件夾下存儲了flink的所有配置文件：
打開flink-conf.yaml分析一下
################################################################################
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
################################################################################
#==============================================================================
# Common
#==============================================================================
# The external address of the host on which the JobManager runs and can be
# reached by the TaskManagers and any clients which want to connect. This setting
# is only used in Standalone mode and may be overwritten on the JobManager side
# by specifying the --host <hostname> parameter of the bin/jobmanager.sh executable.
# In high availability mode, if you use the bin/start-cluster.sh script and setup
# the conf/masters file, this will be taken care of automatically. Yarn/Mesos
# automatically configure the host name based on the hostname of the node where the
# JobManager runs.
jobmanager.rpc.address: localhost
# 本機jobmanager的遠程過程調用地址，即在哪裏啓動當前的集羣，就以當前的localhost作爲當前集羣的
# jobmanager，如果配置高可用可能有多個，一般情況下一個集羣裏就一個jobmanager
# The RPC port where the JobManager is reachable.
jobmanager.rpc.port: 6123
# The heap size for the JobManager JVM
# 以下參數重要，在真正部署提交時可能需要更改的參數
jobmanager.heap.size: 1024m
# 對內存大小，默認1G
# The total process memory size for the TaskManager.
# TaskManager的總進程內存大小
# Note this accounts for all memory usage within the TaskManager process, including JVM metaspace and other overhead.
# 注意，這包括TaskManager進程中的所有內存使用，包括JVM元空間和其他開銷。
taskmanager.memory.process.size: 1568m
# To exclude JVM metaspace and overhead, please, use total Flink memory size instead of 'taskmanager.memory.process.size'.
# 要排除JVM元空間和開銷，請使用總Flink內存大小而不是“taskmanager.memory.process.size”。
# It is not recommended to set both 'taskmanager.memory.process.size' and Flink memory.
# 不建議同時設置兩個'taskmanager.memory.process。大小和內存。
# taskmanager.memory.flink.size: 1280m
# The number of task slots that each TaskManager offers. Each slot runs one parallel pipeline.
# 每個TaskManager提供的任務槽數。每個槽運行一個並行管道。即能執行多少個Task
taskmanager.numberOfTaskSlots: 1
# The parallelism used for programs that did not specify and other parallelism.
# 用於沒有指定的程序的並行度和其他並行度。即默認並行度
parallelism.default: 1
# The default file system scheme and authority.
#
# By default file paths without scheme are interpreted relative to the local
# root file system 'file:///'. Use this to override the default and interpret
# relative paths relative to a different file system,
# for example 'hdfs://mynamenode:12345'
#
# fs.default-scheme
#==============================================================================
# High Availability
#==============================================================================
# The high-availability mode. Possible options are 'NONE' or 'zookeeper'.
# 高可用性模式。可能的選項是“NONE”或“zookeeper”
# high-availability: zookeeper
# The path where metadata for master recovery is persisted. While ZooKeeper stores
# the small ground truth for checkpoint and leader election, this location stores
# the larger objects, like persisted dataflow graphs.
# 保存master持久化元數據的路徑。當ZooKeeper管理員存儲small ground truth作爲checkpoint和leader選舉，
# 這個地點儲存較大的Objects，比如持久數據流圖。
# Must be a durable file system that is accessible from all nodes
# 必須是可從所有節點訪問的持久文件系統
# (like HDFS, S3, Ceph, nfs, ...)
#
# high-availability.storageDir: hdfs:///flink/ha/
# The list of ZooKeeper quorum peers that coordinate the high-availability
# setup. This must be a list of the form:
# "host1:clientPort,host2:clientPort,..." (default clientPort: 2181)
# 協調高可用性配置的ZooKeeper quorum節點的列表，這必須是一個表單的列表:
#
# high-availability.zookeeper.quorum: localhost:2181
# ACL options are based on https://zookeeper.apache.org/doc/r3.1.2/zookeeperProgrammers.html#sc_BuiltinACLSchemes
# It can be either "creator" (ZOO_CREATE_ALL_ACL) or "open" (ZOO_OPEN_ACL_UNSAFE)
# The default value is "open" and it can be changed to "creator" if ZK security is enabled
#
# high-availability.zookeeper.client.acl: open
#==============================================================================
# Fault tolerance and checkpointing
#==============================================================================
# The backend that will be used to store operator state checkpoints if
# checkpointing is enabled.
# 如果啓用了檢查點，backend就會用於存儲操作符狀態檢查點。
#
# Supported backends are 'jobmanager', 'filesystem', 'rocksdb', or the
# <class-name-of-factory>.
#
# state.backend: filesystem
# Directory for checkpoints filesystem, when using any of the default bundled
# state backends.
#
# state.checkpoints.dir: hdfs://namenode-host:port/flink-checkpoints
# Default target directory for savepoints, optional.
#
# state.savepoints.dir: hdfs://namenode-host:port/flink-checkpoints
# Flag to enable/disable incremental checkpoints for backends that
# support incremental checkpoints (like the RocksDB state backend).
#
# state.backend.incremental: false
# The failover strategy, i.e., how the job computation recovers from task failures.
# Only restart tasks that may have been affected by the task failure, which typically includes
# downstream tasks and potentially upstream tasks if their produced data is no longer available for consumption.
# 故障轉移策略，即。，作業計算如何從任務失敗中恢復。
# 僅重啓那些可能受到任務失敗影響的任務，通常包括下游任務和潛在的上游任務(如果它們產生的數據不再可用)。
jobmanager.execution.failover-strategy: region
#==============================================================================
# Rest & web frontend REST或web前端
#==============================================================================
# The port to which the REST client connects to. If rest.bind-port has
# not been specified, then the server will bind to this port as well.
# REST客戶端連接到的端口。如果rest.bind-port未指定，則服務器也將綁定到此端口
#rest.port: 8081
# The address to which the REST client will connect to
#
#rest.address: 0.0.0.0
# Port range for the REST and web server to bind to.
#
#rest.bind-port: 8080-8090
# The address that the REST & web server binds to
#
#rest.bind-address: 0.0.0.0
# Flag to specify whether job submission is enabled from the web-based
# runtime monitor. Uncomment to disable.
# 標記，以指定是否從基於web runtime監視器的啓用作業提交。取消禁用
#web.submit.enable: false
#==============================================================================
# Advanced高級配置
#==============================================================================
# Override the directories for temporary files. If not specified, the
# system-specific Java temporary directory (java.io.tmpdir property) is taken.
#
# For framework setups on Yarn or Mesos, Flink will automatically pick up the
# containers' temp directories without any need for configuration.
#
# Add a delimited list for multiple directories, using the system directory
# delimiter (colon ':' on unix) or a comma, e.g.:
# /data1/tmp:/data2/tmp:/data3/tmp
#
# Note: Each directory entry is read from and written to by a different I/O
# thread. You can include the same directory multiple times in order to create
# multiple I/O threads against that directory. This is for example relevant for
# high-throughput RAIDs.
#
# io.tmp.dirs: /tmp
# The classloading resolve order. Possible values are 'child-first' (Flink's default)
# and 'parent-first' (Java's default).
#
# Child first classloading allows users to use different dependency/library
# versions in their application than those in the classpath. Switching back
# to 'parent-first' may help with debugging dependency issues.
#
# classloader.resolve-order: child-first
# The amount of memory going to the network stack. These numbers usually need
# no tuning. Adjusting them may be necessary in case of an "Insufficient number
# of network buffers" error. The default min is 64MB, the default max is 1GB.
#
# taskmanager.memory.network.fraction: 0.1
# taskmanager.memory.network.min: 64mb
# taskmanager.memory.network.max: 1gb
#==============================================================================
# Flink Cluster Security Configurationflink集羣安全性配置
#==============================================================================
# Kerberos authentication for various components - Hadoop, ZooKeeper, and connectors -
# may be enabled in four steps:
# 1. configure the local krb5.conf file
# 2. provide Kerberos credentials (either a keytab or a ticket cache w/ kinit)
# 3. make the credentials available to various JAAS login contexts
# 4. configure the connector to use JAAS/SASL
# The below configure how Kerberos credentials are provided. A keytab will be used instead of
# a ticket cache if the keytab path and principal are set.
# security.kerberos.login.use-ticket-cache: true
# security.kerberos.login.keytab: /path/to/kerberos/keytab
# security.kerberos.login.principal: flink-user
# The configuration below defines which JAAS login contexts
# security.kerberos.login.contexts: Client,KafkaClient
#==============================================================================
# ZK Security Configuration ZK的安全性配置
#==============================================================================
# Below configurations are applicable if ZK ensemble is configured for security
# 如果ZK集成被配置爲安全的，下面的配置是適用的
# Override below configuration to provide custom ZK service name if configured
# 重寫下面的配置，以在配置時提供自定義ZK服務名稱
# zookeeper.sasl.service-name: zookeeper
# The configuration below must match one of the values set in "security.kerberos.login.contexts"
# 下面的配置必須匹配“security.kerberos.login.contexts”中的一個值
# zookeeper.sasl.login-context-name: Client
#==============================================================================
# HistoryServer 歷史服務器
#==============================================================================
# The HistoryServer is started and stopped via bin/historyserver.sh (start|stop)
# 通過bin/ HistoryServer.sh(啓動|停止)啓動和停止HistoryServer
# Directory to upload completed jobs to. Add this directory to the list of
# monitored directories of the HistoryServer as well (see below).
# 要上傳已完成的作業的目錄。同時將此目錄添加到HistoryServer的監聽目錄列表(參見下面)。
#jobmanager.archive.fs.dir: hdfs:///completed-jobs/
# The address under which the web-based HistoryServer listens.
#historyserver.web.address: 0.0.0.0
# The port under which the web-based HistoryServer listens.
#historyserver.web.port: 8082
# Comma separated list of directories to monitor for completed jobs.
#historyserver.archive.fs.dir: hdfs:///completed-jobs/
# Interval in milliseconds for refreshing the monitored directories.
#historyserver.archive.fs.refresh-interval: 10000
windows 用start-cluster.bat啓動flink
linux 用start-cluster.sh
在windows下裝一個linux子系統就好了，具體怎麼裝參考網上教程，簡單的一批什麼配置都不需要，只要你的windows10夠新哈哈哈
然後linux+windows並行，真香
cd /代表進入我的linux根目錄
cd mnt是我的windows目錄，下面有c d e三個盤符，進入就可以看到相應文件
另外安裝子系統之後要配置jdk才能跑起來flink-scala的腳本，一種方法是wget+rpm文件的網址，另一種方法是tar -xvzf ***.tar.gz文件直接解壓壓縮包到想要安裝的目錄（中間可以用mv 源目錄 目標目錄的命令格式把文件移動到想要安裝的目錄下）
因爲裏面有一個conf目錄下的什麼什麼文件需要JAVA_HOME的配置，而且在jdk1.4版本之後的都不需要配置classpath，vi \etc\profile，i或a進入編輯模式，然後在末尾添加下面的PATH
JAVA_HOME=/usr/local/jdk1.8.0_141 # jdk在哪裏這裏就是哪裏，下面的不需要動的
JAVA_BIN=$JAVA_HOME/bin
JRE_HOME=$JAVA_HOME/jre
JRE_BIN=$JRE_HOME/bin
PATH=$JAVA_BIN:$JRE_BIN:$PATH
export JAVA_HOME JRE_HOME PATH
激活環境變量
source \etc\profile
所以裝完系統之後裝一個jdk，配置好jdk的環境變量，我是配置在\etc\profile裏面的，也就是針對全部用戶進行配置，看教程還有什麼配置在.bashrc和.bashfile亂七八糟的文件裏，對應的就是系統的安全性問題，我是想反正電腦就我一個人用，我就自己配個全局就行了，以後多用戶的時候可能就要好好配環境了。配環境變量的時候用vim命令進入編輯界面，然後a或i進行編輯，按Esc輸入 :wq（保存並退出），如果不想保存就用 :q! 強制退出或者 :q 退出就可以了
裝完jdk直接起我們的flink目錄下的bin目錄下的start-cluster.sh腳本就可以啓動我們的集羣了，jps命令可以查到進程信息
6116 StandaloneSessionClusterEntrypoint單機會話集羣進入點
6475 TaskManagerRunner TaskManger運行器
6556 Jps
這三個東西的具體含義是啥？前面的數字每發送一次jps命令都是會變化的
shell下運行提交一個job
./bin/flink run -c com.atguigu.wc.StreamWordCount -p 1 /mnt/d/MyCodeforIDEA/JAVA/Second_Java/target/Second_Java-1.0-SNAPSHOT-jar-with-dependencies.jar --host localhost --port 7777
bin目錄下的flink腳本+run±c+主類名±p+並行度+打包的jar包位置+運行參數
運行之後會出現：Job has been submitted with JobID 51b585aef6a7f2e7c04dd61caad24719
shell下取消cancel一個job
./bin/flink cancel 51b585aef6a7f2e7c04dd61caad24719
根據運行時起的jobID取消job，如果不記得全部可以用./bin/flink list查看所有的jobID，然後根據jobID取消：
