Centos 7 安裝Airflow

Airflow 安裝環境 ： centos7   python3.7.0  mysql-8.0.22

一、安裝mysql-8.0.22

參考往日博文：Centos 7 安裝 mysql-8.0.22

二、安裝python3.7.0

參考往日博文：Centos 7 安裝python3.7.0

三、安裝配置

[root@hadoop101 python-3.7.0]# yum install gcc
[root@hadoop101 software]# python3 -m pip install --upgrade pip
[root@hadoop101 software]# python3 -m pip --default-timeout=1000 install paramiko
[root@hadoop101 software]# yum -y install zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel gdbm-devel db4-devel libpcap-devel xz-devel

四、安裝airflow

[root@hadoop101 software]# python3 -m pip --default-timeout=1000 install -U apache-airflow[all]

安裝所有 Airflow 功能，此過程比較漫長，可只安裝基本功能

[root@hadoop101 software]# python3 -m pip --default-timeout=1000 install -U apache-airflow

五、安裝pymysql

[root@hadoop101 software]# python3 -m pip install pymysql

六、配置環境變量

[root@hadoop101 software]# vim /etc/profile
#airflow
export AIRFLOW_HOME=/app/airflow
export AIRFLOW_GPL_UNIDECODE=yes
[root@hadoop101 software]# source /etc/profile

七、初始化

1、初始化數據庫表（默認使用本地得sqlite數據庫）

[root@hadoop101 software]# airflow db init

2、查看其生成文件

[root@hadoop101 software]# cd /app/airflow
[root@hadoop101 airflow]# ls
airflow.cfg  airflow.db  logs  unittests.cfg  webserver_config.py

3、配置MySQL數據庫（創建airflow數據庫，並創建用戶和授權，給airflow訪問數據庫使用）：

[root@hadoop101 airflow]# mysql -u root -proot
mysql> CREATE DATABASE airflow CHARACTER SET UTF8mb3 COLLATE utf8_general_ci;
Query OK, 1 row affected, 2 warnings (0.15 sec)

4、配置airflow使用LocalExecutor執行器，及使用MySQL數據庫：

[root@hadoop101 airflow]# vim airflow/airflow.cfg
# The executor class that airflow should use. Choices include
# SequentialExecutor, LocalExecutor, CeleryExecutor, DaskExecutor, KubernetesExecutor
#executor = SequentialExecutor
executor = LocalExecutor

# The SqlAlchemy connection string to the metadata database.
# SqlAlchemy supports many different database engine, more information
# their website
#sql_alchemy_conn = sqlite:data/airflow/airflow.db
sql_alchemy_conn = mysql+pymysql://root:root@localhost:3306/airflow

5、再次初始化數據庫表

[root@hadoop101 airflow]# airflow db init

 

6、更改MySQL配置

[root@hadoop101 airflow]# vim /etc/my.cnf
[mysqld]
explicit_defaults_for_timestamp=1

或者在數據庫中運行一下語句：

set @@global.explicit_defaults_for_timestamp=on;

7、查看創建的airflow數據庫表：

mysql> use airflow;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> show tables;
+-------------------------------+
| Tables_in_airflow             |
+-------------------------------+
| ab_permission                 |
| ab_permission_view            |
| ab_permission_view_role       |
| ab_register_user              |
| ab_role                       |
| ab_user                       |
| ab_user_role                  |
| ab_view_menu                  |
| alembic_version               |
| connection                    |
| dag                           |
| dag_code                      |
| dag_pickle                    |
| dag_run                       |
| dag_tag                       |
| import_error                  |
| job                           |
| log                           |
| rendered_task_instance_fields |
| sensor_instance               |
| serialized_dag                |
| sla_miss                      |
| slot_pool                     |
| task_fail                     |
| task_instance                 |
| task_reschedule               |
| variable                      |
| xcom                          |
+-------------------------------+
28 rows in set (0.00 sec)

8、創建用戶和登錄密碼

[root@hadoop101 airflow]# airflow users create --username admin --firstname admin --lastname admin --role Admin --email [email protected]
[2021-01-13 23:57:21,825] {manager.py:727} WARNING - No user yet created, use flask fab command to do it.
Password:
Repeat for confirmation:
Admin user admin created

八、服務啓動

1、添加airflow-scheduler服務啓動腳本：

[root@hadoop101 airflow]# airflow webserver
[root@hadoop101 airflow]# airflow scheduler

 

 

2、瀏覽器訪問：http://192.168.17.132:8080/

輸入用戶密碼

安裝完成啦~~撒花~~別忘了一鍵三連哦~~