centos7 安裝 cdh5 集羣超詳細教程

系統環境：
主節點： master 10.18.1.117
從節點： slave1, slave2, slave3 10.18.1.118, 10.18.119, 10.18.1.120

一、準備工作，系統環境搭建

預先準備工作：

CDH 系統優化之關閉透明大頁面及設置swappiness

http://blog.csdn.net/post_yuan/article/details/54967508

1. 關閉透明大頁面

首先查看透明大頁是否啓用，[always] never表示已啓用，always [never]表示已禁用

[root@master ~]# cat /sys/kernel/mm/transparent_hugepage/defrag 
[always] madvise never

[root@master ~]# echo never > /sys/kernel/mm/transparent_hugepage/defrag
[root@master ~]# echo never > /sys/kernel/mm/transparent_hugepage/enabled

直接運行上面兩個命令暫時生效，重啓後恢復。

2. 修改swappiness

Linux內核參數vm.swappiness，值的範圍爲0~100，表示系統什麼時候開始進行物理內存與虛擬內存的交換。舉個例子，系統總內存爲64G，vm.swappiness爲60，表示在系統內存使用64*0.4=25.6G的時候開始物理內存與虛擬內存的交換，這個動作勢必會影響系統的性能。因此，Cloudera建議把這個值修改爲1~10。

首先檢查當前設置的vm.swappiness值，

[root@master ~]# cat /proc/sys/vm/swappiness 
30

臨時修改swappiness（重啓後恢復）

[root@master ~]# sysctl -w vm.swappiness=10
vm.swappiness = 10

永久生效（重啓後不恢復）

[root@master ~]# echo "vm.swappiness=10" >> /etc/sysctl.conf

3. 修改hostname，並配置/etc/hosts 文件

修改hostname:

[root@master ~]# hostnamectl set-hostname master

從節點slave同樣操作修改hostname

修改hosts文件：

[root@master ~]# vim /etc/hosts

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

10.18.1.117 master
10.18.1.118 slave1
10.18.1.119 slave2
10.18.1.120 slave3

slave節點同時修改

4. 關閉防火牆和selinux

關閉防火牆

[root@master ~]# systemctl stop firewalld.service 
[root@master ~]# systemctl disable firewalld.service 
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service.

從節點slave同樣操作

關閉selinux

#臨時關閉
[root@slave1 ~]# setenforce 0


#永久關閉，重啓後生效
[root@slave1 ~]# vim /etc/selinux/config

SELINUX=disabled

其他從節點slave同樣操作

5. 配置主節點ssh免密碼登錄從節點

生成ssh祕鑰：

[root@master ~]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
Created directory '/root/.ssh'.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
f5:60:9c:d8:59:da:63:c6:9e:0d:9b:15:2e:57:b0:50 root@master
The key's randomart image is:
+--[ RSA 2048]----+
|            o.E..|
|         + B o + |
|        . X O =  |
|         o * @   |
|        S   * .  |
|                 |
|                 |
|                 |
|                 |
+-----------------+

將master公鑰分發到從節點slave上：

[root@master ~]# scp ~/.ssh/id_rsa.pub root@slave1:/root/
The authenticity of host 'slave1 (10.18.1.118)' can't be established.
ECDSA key fingerprint is 8e:31:12:ba:90:92:37:ef:df:6e:9a:b5:ae:76:4d:fc.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'slave1,10.18.1.118' (ECDSA) to the list of known hosts.
root@slave1's password: 
id_rsa.pub 

配置從節點slave：

[root@master ~]# ssh root@slave1
root@slave1's password: 
Last login: Tue Jun  2 11:39:03 2020 from 10.18.24.127
#配置免密登錄
root@slave1 ~]# ssh-keygen -t rsa
[root@slave1 ~]# cat id_rsa.pub >> ~/.ssh/authorized_keys

其他節點同樣操作

免密測試：

[root@master ~]# ssh root@slave1
Last login: Wed Jun  3 16:13:22 2020 from 10.18.1.117

6. 安裝jdk

首先刪除CentOS自帶的Open JDK

[root@slave1 ~]# yum remove java

下載並安裝jdk

#rpm安裝
[root@master home]# rpm -Uvh jdk-8u131-linux-x64.rpm

#配置java環境
vim /etc/profile
#增加：
export JAVA_HOME=/usr/java/jdk1.8.0_131
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

#然後執行：
[root@master home]# source /etc/profile

其他從節點相同操作

7. 安裝MySQL(使用mysql5.6)

安裝mysql5.7在遠程建表時一直提示密碼強度問題，修改數據庫表中的密碼強度不起作用，後改爲mysql5.6.
在主節點master 上安裝 mysql

#獲取repo
[root@master home]# wget https://dev.mysql.com/get/mysql57-community-release-el7-9.noarch.rpm

#安裝repo
[root@master home]# rpm -ivh mysql57-community-release-el7-9.noarch.rpm

#安裝mysql-server
[root@master home]# yum install mysql-server

#啓動mysql
[root@master home]# systemctl start mysqld

#開機啓動
[root@master home]# systemctl enable mysqld

#查看狀態
[root@master home]# systemctl status mysqld
● mysqld.service - MySQL Server
   Loaded: loaded (/usr/lib/systemd/system/mysqld.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2020-06-04 16:00:23 CST; 18s ago
     Docs: man:mysqld(8)
           http://dev.mysql.com/doc/refman/en/using-systemd.html
  Process: 27140 ExecStart=/usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid $MYSQLD_OPTS (code=exited, status=0/SUCCESS)
  Process: 27077 ExecStartPre=/usr/bin/mysqld_pre_systemd (code=exited, status=0/SUCCESS)
 Main PID: 27143 (mysqld)
   CGroup: /system.slice/mysqld.service
           └─27143 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid

Jun 04 16:00:06 slave1 systemd[1]: Starting MySQL Server...
Jun 04 16:00:23 slave1 systemd[1]: Started MySQL Server.


#配置mysql密碼

#獲取初始密碼
[root@master home]# grep 'temporary password' /var/log/mysqld.log
2020-06-04T08:00:20.783779Z 1 [Note] A temporary password is generated for root@localhost: 9cVXGPq-W-.m

#登錄
[root@master home]# mysql -uroot -p9cVXGPq-W-.m
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 5
Server version: 5.7.30

Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> 

#修改密碼強度策略，否則簡單密碼會報錯
mysql> set global validate_password_policy=0;
Query OK, 0 rows affected (0.00 sec)

#修改密碼
mysql> SET PASSWORD = PASSWORD('passw0rd');
Query OK, 0 rows affected, 1 warning (0.00 sec)

#刷新權限
mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.01 sec)

#退出
mysql> exit
Bye

#測試新密碼
[root@master home]# mysql -uroot -ppassw0rd
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 9
Server version: 5.7.30 MySQL Community Server (GPL)

Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> 

#配置遠程登錄權限
mysql> grant all on *.* to 'root'@'%' identified by 'passw0rd' with grant option;
Query OK, 0 rows affected, 1 warning (0.00 sec)

mysql> grant all on *.* to 'root'@'master' identified by 'passw0rd' with grant option;
Query OK, 0 rows affected (0.00 sec)

mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)

mysql> exit
Bye

8. 配置ntp服務

在所有節點上安裝並啓動ntp服務（若沒有可以通過yum安裝）

[root@master home]# systemctl enable ntpd
Created symlink from /etc/systemd/system/multi-user.target.wants/ntpd.service to /usr/lib/systemd/system/ntpd.service.
[root@master home]# systemctl start ntpd
[root@master home]# systemctl status ntpd
● ntpd.service - Network Time Service
   Loaded: loaded (/usr/lib/systemd/system/ntpd.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2020-06-04 16:42:05 CST; 2min 53s left
  Process: 30359 ExecStart=/usr/sbin/ntpd -u ntp:ntp $OPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 30360 (ntpd)
   CGroup: /system.slice/ntpd.service
           └─30360 /usr/sbin/ntpd -u ntp:ntp -g

Jun 04 16:42:04 master ntpd[30360]: Listen normally on 5 lo ::1 UDP 123
Jun 04 16:42:04 master ntpd[30360]: Listen normally on 6 eth0 fe80::8b24:3796:f7a2:a38a UDP 123
Jun 04 16:42:04 master ntpd[30360]: Listening on routing socket on fd #23 for interface updates
Jun 04 16:42:05 master systemd[1]: Started Network Time Service.
Jun 04 16:42:05 master ntpd[30360]: 0.0.0.0 c016 06 restart
Jun 04 16:42:05 master ntpd[30360]: 0.0.0.0 c012 02 freq_set kernel 0.000 PPM
Jun 04 16:42:05 master ntpd[30360]: 0.0.0.0 c011 01 freq_not_set
Jun 04 16:42:12 master ntpd[30360]: 0.0.0.0 c61c 0c clock_step -185.315932 s
Jun 04 16:39:06 master ntpd[30360]: 0.0.0.0 c614 04 freq_mode
Jun 04 16:39:07 master ntpd[30360]: 0.0.0.0 c618 08 no_sys_peer

#查看同步狀態
[root@master home]# ntpstat 
synchronised to NTP server (111.230.189.174) at stratum 3
   time correct to within 148 ms
   polling server every 64 s

9. 搭建 CDH yum 源(待完善）

下載 cdh parcel 和 rpm 包

下載鏈接：

cdh parcel：https://www.cloudera.com/documentation/enterprise/latest/topics/cm_ig_create_local_parcel_repo.html#cmig_topic_21_3_1

CDH - https://archive.cloudera.com/cdh5/parcels/

rpm tar包：http://archive.cloudera.com/cm5/repo-as-tarball/

-rw-r–r--. 1 root root 1934231261 Dec 19 09:15 CDH-5.13.1-1.cdh5.13.1.p0.2-el7.parcel

-rw-r–r--. 1 root root 41 Dec 19 09:10 CDH-5.13.1-1.cdh5.13.1.p0.2-el7.parcel.sha1

drwxrwxr-x. 3 jenkins jenkins 121 Dec 7 06:32 cm

-rw-r–r--. 1 root root 951176044 Dec 19 09:13 cm5.13.1-centos7.tar.gz

-rw-r–r--. 1 root root 106 Dec 19 09:13 cm5.13.1-centos7.tar.gz.md5

-rw-r–r--. 1 root root 114 Dec 19 09:13 cm5.13.1-centos7.tar.gz.sha1

-rw-r–r--. 1 root root 73766 Dec 20 15:27 manifest.json

注意一定在下載parcel的時候，一定要下載目錄下的manifest.json 文件 ， 否則cm-manager認不到parcel ！！！

在master和slave節點配置repo

[root@master home]# cd /etc/yum.repos.d/

[root@master yum.repos.d]# vim cloudera-cm.repo


[cloudera-cm]

# Packages for Cloudera's Distribution for cm, Version 5, on RedHat or CentOS 7 x86_64

name=Cloudera's Distribution for cm, Version 5

baseurl=http://10.18.218.14/bigdata/cm/5/

gpgkey = http://10.18.218.14/bigdata/cm/RPM-GPG-KEY-cloudera

gpgcheck = 1

二、開始部署CDH集羣

1. 安裝 cm-manager (主節點 master)

主節點master安裝cloudera-manager

[root@master yum.repos.d]# yum install cloudera-manager-daemons.x86_64 cloudera-manager-server

安裝MySQL JDBC Driver

[root@master yum.repos.d]# yum install mysql-connector-java.noarch

配置cm 數據庫

[root@master yum.repos.d]# ls /usr/share/cmf/schema/scm_prepare_database.sh
/usr/share/cmf/schema/scm_prepare_database.sh

[root@master ~]# /usr/share/cmf/schema/scm_prepare_database.sh mysql -h master -uroot -ppassw0rd --scm-host master scm scm scm
JAVA_HOME=/usr/java/jdk1.8.0_131
Verifying that we can write to /etc/cloudera-scm-server
Creating SCM configuration file in /etc/cloudera-scm-server
Executing:  /usr/java/jdk1.8.0_131/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
[                          main] DbCommandExecutor              INFO  Successfully connected to database.
All done, your SCM database is configured correctly!

若此處cm數據庫配置報錯，注意前面的數據庫權限配置，我在這地方踩了好幾個坑，mysql5.7的遠程權限開不了，改成mysql5.6。在卸載時除了用yum remove外，還得刪除mysql的安裝文件，否則5.6無法啓動。

啓動cm-server

[root@master ~]# systemctl start cloudera-scm-server

https://www.cloudera.com/documentation/enterprise/latest/topics/cm_ig_install_path_b.html#cmig_topic_6_6_1

這個過程需要花費一小會兒

Wait several minutes for the Cloudera Manager Server to start. To observe the startup process, run tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log on the Cloudera Manager Server host. If the Cloudera Manager Server does not start, see Troubleshooting Installation and Upgrade Problems.

In a web browser, enter http://Server host:7180, where Server host is the FQDN or IP address of the

啓動成功後，會看見以下日誌輸出：

2020-06-11 16:21:40,852 INFO WebServerImpl:org.mortbay.log: jetty-6.1.26.cloudera.4
2020-06-11 16:21:40,854 INFO WebServerImpl:org.mortbay.log: Started [email protected]:7180
2020-06-11 16:21:40,854 INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.
2020-06-11 16:21:40,998 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Finished constructing repo:2020-06-11T08:21:40.998Z
2020-06-11 16:21:43,549 INFO ScmActive-0:com.cloudera.server.cmf.components.ScmActive: ScmActive completed successfully.

出現問題時注意查看日誌

tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log

2. 開始安裝服務

在瀏覽器中輸入mater地址：

10.18.1.117:7180
進入登錄頁面

輸入用戶名admin，密碼admin，添加主機後進入安裝界面

搜索安裝節點，即maser和slave節點：

搜索結果：

選擇parcel爲自己的搭建的庫：

集羣安裝時，選擇自定義存儲庫：

提供主機密碼：

進入安裝環節：

安裝完成：

parcel分發：

正確性檢查：

選擇需要安裝的服務：

根據需求配置服務：

設置數據庫：

此處需要去master的數據庫進行設置：

#登錄數據庫
[root@master ~]# mysql -uroot -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 365
Server version: 5.6.37 MySQL Community Server (GPL)

Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> 
#建立用戶並賦予權限
#創建hive用戶
mysql> grant all privileges on hive.* to 'hive'@'master' identified by 'hive';
Query OK, 0 rows affected (0.01 sec)

mysql> grant all privileges on hive.* to 'hive'@'%' identified by 'hive';
Query OK, 0 rows affected (0.00 sec)

#創建amon用戶
mysql> grant all privileges on amon.* to 'amon'@'master' identified by 'amon';
Query OK, 0 rows affected (0.00 sec)

mysql> grant all privileges on amon.* to 'amon'@'%' identified by 'amon';
Query OK, 0 rows affected (0.00 sec)

#創建oozie用戶
mysql> grant all privileges on oozie.* to 'oozie'@'master' identified by 'oozie';
Query OK, 0 rows affected (0.00 sec)

mysql> grant all privileges on oozie.* to 'oozie'@'%' identified by 'oozie';
Query OK, 0 rows affected (0.00 sec)

#創建hue用戶
mysql> grant all privileges on hue.* to 'hue'@'master' identified by 'hue';
Query OK, 0 rows affected (0.00 sec)

mysql> grant all privileges on hue.* to 'hue'@'%' identified by 'hue';
Query OK, 0 rows affected (0.00 sec)

#刷新權限
mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)


#創建數據庫
#創建hive庫
mysql> create database hive default character set utf8;
Query OK, 1 row affected (0.00 sec)

#創建amon庫
mysql> create database amon default character set utf8;
Query OK, 1 row affected (0.01 sec)

#創建oozie庫
mysql> create database oozie default character set utf8;
Query OK, 1 row affected (0.00 sec)

#創建hue庫
mysql> create database hue default character set utf8;
Query OK, 1 row affected (0.00 sec)

#完成退出
mysql> exit
Bye

配置完數據庫後進行連接測試：

選擇配置後，進行安裝：

若安裝失敗時，注意查看日誌信息。
安裝成功：

管理頁面：

總結

安裝時遇到各種坑，注意查看日誌！！！