一.安裝版本
---------------------------------------------------------------------------------------------------------------------
linus版本：red hat7.4
cdh版本：6.3.1
java版本：1.8
---------------------------------------------------------------------------------------------------------------------
二.使用介質
---------------------------------------------------------------------------------------------------------------------
CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel mysql-connector-java-5.1.47.jar
cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm jdk-8u202-linux-x64.tar.gz
cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm
manifest.json cloudera-manager-server-db-2-6.3.1-1466458.el7.x86_64.rpm
---------------------------------------------------------------------------------------------------------------------
三.搭建前準備工作(默認所有節點都做)
---------------------------------------------------------------------------------------------------------------------
1.配置Ip與主機名映射，命令：vi /etc/hosts：
192.168.80.81 slave1
192.168.80.82 slave2
192.168.80.83 slave3
192.168.80.84 master
2.禁用IPv6:
echo "alias net-pf-10 off" >> /etc/modprobe.d/dist.conf
echo "alias ipv6 off" >> /etc/modprobe.d/dist.conf
3.關閉防火牆：
firewall-cmd --state
systemctl stop firewalld.service
systemctl disable firewalld.service
4.禁用Selinux：
vi /etc/sysconfig/selinux，改爲SELINUX=disabled
5.設置文件打開數量和用戶最大進程數：
在文件最後增加以下內容：
* soft nofile 65535
* hard nofile 65535
* soft nproc 32000
* hard nproc 32000
6.將redhat7的系統鏡像（*.ISO）做爲yum源：
https://blog.csdn.net/vinke666/article/details/91345290
7.卸載Centos自帶的JDK：
使用sudo rpm -qa|grep jdk 或 sudo rpm -qa|grep java找到自帶的jdk，
使用sudo rpm -e --nodeps xxx yyy zzz卸載自帶的JDK
8.安裝JDK：
1.解壓jdk-8u202-linux-x64.tar.gz到一個目錄，
2.並將路徑填寫到/etc/profile文件的最下方：
JAVA_HOME=/apps/jdk1.8.0_202
JRE_HOME=$JAVA_HOME/jre
PATH=$JAVA_HOME/bin:$PATH
export JAVA_HOME JRE_HOME PATH
3.echo "JAVA_HOME=/app/jdk1.8.0_202" >> /etc/environment
9.將/proc/sys/vm/swappiness 設置爲 0 (修改swap空間的swappiness，降低對硬盤的緩存 ):
echo "vm.swappiness=0" >> /etc/sysctl.conf
10.使用ntp同步集羣時間：
1.yum install ntp(所有節點)
2.主節點：
vi /etc/ntp.conf：
註釋掉這幾個
#server 0.centos.pool.ntp.org iburst
#server 1.centos.pool.ntp.org iburst
#server 2.centos.pool.ntp.org iburst
#server 3.centos.pool.ntp.org iburst
添加以下內容
server 127.127.1.0
fudge 127.127.1.0 stratum 10
vi /etc/sysconfig/ntpd 加入下面一句話，用於配置boot時間和系統時間同步
SYNC_HWCLOCK=yes
啓動ntpd服務器
sudo service ntpd start
sudo service ntpd status
chkconfig ntpd on
chkconfig --list |grep ntpd
3.從節點：
vi /etc/ntp.conf：
添加以下內容
server master
4.所有節點：
service ntp restart
11.配置免密登錄:
1.在所有節點執行ssh-keygen (提示時，直接回車即可)
2.在每個節點的/root/.ssh/下創建authorized_keys文件，將其它節點生成的公鑰/root/.ssh/id_rsa.pub的內容放到自己的authorized_keys文件
3.每個節點分別嘗試免密連一次其它節點
12.安裝mysql數據庫(主節點)：
groupadd mysql
useradd -r -g mysql mysql
cd /usr/local/mysql/
chown -R mysql:mysql ./
cp /usr/local/mysql/support-files/mysql.server /etc/init.d/mysql
vi /etc/init.d/mysql
basedir=/usr/local/mysql
datadir=/usr/local/mysql/data
cd /etc
rm -rf my.cnf
vi my.cnf
# *** DO NOT EDIT THIS FILE. It's a template which will be copied to the
# *** default location during install, and will be replaced if you
# *** upgrade to a newer version of MySQL.
[client]
port = 3306
default-character-set=utf8
[mysqld]
# 一般配置選項
basedir = /usr/local/mysql
datadir = /usr/local/mysql/data
port = 3306
character-set-server=utf8
default_storage_engine = InnoDB
sql_mode=STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
cd /usr/local/mysql/bin/
./mysqld --initialize --user=mysql --basedir=/usr/local/mysql --datadir=/usr/local/mysql/data
記錄界面彈出的密碼
b_eFbcYEt60n.
service mysql start
./mysql -uroot -p後邊跟着上邊的密碼
set password=password('123456');
GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '123456';
flush privileges;
chkconfig --add mysql
chkconfig mysql on
vi /etc/profile
export MYSQL_HOME=/usr/local/mysql
export PATH=$PATH:$MYSQL_HOME/bin
source /etc/profile
---------------------------------------------------------------------------------------------------------------------
四.安裝cdh
---------------------------------------------------------------------------------------------------------------------
1.安裝主節點的cdh相關組件：
yum install -y perl
yum install -y psmisc
yum install -y postgresql-server
rpm -ivh cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm
rpm -ivh cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm
rpm -ivh cloudera-manager-server-db-2-6.3.1-1466458.el7.x86_64.rpm
2.配置連接mysql的java驅動：
mkdir -p /usr/share/java/
cp mysql-connector-java-5.1.47.jar /usr/share/java/mysql-connector-java.jar
3.創建cdh相關數據庫：
mysql -u root -p123456
CREATE DATABASE hive DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY 'scm@123';
CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY 'scm@123';
CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY ' amon@123';
CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY 'rman@123';
CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY 'hue@123';
CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON metastore.* TO 'metastore'@'%' IDENTIFIED BY 'metastore@123';
CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY 'sentry@123';
CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY 'nav@123';
CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY 'navms@123';
CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY 'oozie@123';
4.初始化cdh的數據庫：
/opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm root 123456
5.上傳大數據服務組件資源包(CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel、manifest.json)到/opt/cloudera/parcel-repo，
創建CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha，
在manifest.json文件中找到CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel對應的hash值，寫入CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha文件
給他們賦權：chmod -R guo+wr CDH-5.11.0-1.cdh5.11.0.p0.34-el6.parcel
chmod -R guo+wr CDH-5.11.0-1.cdh5.11.0.p0.34-el6.parcel.sha
chmod -R guo+wr manifest.json
6.在主節點啓動server：
systemctl start cloudera-scm-server
7.在所有節點安裝agent：
1.yum install -y bind-utils cyrus-sasl-plain cyrus-sasl-gssapi portmap /lib/lsb/init-functions httpd mod_ssl openssl-devel python-psycopg2 MySQL-python
yum install -y bind-utils libxslt cyrus-sasl-plain cyrus-sasl-gssapi fuse portmap fuse-libs httpd mod_ssl openssl-devel python-psycopg2 MySQL-python
2.rpm -ivh cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm
rpm -ivh cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm
8.修改agent所有的節點配置文件config.ini內容
vi /etc/cloudera-scm-agent/config.ini：
server_host=master （server節點）
9.所有節點啓動agent：
systemctl start cloudera-scm-agent
10.進入http://master:7180進行集羣和服務的安裝
---------------------------------------------------------------------------------------------------------------------
一.異常處理
---------------------------------------------------------------------------------------------------------------------
問題：啓動server失敗
[root@iZ8vbi6snaln13lk0b9ybiZ cloudera]# systemctl status cloudera-scm-server
a—? cloudera-scm-server.service - Cloudera CM Server Service
Loaded: loaded (/usr/lib/systemd/system/cloudera-scm-server.service; enabled; vendor preset: disabled)
Active: failed (Result: start-limit) since Thu 2020-04-09 13:55:48 CST; 5s ago
Process: 2900 ExecStart=/opt/cloudera/cm/bin/cm-server (code=exited, status=1/FAILURE)
Process: 2897 ExecStartPre=/opt/cloudera/cm/bin/cm-server-pre (code=exited, status=0/SUCCESS)
Main PID: 2900 (code=exited, status=1/FAILURE)
Apr 09 13:55:48 iZ8vbi6snaln13lk0b9ybiZ systemd[1]: cloudera-scm-server.service: main process exited, code=exited, status=1/FAILURE
Apr 09 13:55:48 iZ8vbi6snaln13lk0b9ybiZ systemd[1]: Unit cloudera-scm-server.service entered failed state.
Apr 09 13:55:48 iZ8vbi6snaln13lk0b9ybiZ systemd[1]: cloudera-scm-server.service failed.
Apr 09 13:55:48 iZ8vbi6snaln13lk0b9ybiZ systemd[1]: cloudera-scm-server.service holdoff time over, scheduling restart.
Apr 09 13:55:48 iZ8vbi6snaln13lk0b9ybiZ systemd[1]: Stopped Cloudera CM Server Service.
Apr 09 13:55:48 iZ8vbi6snaln13lk0b9ybiZ systemd[1]: start request repeated too quickly for cloudera-scm-server.service
Apr 09 13:55:48 iZ8vbi6snaln13lk0b9ybiZ systemd[1]: Failed to start Cloudera CM Server Service.
Apr 09 13:55:48 iZ8vbi6snaln13lk0b9ybiZ systemd[1]: Unit cloudera-scm-server.service entered failed state.
Apr 09 13:55:48 iZ8vbi6snaln13lk0b9ybiZ systemd[1]: cloudera-scm-server.service failed.
解決辦法：
[root@master cloudera-scm-server]# mkdir -p /usr/java
[root@master cloudera-scm-server]# ln -s /app/jdk1.8.0_202 /usr/java/default
1 異常1
已啓用“透明大頁面”，它可能會導致重大的性能問題。版本爲“CentOS release 6.3 (Final)”且版本爲“2.6.32-279.el6.x86_64”的 Kernel 已將 enabled 設置爲“[always] never”，並將 defrag 設置爲“[always] never”。請運行“echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag”以禁用此設置，然後將同一命令添加到一個 init 腳本中，如 /etc/rc.local，這樣當系統重啓時就會設置它
（每個節點）
解決方案
echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never >> /etc/rc.local
------------------------------------------
echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled
並將上面的兩條命令寫入開機自啓動
vim /etc/rc.local
--------------------------------------------------------------
2 異常2
Cloudera 建議將 /proc/sys/vm/swappiness 設置爲 0。當前設置爲 60。使用 sysctl 命令在運行時更改該設置並編輯 /etc/sysctl.conf 以在重啓後保存該設置。您可以繼續進行安裝，但可能會遇到問題，Cloudera Manager 報告您的主機由於交換運行狀況不佳。以下主機受到影響：
解決方案：
echo 0 > /proc/sys/vm/swappiness
sysctl vm/swappiness
echo 'vm.swappiness = 0'>/etc/sysctl.conf
sysctl -p
3 異常3
pstree: 未找到命令
解決方案
yum install -y psmisc
4，找不到jdk
輸入：sudo vi /etc/sudoers ，在sudoers最後面添加以下內容
Defaults env_keep+=JAVA_HOME
輸入：sudo vi /etc/default/bigtop-utils 在 bigtop-utils 文件中添加以下內容
export JAVA_HOME=/usr/java/latest
然後執行以下命令
source /etc/default/bigtop-utilssource /etc/default/bigtop-utils
5，異常
如果出現agent主機不良狀況
Error, CM server guid updated, expected 85587073-270d-43d9-a44a-e213d9f7e45b, received 4c1402a5-8364-4598-a382-0c760710e897
，嘗試運行以下命令
rm -rf /var/lib/cloudera-scm-agent/cm_guid
之後重啓agent
service cloudera-scm-agent restart
異常
[root@master cloudera-scm-server]# service cloudera-scm-server start
Starting cloudera-scm-server: [失敗]
[root@master cloudera-scm-server]# more cloudera-scm-server.out
+======================================================================+
| Error: JAVA_HOME is not set and Java could not be found |
+----------------------------------------------------------------------+
| Please download the latest Oracle JDK from the Oracle Java web site |
| > http://www.oracle.com/technetwork/java/javase/index.html < |
| |
| Cloudera Manager requires Java 1.6 or later. |
| NOTE: This script will find Oracle Java whether you install using |
| the binary or the RPM based installer. |
+======================================================================+
[root@master cloudera-scm-server]#
[root@master cloudera-scm-server]# mkdir -p /usr/java
[root@master cloudera-scm-server]# ln -s /app/jdk1.8.0_202 /usr/java/default
[root@master cloudera-scm-server]# service cloudera-scm-server start
Starting cloudera-scm-server: [確定]
[root@master cloudera-scm-server]#
把jdk換成兼容的11版本就好了
Dec 03 10:51:57 bogon cm-server[3060]: +======================================================================+
Dec 03 10:51:57 bogon cm-server[3060]: | Error: Unable to find a compatible version of Java on this host,|
Dec 03 10:51:57 bogon cm-server[3060]: | either because JAVA_HOME has not been set or because a |
Dec 03 10:51:57 bogon cm-server[3060]: | compatible version of Java is not installed. |
Dec 03 10:51:57 bogon cm-server[3060]: +----------------------------------------------------------------------+
Dec 03 10:51:57 bogon cm-server[3060]: | Please install either: |
Dec 03 10:51:57 bogon cm-server[3060]: | - a supported version of the Oracle JDK from the Oracle Java web |
Dec 03 10:51:57 bogon cm-server[3060]: | site: |
Dec 03 10:51:57 bogon cm-server[3060]: | > http://www.oracle.com/technetwork/java/javase/index.html < |
Dec 03 10:51:57 bogon cm-server[3060]: | OR |
Dec 03 10:51:57 bogon cm-server[3060]: | - a supported version of the OpenJDK from your OS vendor. Help for |
Dec 03 10:51:57 bogon cm-server[3060]: | some OSes are available at: |
Dec 03 10:51:57 bogon cm-server[3060]: | > http://openjdk.java.net/install/ < |
Dec 03 10:51:57 bogon cm-server[3060]: | |
Dec 03 10:51:57 bogon cm-server[3060]: | Cloudera Manager requires Oracle JDK or OpenJDK 1.8 or later. |
Dec 03 10:51:57 bogon cm-server[3060]: | NOTE: Cloudera Manager will find the Oracle JDK when starting, |
Dec 03 10:51:57 bogon cm-server[3060]: | regardless of whether you installed the JDK using a binary |
Dec 03 10:51:57 bogon cm-server[3060]: | installer or the RPM-based installer. |
Dec 03 10:51:57 bogon cm-server[3060]: +======================================================================+
-bash: scp: command not found
執行命令
yum install openssh-clients
查看mysql的狀態：
/usr/local/mysql/support-files/mysql.server status
安裝Kafka服務
但是有提示:Before adding this service, ensure that either the Kafka parcel is activated or the Kafka package is installed. 這樣子直接添加kafka的服務是不正確的
1.在Cloudera Manager頁面找到主機->Parcel點擊進入
2.在頁面的Parcel列找到KAFKA並點擊下載,下載完成後點擊 分配 進行分配,然後點擊 激活, 出現 已分配,已激活 證明分配激活成功
回到Cloudera Manager頁通過添加服務添加kafka服務,但是在啓動kafka服務的過程中又報錯了:
報的錯誤是內存溢出錯誤,默認的是50M,
內存溢出錯誤問題解決:回到Cloudera Manager頁面進入Kafka管理界面,點擊配置項,然後在其配置欄中點擊 kafka broker 找到Heap Size of Broker這一欄,改成1G就可以了,然後保存更改重啓該服務
cdh集羣卸載:
查看安裝了那些包
rpm -qa | grep cloudera
依次卸載安裝包
rpm -e cloudera-manager-server-db-2-5.15.1-1.cm5151.p0.3.el6.x86_64
rpm -e cloudera-manager-server-5.15.1-1.cm5151.p0.3.el6.x86_64
rpm -e cloudera-manager-agent-5.15.1-1.cm5151.p0.3.el6.x86_64
rpm -e cloudera-manager-daemons-5.15.1-1.cm5151.p0.3.el6.x86_64
rm -rf /usr/share/cmf /var/lib/cloudera* /var/cache/yum/x86_64/6/cloudera* /var/log/cloudera* /var/run/cloudera*
rm -rf /etc/cloudera*
rm -rf /opt/cloudera/parcel-cache /opt/cloudera/parcels
rm: 無法刪除"/var/run/cloudera-scm-agent/process": 設備或資源忙
這個問題需要重啓各個節點
然後再執行rm -rf 命令
重啓後，登陸mysql ,刪除scm庫，重新執行初始化數據庫命令。
agent節點啓動過後立馬死掉
[root@node2 cloudera-scm-agent]# netstat -tupnl |grep 19001
tcp 0 0 127.0.0.1:19001 0.0.0.0:* LISTEN 2857/python
[root@node2 cloudera-scm-agent]# kill -9 2857
檢查ssh 免密碼登陸到自己，到各個子節點
檢查ntp服務
---------------------------------------------------------------------------------------------------------------------