CDH環境搭建

1.CDH安裝前配置

1.1   HOSTS配置

      1.1.1  hostname修改

修改配置文件/etc/sysconfig/network

root@hadoop01~# vim /etc/sysconfig/network

 NETWORKING=yes

HOSTNAME=hadoop01

           

修改完成後需要重啓服務器使其生效。修改集羣所有節點的hostname爲統一格式方便集羣管理。

 

 1.1.2   配置/etc/hosts文件

 配置/etc/hosts文件

[root@hadoop01 ~]# vim /etc/hosts

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4

::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

172.23.216.73   hadoop01

172.23.216.74   hadoop02

172.23.216.75   hadoop03                                                                                                                                                   

    

1.2 免密碼登陸配置

SSH免密碼登陸，因爲HADOOP環境中需要機器傳輸文件啥的。 

1.2.1、在五臺機器上均執行命令ssh-keygen -t rsa，遇到提示一路回車，生成公鑰文件 
1.2.2、在master上面執行命令cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys 
1.2.3、在master上面將授權文件分別拷貝到其它四個節點scp ~/.ssh/authorized_keys root@hadoop01:~/.ssh/

此時的你可以在master上任意scp一個文件到其它節點機器，發現不用輸入密碼了，恭喜成功了。

注意這裏也有一種比較簡單的方式配置免密，在各個節點上生成祕鑰之後，可以用ssh-copy-id -i hadoop01,將祕鑰拷貝到各個節點

1.3   禁用SELinux

1.3.1 在集羣所有節點執行如下命令 

 [root@hadoop01 ~]# setenforce 0

1.3.2 修改集羣所有節點配置文件/etc/selinux/config

     [root@hadoop01 ~]# vim /etc/selinux/config

      

1.3.3 驗證是否禁用成功

 [root@hadoop01 ~]# sestatus -v

SELinux status: disabled

      

1.4   關閉防火牆

1.4.1 關閉集羣所有服務器的防火牆

[root@hadoop01 ~]# chkconfig iptables off

 service ipta[root@hadoop01 ~]#bles stop

iptables: Setting chains to policy ACCEPT: filter [ OK ]

iptables: Flushing firewall rules: [ OK ]

iptables: Unloading modules: [ OK ]

1.4.2   檢查所有服務器防火牆是否關閉

[root@hadoop01 ~]# service iptables status

iptables: Firewall is not running.

   

1.5  設置SWAP

1.5.1 在集羣所有節點修改/etc/sysctl.conf配置文件，增加如下配置

  [root@hadoop01 ~]#  vim /etc/sysctl.conf

   

1.5.2  檢查設置是否生效

 [root@hadoop01 ~]# sysctl -p

kernel.msgmnb= 65536

kernel.msgmax= 65536

kernel.shmmax= 68719476736

kernel.shmall= 4294967296

vm.swappiness= 10

  

1.6 關閉透明大頁面

  1.6.1 在集羣所有節點執行如下命令，使其立即生效

 [root@hadoop01 ~]#echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag

[root@hadoop01 ~]# echo never >/sys/kernel/mm/redhat_transparent_hugepage/enabled

1.6.2  在集羣所有節點/etc/rc.d/rc.local腳本中增加如下代碼，使其永久生效

 if test -f/sys/kernel/mm/redhat_transparent_hugepage/enabled; then

echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled

fi

if test -f /sys/kernel/mm/redhat_transparent_hugepage/defrag; then

echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag

fi

1.6.3   賦予rc.local腳本可執行權限

 [root@hadoop01_rc.d# chmod +x /etc/rc.d/rc.local

1.7  配置時鐘同步

 1.7.1  在集羣所有節點安裝NTP服務

[root@hadoop01 ~] yum -y install ntp

 1.7.2   選擇集羣一臺服務器作爲NTP Server，配置如下

 [root@hadoop01 ~] vim /etc/ntp.conf

 server 127.0.0.1

fudge 127.0.0.1 stratum 10

1.7.3  集羣其它節點與其同步，配置如下：

[root@hadoop02 ~]# vim /etc/ntp.conf

server 172.23.216.73

1.7.4  完成以上配置後，在集羣所有節點執行如下命令

[root@hadoop01 ~]#  chkconfig --add ntpd

[root@hadoop01 ~]#  chkconfig ntpd on

[root@hadoop01 ~]#  service ntpd restart

Shutting down ntpd: [ OK ]

Starting ntpd: [ OK ]

1.7.5 .驗證所有節點NTP是否同步

[root@hadoop01 ~]#ntpq -p

remote refid st t when pollreach delay offset jitter

*ip-172-31-6-148210.23.25.77 2 u 448 1024 377 0.255 3.245 7.859 

 

注：上圖標註看到有*號則表示同步成功。

1.8   安裝MySQL數據庫

1.8.1  在集羣中CM節點安裝MySQL服務

 [root@hadoop01 /]# yum -y install mysql mysql-serve

        

1.8.2  啓動MySQL服務並將mysqld加入開機自啓動服務

[root@hadoop01 /]# chkconfig --add mysqld

[root@hadoop01 /]#  chkconfig mysqld on

[root@hadoop01 /]# service mysqld start

Starting mysqld: [ OK ]

1.8.3  執行如下腳本初始化MySQL

 [root@hadoop01 ~]# mysql_secure_installation

NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FORALL MySQL

SERVERS IN PRODUCTION USE! PLEASE READEACH STEP CAREFULLY!

In order to log into MySQL to secure it, we'll needthe current

password for the root user. If you'vejust installed MySQL, and

you haven't set the root password yet, the passwordwill be blank,

so you should just press enter here.

Enter current password for root (enter for none): 

OK, successfully used password, moving on...

Setting the root password ensures that nobody can log into the MySQL

root user without the proper authorisation.

Set root password? Y/n y

New password: 

Re-enter new password: 

Password updated successfully!

Reloading privilege tables..

... Success!

By default, a MySQL installation has an anonymous user, allowing anyone

to log into MySQL without having to have a user account created for

them. This is intended only for testing,and to make the installation

go a bit smoother. You should removethem before moving into a

production environment.

Remove anonymous users? Y/n y

... Success!

Normally, root should only be allowed to connect from 'localhost'. This

ensures that someone cannot guess at the root password from the network.

Disallow root login remotely? Y/n n

... skipping.

By default, MySQL comes with a database named 'test' that anyone can

access. This is also intended only fortesting, and should be removed

before moving into a production environment.

Remove test database and access to it? Y/n y

- Dropping test database...

... Success!

- Removing privileges on testdatabase...

... Success!

Reloading the privilege tables will ensure that all changes made so far

will take effect immediately.

Reload privilege tables now? Y/n y

... Success!

Cleaning up...

All done! If you've completed all of the above steps, your MySQL

installationshould now be secure.

Thanks for usingMySQL!

1.8.4  登錄MySQL驗證是否安裝成功

[root@hadoop01 ~]#  mysql -uroot -p123456

Welcome to the MySQL monitor. Commands end with ; or \g.

Your MySQL connection id is 9

Server version: 5.1.73 Source distribution

Copyright (c) 2000, 2013,Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporationand/or its

affiliates.Other names may be trademarks of theirrespective

owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current inputstatement.

mysql>

     

1.8.5 創建CDH所需要的數據庫

====================hive=============

    create database metastore default character set utf8;

CREATE USER 'hive'@'%'IDENTIFIED BY '123456';

GRANT ALL PRIVILEGES ON metastore.* TO 'hive'@'%';

FLUSH PRIVILEGES;

==================hive==========

create database hive default character set utf8;

GRANT ALL PRIVILEGES ON hive.* TO 'hive'@'%';

FLUSH PRIVILEGES;

==================cm==========

create database cm default character set utf8;

CREATE USER 'cm'@'%'IDENTIFIED BY '123456';

GRANT ALL PRIVILEGES ON cm.* TO 'cm'@'%';

FLUSH PRIVILEGES;

============================xxx============

create database am default character set utf8;

CREATE USER 'am'@'%'IDENTIFIED BY '123456';

GRANT ALL PRIVILEGES ON am.* TO 'am'@'%';

FLUSH PRIVILEGES;

=======================xxx=====

create database rm default character set utf8;

CREATE USER 'rm'@'%'IDENTIFIED BY '123456';

GRANT ALL PRIVILEGES ON rm.* TO 'rm'@'%';

FLUSH PRIVILEGES;

=======hue====

create database hue default character set utf8;

CREATE USER 'hue'@'%'IDENTIFIED BY '123456';

GRANT ALL PRIVILEGES ON hue.* TO 'hue'@'%';

FLUSH PRIVILEGES;

======oozie=====

create database oozie default character set utf8;

CREATE USER 'oozie'@'%' IDENTIFIED BY '123456';

GRANT ALL PRIVILEGES ON oozie.* TO 'oozie'@'%';

FLUSH PRIVILEGES;

========sentry====

create database sentry default character set utf8;

CREATE USER 'sentry'@'%' IDENTIFIED BY '123456';

GRANT ALL PRIVILEGES ON sentry.* TO 'sentry'@'%';

FLUSH PRIVILEGES;

1.9  安裝jdk

1.9.1 卸載OpenJDK  

           rpm -qa|grep java

           yum remove java*

1.9.2安裝 jdk1.8  

這裏自行百度

2. 正式開始安裝CDH

2.1  下載

2.1.1 下載cloudera-manager

 Cloudera Manager下載地址：
http://archive.cloudera.com/cm5/cm/5/cloudera-manager-el6-cm5.1.3_x86_64.tar.gz

部署CDH的Parcel

2.1.2 下載CDH的Parcel文件

下載地址：http://archive.cloudera.com/cdh5/parcels/5.13.0/

CDH-5.13.0-1.cdh5.13.0.p0.29-el6.parcel

CDH-5.13.0-1.cdh5.13.0.p0.29-el6.parcel.sha1

manifest.json      

2.2 將下載的 Cloudera Manager放到指定位置

2.2.1 將下載的 Cloudera Manager文件放到指定位置

 將 cloudera-manager-el6-cm5.13.0_x86_64.tar.gz 傳輸到所有節點的/opt目錄下並解壓到當前目錄，

[root@hadoop01 opt]#  tar -zxvf  cloudera-manager-el6-cm5.13.0_x86_64.tar.gz

2.2.2.Agent配置

修改所有節點中/opt/cm-5.13.0/etc/cloudera-scm-agent/config.ini中的server_host爲主節點的主機名

 [root@hadoop01 ~]#vim /opt/cm-5.15.0/etc/cloudera-scm-agent/config.ini

server_host=hadoop01

2..3 創建用戶

2.3.1 在所有節點創建cloudera-scm用戶 

[root@hadoop01 ~]# useradd --system --home=/opt/cm-5.13.0/run/cloudera-scm-server/ --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm

修改權限

 [root@hadoop01 opt]# chown -R cloudera-scm:cloudera-scm cloudera

 [root@hadoop02 opt]# chown -R cloudera-scm:cloudera-scm cm-5.13.0    

2.4 準備Parcels，用以安裝CDH5

   2.4.1將CHD5相關的Parcel包放到主節點的/opt/cloudera/parcel-repo/目錄中

    相關的文件如下：

     CDH-5.13.0-1.cdh5.13.0.p0.29-el6.parcel

CDH-5.13.0-1.cdh5.13.0.p0.29-el6.parcel.sha1

manifest.json

2.4.2 修改 CDH-5.13.0-1.cdh5.13.0.p0.29-el6.parcel.sha1命名

將CDH-5.13.0-1.cdh5.13.0.p0.29-el6.parcel.sha1，重命名爲CDH-5.13.0-1.cdh5.13.0.p0.29-el6.parcel.sha,這點必須注意，否則，系統會重新下載CDH-5.1.3-1.cdh5.1.3.p0.12-el6.parcel文件

2.5 爲Cloudera Manager 5建立數據庫

2.5.1安裝mysql驅動包

首先需要去MySql的官網下載JDBC驅動，http://dev.mysql.com/downloads/connector/j/，解壓後，找到mysql-connector-java-5.1.33-bin.jar，放到/usr/share/java

2.5.2在主節點初始化CM5的數據庫：

 [root@hadoop01 ~]# /opt/cm-5.15.0/share/cmf/schema/scm_prepare_database.sh mysql cm -hlocalhost -uroot -p123456 --scm-host localhost scm scm 123456

2.5.3啓動

在主節點啓動cloudera-scm-server：

[root@hadoop01 ~]#/opt/cm-5.15.0/etc/init.d/cloudera-scm-server start

在所有節點啓動cloudera-scm-agent：

[root@hadoop01 ~]#/opt/cm-5.15.0/etc/init.d/cloudera-scm-agent start

檢查端口是否監聽：

[root@hadoop01 ~]# netstat -apn |grep 7180

      

3.集羣安裝

3.1  訪問CM

通過http:// 172.23.216.73:7180/cmf/login訪問CM

 

用戶名：admin   密碼：admin

3.2  爲cdh 集羣指定安裝主機

3.3. 設置CDH的Parcel地址

 

 

3.4向集羣所有主機安裝並激活Parcel

 

3.5 檢查主機正確性

 

3.6設置集羣安裝的服務組合

3.7自定義角色分配

3.8配置審覈更改

   

 3.9 集羣首次運行

  

3.10集羣安裝成功

   

補充：

  搭建中曾碰到的問題：

1. cm註冊到mysql的庫和hive註冊到MySQL的庫不可以是同一個
2. 拷貝parcel包到parcel-repo目錄下時，manifest.json文件不要忘了拷貝，安裝其他應用時，其他組件的parcel包如果也帶有manifest.json，記得要將manifest.json合併。
3.  Hive的驅動包要放在/usr/share/java
4. opt目錄下相關文件的權限不要忘了修改
5. spark安裝時，jdk的目錄需要放在/usr/java下
6. phonex安裝需要重新編譯
7. 刪除cdh集羣前需要將集羣中的組件全部停止
8. zookeeper安裝時要選最好是奇數個節點，且一般五個節點就夠了
9. 給組件分配角色的時候最好將Cloudera Manager的監控角色和大數據角色分開
10. spark和hive最好放一個節點，metastore不然無法同步

費勁了千辛萬苦終於把公司的CDH大數據集羣裝好了，且能正常運行