0024-CENTOS6.5安裝CDH5.12.1(一)

溫馨提示:要看高清無碼套圖,請使用手機打開並單擊圖片放大查看。

1.概述

本文檔主要描述離線環境下,CentOS6.5操作系統部署CDH5.12.1企業版的過程。本安裝文檔主要分爲4個步驟:

1.前期準備(包括hostname、關閉防火牆、關閉SELinux、時鐘同步等)

2.安裝Cloudera Manager Server

3.安裝CDH集羣

4.集羣完整性檢查(包括HDFS文件系統、MapReduce、Hive等服務是否可以正常運行)

這篇文檔將着重介紹Cloudera管理器與CDH的安裝,並基於以下假設:

1.操作系統版本:CentOS6.5

2.MySQL版本:5.1.73

3.CM版本:CM 5.12.1

4.CDH版本:CDH 5.12.1

5.採用root對集羣進行部署

6.您已經下載CDH和CM的安裝包(CDH使用parcels包方式安裝)

2.前期準備

2.1hostname及hosts配置

集羣中各個節點之間能互相通信使用靜態IP地址。IP地址和主機名通過/etc/hosts配置,主機名/etc/hostname進行配置(推薦hostname使用FQDN完全域名的方式配置)。

以服務器(172.31.6.148)爲例:

  • hostname配置

修改/etc/sysconfig/network文件:

[root@ip-172-31-6-148~]# vim /etc/sysconfig/network

NETWORKING=yes

NETWORKING_IPV6=no

HOSTNAME=ip-172-31-6-148.fayson.com

0024-CENTOS6.5安裝CDH5.12.1(一)

注意:修改hostname後需要重啓服務器。

檢查hostname是否修改成功:

[root@ip-172-31-6-148~]# hostname

ip-172-31-6-148.fayson.com

[root@ip-172-31-6-148 ~]#

0024-CENTOS6.5安裝CDH5.12.1(一)

  • hosts配置

/etc/hosts文件如下:

[root@ip-172-31-6-148~]# more /etc/hosts

127.0.0.1 localhost.localdomain localhost

::1 localhost6.localdomain6 localhost6

172.31.6.148 ip-172-31-6-148.fayson.com

172.31.5.190 ip-172-31-5-190.fayson.com

172.31.10.118 ip-172-31-10-118.fayson.com

172.31.9.33 ip-172-31-9-33.fayson.com

[root@ip-172-31-6-148 ~]#

0024-CENTOS6.5安裝CDH5.12.1(一)

以上兩步操作,在集羣中其它節點做相應配置。

2.2禁用SELinux

在所有節點執行sudo setenforce 0 命令

[root@ip-172-31-6-148~]# sudo setenforce 0

修改集羣所有節點的/etc/selinux/config文件,內容如下:

[root@ip-172-31-6-148~]# vim /etc/selinux/config

This file controls the state ofSELinux on the system.

SELINUX= can take one of these three values:

enforcing - SELinux security policyis enforced.

permissive - SELinux printswarnings instead of enforcing.

disabled - No SELinux policy isloaded.

SELINUX=disabled

SELINUXTYPE= can take one of these twovalues:

targeted - Targeted processes areprotected,

mls - Multi Level Securityprotection.

SELINUXTYPE=targeted

0024-CENTOS6.5安裝CDH5.12.1(一)

2.3關閉防火牆

在集羣所有節點執行如下操作,並永久關閉防火牆

[root@ip-172-31-6-148~]# service iptables stop

iptables: Setting chains to policy ACCEPT: filter [ OK ]

iptables: Flushing firewall rules: [ OK ]

iptables: Unloading modules: [ OK ]

[root@ip-172-31-6-148 ~]# chkconfig iptables off

[root@ip-172-31-6-148 ~]#

0024-CENTOS6.5安裝CDH5.12.1(一)

2.4配置操作系統本地yum源

  • 掛載操作系統iso文件

[root@ip-172-31-6-148~]# mkdir /mnt/iso

[root@ip-172-31-6-148 ~]# mount -o loop CentOS-6.5-x86_64-minimal.iso/mnt/iso/

[root@ip-172-31-6-148 ~]# df -h

Filesystem Size UsedAvail Use% Mounted on

/dev/xvde 99G 3.4G 91G 4% /

tmpfs 7.4G 0 7.4G 0%/dev/shm

/root/CentOS-6.5-x86_64-minimal.iso 398M 398M 0 100% /mnt/iso

[root@ip-172-31-6-148 ~]#

0024-CENTOS6.5安裝CDH5.12.1(一)

  • 配置操作系統repo

[root@ip-172-31-6-148~]# vim /etc/yum.repos.d/os.repo

localrepo

name=CentOS6.5

baseurl=file:///mnt/iso

gpgcheck=false

enabled=true

0024-CENTOS6.5安裝CDH5.12.1(一)

查看yum源列表是否有新建的localrepo

[root@ip-172-31-6-148~]# yum repolist

Loaded plugins: fastestmirror, presto

Loading mirror speeds from cached hostfile

* base: mirror.0x.sg

* extras: mirror.0x.sg

* updates: mirror.0x.sg

localrepo | 3.6 kB 00:00...

localrepo/primary_db | 492 kB 00:00 ...

repo id reponame status

base CentOS-6 - Base 6,706

cloudera-manager Cloudera Manager, Version 5.12.0 7

cmrepo cm_repo 7

extras CentOS-6 - Extras 45

localrepo CentOS6.5 248

updates CentOS-6 - Updates 589

repolist: 7,602

[root@ip-172-31-6-148 ~]#

0024-CENTOS6.5安裝CDH5.12.1(一)

2.5安裝http服務

  • 使用yum安裝http服務

在集羣其中一節點上安裝http服務,執行如下命令:

[root@ip-172-31-6-148~]# yum -y install httpd

0024-CENTOS6.5安裝CDH5.12.1(一)

如上圖所示則表明安裝成功。

  • 將httpd服務加入系統自啓動服務並設置開機啓動

[root@ip-172-31-6-148~]# chkconfig --add httpd

[root@ip-172-31-6-148 ~]# chkconfig httpd on

[root@ip-172-31-6-148 ~]# chkconfig --list |grep httpd

httpd 0:off 1:off 2:on 3:on 4:on 5:on 6:off

[root@ip-172-31-6-148 ~]#

0024-CENTOS6.5安裝CDH5.12.1(一)

  • 啓動http服務

[root@ip-172-31-6-148~]# service httpd start

Starting httpd: [ OK ]

0024-CENTOS6.5安裝CDH5.12.1(一)

2.6重做操作系統yum源

在2.5安裝了httpd服務後,重新制作操作系統的yum源,使用http的方式,操作如下:

  • 在/var/www/html目錄下創建centos6.5目錄

_root@ip-172-31-6-148_html# mkdir centos6.5

[root@ip-172-31-6-148 html]# ll

total 4

drwxr-xr-x 2 rootroot 4096 Sep 4 16:04 centos6.5

[root@ip-172-31-6-148 html]#

0024-CENTOS6.5安裝CDH5.12.1(一)

  • 將/mnt/iso目錄下的所有文件拷貝至/var/www/html/centos6.5目錄下

_root@ip-172-31-6-148_html# scp -r /mnt/iso/*/var/www/html/centos6.5/

0024-CENTOS6.5安裝CDH5.12.1(一)

  • 修改/etc/yum.repo.d/os.repo配置

[root@ip-172-31-6-148~]# vim /etc/yum.repos.d/os.repo

localrepo

name=CentOS6.5

baseurl=http://ip-172-31-6-148.fayson.com/centos6.5

gpgcheck=false

enabled=true

0024-CENTOS6.5安裝CDH5.12.1(一)

  • 檢查yum源是否配置正常

0024-CENTOS6.5安裝CDH5.12.1(一)

將/etc/yum.repo.d/os.repo配置文件同步到集羣所有節點的/etc/yum.repo.d/目錄下

2.7集羣時鐘同步

在集羣的所有服務器上安裝ntp服務,用於集羣時鐘同步,此處選擇ip-172-31-6-148.fayson.com服務器作爲本地NTP服務器,集羣其它節點與其保持同步,配置如下:

  • 在集羣所有節點安裝ntp服務,命令如下

[root@ip-172-31-6-148~]# yum -y install ntp

0024-CENTOS6.5安裝CDH5.12.1(一)

如上圖顯示,則說明安裝成功。

  • 將ntpd加入系統自啓動服務並設置開機啓動

在所有節點執行如下操作:

[root@ip-172-31-6-148~]# chkconfig --add ntpd

[root@ip-172-31-6-148 ~]# chkconfig ntpd on

0024-CENTOS6.5安裝CDH5.12.1(一)

  • 在ip-172-31-6-148.fayson.com上配置與自己同步

[root@ip-172-31-6-148~]# vim /etc/ntp.conf

...

Use public servers from thepool.ntp.org project.

Please consider joining the pool (http://www.pool.ntp.org/join.html).

#server 0.centos.pool.ntp.org iburst

#server 1.centos.pool.ntp.org iburst

#server 2.centos.pool.ntp.org iburst

#server 3.centos.pool.ntp.org iburst

server 127.127.1.0 #local clock

fudge 127.127.1.0 stratum 10

0024-CENTOS6.5安裝CDH5.12.1(一)

  • 集羣其它節點配置如下

[root@ip-172-31-5-190~]# vim /etc/ntp.conf

...

Use public servers from thepool.ntp.org project.

Please consider joining the pool (http://www.pool.ntp.org/join.html).

#server 0.centos.pool.ntp.org iburst

#server 1.centos.pool.ntp.org iburst

#server 2.centos.pool.ntp.org iburst

#server 3.centos.pool.ntp.org iburst

server 172.31.6.148 iburst

0024-CENTOS6.5安裝CDH5.12.1(一)

  • 重啓集羣所有節點的ntpd服務

[root@ip-172-31-6-148~]# service ntpd restart

Shutting down ntpd: [ OK ]

Starting ntpd: [ OK ]

[root@ip-172-31-6-148 ~]#

0024-CENTOS6.5安裝CDH5.12.1(一)

  • 驗證時鐘同步,在所有節點執行ntpq -p命令

[root@ip-172-31-6-148~]# ntpq -p

remote refid stt when poll reach delay offset jitter

==============================================================================

*LOCAL(0) .LOCL. 10 l 15 64 7 0.000 0.000 0.001

[root@ip-172-31-6-148 ~]# ssh -i fayson.pem.txtip-172-31-5-190.fayson.com "ntpq-p"

remote refid st t when pollreach delay offset jitter

==============================================================================

*ip-172-31-6-148 LOCAL(0) 11 u 68 128 377 0.274 1.005 0.243

[root@ip-172-31-6-148 ~]# ssh -i fayson.pem.txtip-172-31-10-118.fayson.com "ntpq-p"

remote refid stt when poll reach delay offset jitter

==============================================================================

*ip-172-31-6-148 LOCAL(0) 11 u 75 128 377 0.285 1.214 0.213

[root@ip-172-31-6-148 ~]# ssh -i fayson.pem.txt ip-172-31-9-33.fayson.com"ntpq -p"

remote refid stt when poll reach delay offset jitter

==============================================================================

*ip-172-31-6-148 LOCAL(0) 11 u 1 64 367 0.294 -9.249 1.511

[root@ip-172-31-6-148 ~]#

0024-CENTOS6.5安裝CDH5.12.1(一)

注:標記部分,有“*”顯示則表示同步成功。

2.8安裝MySQL

  • 安裝MySQL

[root@ip-172-31-6-148~]# yum -y install mysql mysql-server

0024-CENTOS6.5安裝CDH5.12.1(一)

如上截圖則表示MySQL安裝成功。

  • 將mysqld加入系統自啓動服務並設置開機啓動

[root@ip-172-31-6-148~]# chkconfig --add mysqld

[root@ip-172-31-6-148 ~]# chkconfig mysqld on

  • 啓動並配置Mysql

[root@ip-172-31-6-148~]# service mysqld start

...

[ OK ]

Starting mysqld: [ OK ]

0024-CENTOS6.5安裝CDH5.12.1(一)

  • 初始化MySQL

[root@ip-172-31-6-148~]# mysql_secure_installation

NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MySQL

SERVERS IN PRODUCTIONUSE! PLEASE READ EACH STEP CAREFULLY!

In order to log into MySQL to secure it, we'll need the current

password for the root user. If you've just installed MySQL, and

you haven't set the root password yet, the passwordwill be blank,

so you should just press enter here.

Enter current password for root (enter for none):

OK, successfully used password, moving on...

Setting the root password ensures that nobody can log into the MySQL

root user without the proper authorisation.

Set root password? Y/n y

New password:

Re-enter new password:

Password updated successfully!

Reloading privilege tables..

... Success!

By default, a MySQL installation has an anonymous user, allowing anyone

to log into MySQL without having to have a user account created for

them. This is intended only for testing,and to make the installation

go a bit smoother. You should removethem before moving into a

production environment.

Remove anonymous users? Y/n y

... Success!

Normally, root should only be allowed to connect from 'localhost'. This

ensures that someone cannot guess at the root password from the network.

Disallow root login remotely? Y/n n

... skipping.

By default, MySQL comes with a database named 'test' that anyone can

access. This is also intended only fortesting, and should be removed

before moving into a production environment.

Remove test database and access to it? Y/n y

  • Dropping test database...
    ... Success!
  • Removing privileges on test database...
    ... Success!

Reloading the privilege tables will ensure that all changes made so far

will take effect immediately.

Reload privilege tables now? Y/n y

... Success!

Cleaning up...

All done! If you've completed all of the above steps, your MySQL

installation should now be secure.

Thanks for usingMySQL!

[root@ip-172-31-6-148 ~]#

注意:標紅部分爲輸入項,出現“Thanks for using MySQL!”則表示初始化成功。

  • 創建CM及CDH服務的數據庫

建庫語句:

create database metastore default character set utf8;

CREATE USER 'hive'@'%' IDENTIFIED BY 'password';

GRANT ALL PRIVILEGES ON metastore. * TO 'hive'@'%';

FLUSH PRIVILEGES;

create database cm default character set utf8;

CREATE USER 'cm'@'%' IDENTIFIED BY 'password';

GRANT ALL PRIVILEGES ON cm. * TO 'cm'@'%';

FLUSH PRIVILEGES;

create database am default character set utf8;

CREATE USER 'am'@'%' IDENTIFIED BY 'password';

GRANT ALL PRIVILEGES ON am. * TO 'am'@'%';

FLUSH PRIVILEGES;

create database rm default character set utf8;

CREATE USER 'rm'@'%' IDENTIFIED BY 'password';

GRANT ALL PRIVILEGES ON rm. * TO 'rm'@'%';

FLUSH PRIVILEGES;

create database hue default character set utf8;

CREATE USER 'hue'@'%' IDENTIFIED BY 'password';

GRANT ALL PRIVILEGES ON hue. * TO 'hue'@'%';

FLUSH PRIVILEGES;

create database oozie default character set utf8;

CREATE USER 'oozie'@'%' IDENTIFIED BY 'password';

GRANT ALL PRIVILEGES ON oozie. * TO 'oozie'@'%';

FLUSH PRIVILEGES;

命令行操作如下:

[root@ip-172-31-6-148~]# mysql -uroot -p

Enter password:

...

mysql> create database metastore default character set utf8;

Query OK, 1 row affected (0.00 sec)

mysql> CREATE USER 'hive'@'%' IDENTIFIEDBY 'password';

Query OK, 0 rows affected (0.00 sec)

mysql> GRANT ALL PRIVILEGES ON metastore. * TO 'hive'@'%';

Query OK, 0 rows affected (0.00 sec)

mysql> FLUSH PRIVILEGES;

Query OK, 0 rows affected (0.00 sec)

mysql>

mysql> create databasecm default character set utf8;

Query OK, 1 row affected (0.00 sec)

mysql> CREATE USER 'cm'@'%' IDENTIFIEDBY 'password';

Query OK, 0 rows affected (0.00 sec)

mysql> GRANT ALL PRIVILEGES ON cm. * TO 'cm'@'%';

Query OK, 0 rows affected (0.00 sec)

mysql> FLUSH PRIVILEGES;

Query OK, 0 rows affected (0.00 sec)

mysql_> create database_am default character set utf8;

Query OK, 1 row affected (0.00 sec)

mysql> CREATE USER 'am'@'%' IDENTIFIEDBY 'password';

Query OK, 0 rows affected (0.00 sec)

mysql> GRANT ALL PRIVILEGES ON am. * TO 'am'@'%';

Query OK, 0 rows affected (0.00 sec)

mysql> FLUSH PRIVILEGES;

Query OK, 0 rows affected (0.00 sec)

mysql>

mysql> create databaserm default character set utf8;

Query OK, 1 row affected (0.00 sec)

mysql> CREATE USER 'rm'@'%' IDENTIFIEDBY 'password';

Query OK, 0 rows affected (0.00 sec)

mysql> GRANT ALL PRIVILEGES ON rm. * TO 'rm'@'%';

Query OK, 0 rows affected (0.00 sec)

mysql> FLUSH PRIVILEGES;

Query OK, 0 rows affected (0.00 sec)

mysql>

mysql> create databasehue default character set utf8;

Query OK, 1 row affected (0.00 sec)

mysql> CREATE USER 'hue'@'%' IDENTIFIEDBY 'password';

Query OK, 0 rows affected (0.00 sec)

mysql> GRANT ALL PRIVILEGES ON hue. * TO 'hue'@'%';

Query OK, 0 rows affected (0.00 sec)

mysql> FLUSH PRIVILEGES;

Query OK, 0 rows affected (0.00 sec)

mysql>

mysql> create databaseoozie default character set utf8;

Query OK, 1 row affected (0.00 sec)

mysql> CREATE USER 'oozie'@'%' IDENTIFIEDBY 'password';

Query OK, 0 rows affected (0.00 sec)

mysql> GRANT ALL PRIVILEGES ON oozie. * TO 'oozie'@'%';

Query OK, 0 rows affected (0.00 sec)

mysql> FLUSH PRIVILEGES;

Query OK, 0 rows affected (0.00 sec)

mysql>

0024-CENTOS6.5安裝CDH5.12.1(一)

  • 安裝jdbc驅動

[root@ip-172-31-6-148~]# mkdir -p /usr/share/java

[root@ip-172-31-6-148 ~]# mv mysql-connector-java-5.1.34.jar/usr/share/java/

[root@ip-172-31-6-148 ~]# cd /usr/share/java/

[root@ip-172-31-6-148 java]# ln -s mysql-connector-java-5.1.34.jar mysql-connector-java.jar

[root@ip-172-31-6-148 java]# ll

total 940

-rwxr-xr-x. 1 root root 960372 Aug 30 15:16 mysql-connector-java-5.1.34.jar

lrwxrwxrwx 1root root 31 Sep 4 16:32 mysql-connector-java.jar -> mysql-connector-java-5.1.34.jar

[root@ip-172-31-6-148 java]#

0024-CENTOS6.5安裝CDH5.12.1(一)

3.Cloudera Manager安裝

3.1配置CM本地repo源

CM5.12.1下載地址:

http://archive.cloudera.com/cm5/redhat/6/x86\_64/cm/5.12.1/RPMS/x86\_64/cloudera-manager-agent-5.12.1-1.cm5121.p0.6.el6.x86\_64.rpm

http://archive.cloudera.com/cm5/redhat/6/x86\_64/cm/5.12.1/RPMS/x86\_64/cloudera-manager-daemons-5.12.1-1.cm5121.p0.6.el6.x86\_64.rpm

http://archive.cloudera.com/cm5/redhat/6/x86\_64/cm/5.12.1/RPMS/x86\_64/cloudera-manager-server-5.12.1-1.cm5121.p0.6.el6.x86\_64.rpm

http://archive.cloudera.com/cm5/redhat/6/x86\_64/cm/5.12.1/RPMS/x86\_64/cloudera-manager-server-db-2-5.12.1-1.cm5121.p0.6.el6.x86\_64.rpm

http://archive.cloudera.com/cm5/redhat/6/x86\_64/cm/5.12.1/RPMS/x86\_64/enterprise-debuginfo-5.12.1-1.cm5121.p0.6.el6.x86\_64.rpm

http://archive.cloudera.com/cm5/redhat/6/x86\_64/cm/5.12.1/RPMS/x86\_64/jdk-6u31-linux-amd64.rpm

http://archive.cloudera.com/cm5/redhat/6/x86\_64/cm/5.12.1/RPMS/x86\_64/oracle-j2sdk1.7-1.7.0+update67-1.x86\_64.rpm

  • 將以上7個文件下載至服務器的/var/www/html/cm5.12.1目錄下

[root@ip-172-31-6-148~]# mkdir -p /var/www/html/cm5.12.1

[root@ip-172-31-6-148 ~]# cd /var/www/html/cm5.12.1/

[root@ip-172-31-6-148 cm5.12.1]# ll

total 942088

-rw-r--r-- 1 rootroot 9676820 Aug 30 14:45cloudera-manager-agent-5.12.1-1.cm5121.p0.6.el6.x86_64.rpm

-rw-r--r-- 1 rootroot 709818804 Aug 30 14:45 cloudera-manager-daemons-5.12.1-1.cm5121.p0.6.el6.x86_64.rpm

-rw-r--r-- 1 rootroot 8692 Aug 30 14:45cloudera-manager-server-5.12.1-1.cm5121.p0.6.el6.x86_64.rpm

-rw-r--r-- 1 rootroot 10600 Aug 30 14:45cloudera-manager-server-db-2-5.12.1-1.cm5121.p0.6.el6.x86_64.rpm

-rw-r--r-- 1 rootroot 31918952 Aug 30 14:45enterprise-debuginfo-5.12.1-1.cm5121.p0.6.el6.x86_64.rpm

-rw-r--r-- 1 rootroot 71204325 Aug 30 14:45jdk-6u31-linux-amd64.rpm

-rw-r--r-- 1 rootroot 142039186 Aug 30 14:45 oracle-j2sdk1.7-1.7.0+update67-1.x86_64.rpm

[root@ip-172-31-6-148 cm5.12.1]#

0024-CENTOS6.5安裝CDH5.12.1(一)

  • 在/var/www/html/cm5.12.1目錄下執行命令

[root@ip-172-31-6-148 cm5.12.1]# pwd

/var/www/html/cm5.12.1

[root@ip-172-31-6-148 cm5.12.1]#createrepo .

Spawning worker 0 with 7 pkgs

Workers Finished

Gathering worker results

Saving Primary metadata

Saving file lists metadata

Saving other metadata

Generating sqlite DBs

Sqlite DBs complete

[root@ip-172-31-6-148 cm5.12.1]#

0024-CENTOS6.5安裝CDH5.12.1(一)

  • 確認http是否能正常訪問

0024-CENTOS6.5安裝CDH5.12.1(一)

  • 在/etc/yum.repo.d/目錄下增加cm.repo文件,內容如下

[root@ip-172-31-6-148~]# vim /etc/yum.repos.d/cm.repo

cmrepo

name=CM5.12.1

baseurl=http://ip-172-31-6-148.fayson.com/cm5.12.1

gpgcheck=false

enabled=true

0024-CENTOS6.5安裝CDH5.12.1(一)

  • 驗證CM源是否配置成功

[root@ip-172-31-6-148~]# yum repolist

...

cmrepo CM5.12.1 7

localrepo CentOS6.5 248

0024-CENTOS6.5安裝CDH5.12.1(一)

  • 驗證安裝JDK

[root@ip-172-31-6-148 ~]#yum -y install oracle-j2sdk1.7-1.7.0+update67-1

3.2配置http訪問CDH Parcel包

CDH5.12.1下載地址:

http://archive.cloudera.com/cdh5/parcels/5.12.1/CDH-5.12.1-1.cdh5.12.1.p0.3-el6.parcel

http://archive.cloudera.com/cdh5/parcels/5.12.1/CDH-5.12.1-1.cdh5.12.1.p0.3-el6.parcel.sha1

http://archive.cloudera.com/cdh5/parcels/5.12.1/manifest.json

  • 將以上3個文件下載至/var/www/html/cdh5.12.1目錄下

[root@ip-172-31-6-148~]# mkdir -p /var/www/html/cdh5.12.1

[root@ip-172-31-6-148 ~]# cd /var/www/html/cdh5.12.1/

[root@ip-172-31-6-148 cdh5.12.1]# ll

total 1582508

-rw-r--r-- 1 rootroot 1620405492 Aug 30 19:09 CDH-5.12.1-1.cdh5.12.1.p0.3-el6.parcel

-rw-r--r-- 1 rootroot 41 Aug 30 19:09 CDH-5.12.1-1.cdh5.12.1.p0.3-el6.parcel.sha1

-rw-r--r-- 1 rootroot 72312 Aug 30 19:11 manifest.json

[root@ip-172-31-6-148 cdh5.12.1]#

0024-CENTOS6.5安裝CDH5.12.1(一)

  • 確認http是否能正常訪問

0024-CENTOS6.5安裝CDH5.12.1(一)

3.3安裝Cloudera Manager Server

  • 通過yum安裝ClouderaManager Server

[root@ip-172-31-6-148~]# yum -y installcloudera-manager-server

0024-CENTOS6.5安裝CDH5.12.1(一)

如上圖所示則表明安裝成功。

  • 初始化CM數據庫

[root@ip-172-31-6-148~]#/usr/share/cmf/schema/scm_prepare_database.sh mysql cm cm password

JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera

Verifying that we can write to /etc/cloudera-scm-server

Creating SCM configuration file in /etc/cloudera-scm-server

Executing: /usr/java/jdk1.7.0_67-cloudera/bin/java-cp/usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/cmf/schema/../lib/*com.cloudera.enterprise.dbutil.DbCommandExecutor/etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.

[ main] DbCommandExecutor INFO Successfully connected to database.

All done, your SCM database is configured correctly!

[root@ip-172-31-6-148 ~]#

0024-CENTOS6.5安裝CDH5.12.1(一)

  • 啓動ClouderaManager Server

[root@ip-172-31-6-148~]# service cloudera-scm-server start

Starting cloudera-scm-server: [ OK ]

[root@ip-172-31-6-148 ~]#

  • 檢查端口是否監聽

[root@ip-172-31-6-148~]# netstat -lnpt | grep 7180

tcp 0 0 0.0.0.0:7180 0.0.0.0:* LISTEN 4056/java

[root@ip-172-31-6-148 ~]#

0024-CENTOS6.5安裝CDH5.12.1(一)

  • 通過http:// 52.221.194.202:7180/cmf/login訪問CM控制檯

0024-CENTOS6.5安裝CDH5.12.1(一)

4.CDH安裝

4.1CDH集羣安裝嚮導

1.admin/admin登錄到CM

2.同意license協議,點擊繼續

0024-CENTOS6.5安裝CDH5.12.1(一)

3.選擇60試用,點擊繼續

0024-CENTOS6.5安裝CDH5.12.1(一)

4.點擊“繼續”

0024-CENTOS6.5安裝CDH5.12.1(一)

5.輸入主機ip或者名稱,點擊搜索找到主機後點擊繼續

0024-CENTOS6.5安裝CDH5.12.1(一)

0024-CENTOS6.5安裝CDH5.12.1(一)

6.點擊“繼續”

0024-CENTOS6.5安裝CDH5.12.1(一)

7.使用parcel選項,點擊“更多選項”,點擊“-”刪除其它所有地址,輸入

http:// ip-172-31-6-148.fayson.com/cdh5.12.1/點擊“保存更改”

0024-CENTOS6.5安裝CDH5.12.1(一)

8.選擇自定義存儲庫,輸入cm的http地址

0024-CENTOS6.5安裝CDH5.12.1(一)

9.點擊“繼續”,進入下一步安裝jdk

0024-CENTOS6.5安裝CDH5.12.1(一)

10.點擊“繼續”,進入下一步,默認多用戶模式

0024-CENTOS6.5安裝CDH5.12.1(一)

11.點擊“繼續”,進入下一步配置ssh賬號密碼

0024-CENTOS6.5安裝CDH5.12.1(一)

12.點擊“繼續”,進入下一步,安裝Cloudera Manager相關到各個節點

0024-CENTOS6.5安裝CDH5.12.1(一)

0024-CENTOS6.5安裝CDH5.12.1(一)

13.點擊“繼續”,進入下一步安裝cdh到各個節點

0024-CENTOS6.5安裝CDH5.12.1(一)

0024-CENTOS6.5安裝CDH5.12.1(一)

14.點擊“繼續”,進入下一步主機檢查,確保所有檢查項均通過

0024-CENTOS6.5安裝CDH5.12.1(一)

0024-CENTOS6.5安裝CDH5.12.1(一)

點擊完成進入服務安裝嚮導。

4.2集羣設置安裝嚮導

1.選擇需要安裝的服務

此處使用自定義服務,如下圖

0024-CENTOS6.5安裝CDH5.12.1(一)

2.點擊“繼續”,進入集羣角色分配

HDFS角色分配:

0024-CENTOS6.5安裝CDH5.12.1(一)

Hive角色分配:

0024-CENTOS6.5安裝CDH5.12.1(一)

Cloudera Manager Service角色分配:

0024-CENTOS6.5安裝CDH5.12.1(一)

Spark角色分配:(Spark on Yarn所以沒有spark的master和worker角色)

0024-CENTOS6.5安裝CDH5.12.1(一)

Yarn角色分配:

0024-CENTOS6.5安裝CDH5.12.1(一)

Zookeeper角色分配:(至少3個Server)

0024-CENTOS6.5安裝CDH5.12.1(一)

0024-CENTOS6.5安裝CDH5.12.1(一)

3.角色分配完成點擊“繼續”,進入下一步,測試數據庫連接

0024-CENTOS6.5安裝CDH5.12.1(一)

4.測試成功,點擊“繼續”,進入目錄設置,此處使用默認默認目錄,根據實際情況進行目錄修改

0024-CENTOS6.5安裝CDH5.12.1(一)

0024-CENTOS6.5安裝CDH5.12.1(一)

0024-CENTOS6.5安裝CDH5.12.1(一)

5.點擊“繼續”,等待服務啓動成功

0024-CENTOS6.5安裝CDH5.12.1(一)

0024-CENTOS6.5安裝CDH5.12.1(一)

6.點擊“繼續”,顯示集羣安裝成功

0024-CENTOS6.5安裝CDH5.12.1(一)

7.安裝成功後進入home管理界面

0024-CENTOS6.5安裝CDH5.12.1(一)

醉酒鞭名馬,少年多浮誇! 嶺南浣溪沙,嘔吐酒肆下!摯友不肯放,數據玩的花!
溫馨提示:要看高清無碼套圖,請使用手機打開並單擊圖片放大查看。

歡迎關注Hadoop實操,第一時間,分享更多Hadoop乾貨,喜歡請關注分享。

0024-CENTOS6.5安裝CDH5.12.1(一)
原創文章,歡迎轉載,轉載請註明:轉載自微信公衆號Hadoop實操

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章