CDH與CM下載:
CDH5.4
http://archive.cloudera.com/cdh5/
Cloudera Manager5.4.3:
http://www.cloudera.com/downloads/manager/5-4-3.html
三臺機器:內存分配 10g 2g 2g
系統環境準備
1.網絡配置 hosts
2.ssh免密鑰登錄
ssh-keygen -t rsa -P ‘’ -f ~/.ssh/id_rsa
ssh-copy-id host
3.防火牆關閉
4.selinux關閉(vi /etc/selinux/config{SELINUX=disabled})
5.安裝jdk配置環境變量
6.安裝ntp
7.安裝配置mysql,裝一臺就夠了
yum install mysql-server
systemctl start mysqld
/usr/bin/mysql_secure_installation
mysql -uroot -phadoop
刪除 mysql.user
grant all_privileges on . To ‘root’@’%’ identified by ‘123456’ with grant option;
flush_privileges
8.下載第三方依賴包 補全操作系統的工具
yum install -y chkconfig python bind-utils psmic libxslt zlib sqlite cyrus-sasl-plain cyrus-sasl-gssapi fuse fuse-libs redhat-lsb
ClouderaManager安裝
1.安裝Cloudera Manager Server、Agent
mkdir /opt/cloudera-manager
tar xvzf cloudera-manager*.tar.gz -C /opt/cloudera-manager
2.創建用戶cloudera-scm (後臺用戶 無家目錄 無bash)(all)
useradd --system --no-create-home --shell=/bin/false --comment “Cloudera SCM user” cloudera-scm
3.配置CM Agent, agent尋址server(all)
修改文件/opt/cloudera-manager/cm-5.4.3/etc/cloudera-scm-agent/config.ini 中 hosts
4.配置CM Server 數據庫
拷貝mysql.jar文件到目錄 /usr/share/java/
注意.jar包名稱要修改mysql-connector-java.jar
grant all on . to ‘temp’@’%’ identified by ‘temp’ with grant option
cd /opt/cloudera-manager/cm-5.4.3/share/cmf/schema/
./scm_prepare_database.sh mysql temp -h node01 -utemp -ptemp --scm-host node01 scm scm scm
格式:數據庫類型、數據庫、數據庫服務器、用戶名、密碼、cm server服務器
5.創建Parcel目錄
-Server節點
mkdir -p /opt/cloudera/parcel-repo
chown cloudera-scm:cloudera-scm /opt/cloudera/parcel-repo
-Agent節點
mkdir-p /opt/cloudera/parcel
chown cloudera-scm:cloudera-scm /opt/cloudera/parcel
6.製作CDH本地源
下載好CDH-5.4.0-1.cdh5.4.0.p0.27-el6.parcel(http://archive.cloudera.com/cdh5/parcels/5.4.0/CDH-5.4.0-1.cdh5.4.0.p0.27-el6.parcel)以及manifest.json(http://archive.cloudera.com/cdh5/parcels/5.4.0/manifest.json),將這兩個文件放到server節點/opt/cloudera/parcel-repo下
打開json文件。找到與下載版本對應的hash碼,新建文件,文件名與你的parel包名一致,並加上.sha後綴,保存。
7.啓動CM Server 、Agent
cd /opt/cloudera-manager/cm-5.4.3/etc/init.d
./cloudera-scm-server start
注意:首次啓動會創建表及數據
./cloudera-scm-agent start
user:admin
password:admin
jobs -l 列出後臺進程號
netstat -natp | grep 50070
tar xf cdh.tar.gz 1>/dev/null