Amabri2.6.0、hdp2.6.1在centos7下安裝
一、環境準備
1、修改各個節點主機名
vi /etc/hostname
2、配置主節點hosts
vi /etc/hosts
2、配置免密
(1)手動配置
主節點執行:
ssh-keygen -t rsa
ssh-copy-id $host(第1步中的各個節點名稱)
(2)腳本配置
3、同步hosts
主節點執行:scp /etc/hosts $host:/etc/ ($host各從節點名稱)
4、關閉防火牆、selinux
各個節點執行:
systemctl stop firewalld.service
systemctl disable firewalld.service
sed -i \”s;enforcing;disabled;g\” /etc/selinux/config
重啓:reboot
5、安裝httpd
查看httpd是否已安裝:rpm -qa |grep httpd
安裝:rpm -ivh httpd/*.rpm
6、鏡像本地yum源配置
新建文件夾:mkdir /var/www/html/iso
將安裝鏡像中的文件全部複製到iso文件夾下
啓動httpd:systemctl start httpd.service
驗證:瀏覽器打開http://ip/iso/
編輯:vi /etc/yum.repos.d/local.repo
local.repo同步到其他節點
yum clean all
yum list update
yum makecache
yum repolist
7、檢查更改最大文件描述符
ulimit -Sn
ulimit -Hn
如果\<10000,運行ulimit -n 10000
二、依賴檢查
1、Python 2.7.X(所有節點)
Centos7自帶python2.7.5即可。
(Anaconda3.6 還原爲系統自帶2.7:bashrc
註釋掉anaconda的環境變量,退出用戶,重新登錄。)
2、JDK (所有節點)
HDP 2.6.1 JDK Support
JDK | Version |
---|---|
Open Source | JDK8† |
JDK7† , deprecated | |
Oracle | JDK 8, 64-bit (minimum JDK 1.8.0_77), default |
JDK 7, 64-bit (minimum JDK 1.7_67), deprecated |
(1)檢測並卸載之前安裝的版本:
rpm -qa |grep java
rpm -e java-1.8.0-openjdk-1.8.0.151-1.b12.el7_4.x86_64 –nodeps
rpm -e java-1.8.0-openjdk-headless-1.8.0.151-1.b12.el7_4.x86_64 –nodeps
(2)安裝
rpm -ivh jdk-8u77-linux-x64.rpm
(3)驗證
java -version
若找不到java,需配置環境變量:
在/etc/profile添加
source /etc/profile
3、MySQL5.6 (主節點)
(1)下載
http://mirrors.sohu.com/mysql/MySQL-5.6/
(2)查詢並卸載系統自帶mariadb
rpm -qa | grep mariadb -i
rpm -e MariaDB-common-10.2.13-1.el7.centos.x86_64
MariaDB-compat-10.2.13-1.el7.centos.x86_64
MariaDB-client-10.2.13-1.el7.centos.x86_64
MariaDB-server-10.2.13-1.el7.centos.x86_64 –nodeps
(3)安裝perl、net-tools
yum install perl
yum install net-tools
(4)安裝
tar -xvf MySQL-5.6.36-1.el7.x86_64.rpm-bundle.tar
rpm -ivh MySQL-server-5.6.36-1.el7.x86_64.rpm
密碼在/root/.mysql_secret
service mysql start
rpm -ivh MySQL-client-5.6.36-1.el7.x86_64.rpm
執行mysql_secure_installation
(5)設置用戶
驗證登陸MySQL控制檯:mysql -uroot –p
grant all privileges on *.* to root\@’%’ identified by “123456”;
flush privileges;
創建ambari數據庫及用戶:
create database ambari character set utf8 ;
CREATE USER ‘ambari’\@’%’ IDENTIFIED BY ‘ambari’;
GRANT ALL PRIVILEGES ON *.* TO ‘ambari’\@’%’ IDENTIFIED BY ‘ambari’;
FLUSH PRIVILEGES;
創建hive數據庫及用戶:
create database hive character set utf8 ;
CREATE USER ‘hive’\@’%’ IDENTIFIED BY ‘hive’;
GRANT ALL PRIVILEGES ON *.* TO ‘hive’\@’%’ IDENTIFIED BY ‘hive’;
FLUSH PRIVILEGES;
創建oozie數據庫及用戶:
create database oozie character set utf8 ;
CREATE USER ‘oozie’\@’%’ IDENTIFIED BY ‘oozie’;
GRANT ALL PRIVILEGES ON *.* TO ‘oozie’\@’%’ IDENTIFIED BY ‘oozie’;
FLUSH PRIVILEGES;
(6)設置MySQL爲開機啓動
chkconfig mysql on;
chkconfig –list mysql;
查看是否如同如下結果
mysql 0:off 1:off 2:on 3:on 4:on 5:on 6:off
(7)安裝:
rpm -ivh MySQL-shared-compat-5.6.36-1.el7
rpm -ivh MySQL-shared-5.6.36-1.el7
(file /usr/share/mysql/charsets/README from install of
mariadb-libs-1:5.5.52-1.el7.x86_64 conflicts with file from package
MySQL-server-5.6.36-1.el7.x86_64)
三、ambari安裝
hdp、hdp-utils下載地址
ambari下載地址
1、準備本地yum源
mkdir -p /var/www/html/ambari/HDP-UTILS-1.1.0.21
tar -zxf HDP-UTILS-1.1.0.21-centos7.tar.gz -C
/var/www/html/ambari/HDP-UTILS-1.1.0.21
tar -zxf HDP-2.6.1.0-centos7-rpm.tar.gz -C /var/www/html/ambari/
tar -zxf ambari-2.6.0.0-centos7.tar.gz -C /var/www/html/ambari/
重啓httpd:systemctl restart httpd.service
驗證:
2、配置repo
Ambari
cp /var/www/html/ambari/ambari/centos7/2.6.0.0-267/ambari.repo /etc/yum.repos.d/
HDP
cp /var/www/html/ambari/HDP/centos7/hdp.repo /etc/yum.repos.d
HDP-UTILS
cp /var/www/html/ambari/HDP-UTILS-1.1.0.21/hdp-util.repo /etc/yum.repos.d
scp到所有節點,執行:
yum clean all
yum list update
yum makecache
yum repolist
3、ambari-server安裝
(1)安裝mysql-connect
yum install mysql-connector-java
(2)安裝ambari-server
yum install ambari-server
(3)配置ambari
ambari-server setup
(4)將ambari數據庫腳本導入到數據庫
ambari用戶(上面設置的用戶)登錄mysql
mysql -u ambari -p
use ambari;
source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
(5)設置mysql-connector路徑
ambari-server setup –jdbc-db=mysql
–jdbc-driver=/usr/share/java/mysql-connector-java.jar
(6)啓動ambari-server
ambari-server start
(7)驗證
瀏覽器打開http://ip:8080/,用戶名admin,密碼admin
四、hdp安裝
1、libtirpc安裝
rpm -ivh libtirpc-0.2.4-0.10.el7.x86_64.rpm
libtirpc-devel-0.2.4-0.10.el7.x86_64.rpm
若提示與舊版本衝突,可用rpm -Uvh
2、登錄ambari進行安裝
點擊Launch Install Wizard
(1) 設置集羣名稱
(2) 選擇本地安裝源
(3)設置集羣
(4)確認hosts
(5)選擇要安裝的服務
(6)分配各服務master
安裝組件時注意hive-metastore與mysql在同一臺機器。
(7)分配slaves
(8)更改服務配置
刪掉/boot/efi/hadoop/hdfs/namenode、/boot/efi/hadoop/hdfs/data
更改hive數據庫爲mysql
(9)顯示配置信息
(10)開始安裝
四、增加節點
五、啓用HA
按照提示輸入命令進行之後的安裝。