Hadoop 學習第一篇之Ambari部署。

 

一）install JDK

必須下載jdk1.8，這才能穩定版本服務，

安裝步驟

#解壓下載好的安裝包
tar -zxvf jdk1_8xxxxxxx


#配置環境變量
vim /etc/profile

export JAVA_HOME=/usr/local/java/jdk1.8.0_171
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=${JAVA_HOME}/bin:$PATH

#讓配置文件生效
source /etc/profile
#查看版本
java -version

注意：所涉及到的集羣服務器都要部署jdk。

二）配置本機名

注意：根據自己的需求自行定義

三）修改文件打開限制

[root@master ~]# vi /etc/security/limits.conf
# End of file
* soft nofile 65536
* hard nofile 65536
* soft nproc 131072
* hard nproc 131072

四）設置時間同步

#安裝ntp服務：如果是企業服務器的話不需要管理
yum -y install ntp

五）SSH免密

#每臺服務器都要進行操作
ssh-keygen -t rsa
詳情:https://blog.csdn.net/qq_36523839/article/details/78885216。

六）安裝Httpd服務

[root@master ~]# yum -y install httpd
[root@master ~]# service httpd restart
Redirecting to /bin/systemctl restart httpd.service
[root@master ~]# chkconfig httpd on

注意：1、端口號默認是80，系要去/etc/httpd/httpd.conf進行修改，切勿衝突

           2、通過Httpd 去創建數據源路徑，常理是放在/var/www/html下面，但是內存不足的話建議使用軟連接

#軟連接的格式
# ln -s /你想要查找的路徑   httpd的URl後端拼接的名稱
ln  -s  /onstardata/hadoop/  ambari

七）安裝本地數據源製作相關工具，只在主服務器上即可

[root@master ambari]#  yum install yum-utils createrepo yum-plugin-priorities -y
[root@master ambari]#  createrepo  ./

八）修改文件裏面的源地址（主服務器）

[root@master ambari]# vi ambari/centos7/2.6.0.0-267/ambari.repo
#VERSION_NUMBER=2.6.0.0-267
[ambari-2.6.0.0]
name=ambari Version - ambari-2.6.0.0
baseurl=http://ip+port/ambari/ambari/centos7/2.6.0.0-267
gpgcheck=1
gpgkey=http://ip+port/ambari/ambari/centos7/2.6.0.0-267/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
[root@master ambari]# cp ambari/centos7/2.6.0.0-267/ambari.repo /etc/yum.repos.d/
[root@master ambari]# vi HDP/centos7/2.6.3.0-235/hdp.repo
#VERSION_NUMBER=2.6.3.0-235
[HDP-2.6.3.0]
name=HDP Version - HDP-2.6.3.0
baseurl=http://ip+port/ambari/HDP/centos7/2.6.3.0-235
gpgcheck=1
gpgkey=http://ip+port/ambari/HDP/centos7/2.6.3.0-235/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1


[HDP-UTILS-1.1.0.21]
name=HDP-UTILS Version - HDP-UTILS-1.1.0.21
baseurl=http://ip+port/ambari/HDP-UTILS
gpgcheck=1
gpgkey=http://ip+port/ambari/HDP-UTILS/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
[root@master ambari]# cp HDP/centos7/2.6.3.0-235/hdp.repo /etc/yum.repos.d/

#清理yum緩存
[root@master ambari]# yum clean all
[root@master ambari]# yum makecache
[root@master ambari]# yum repolist

注意：把這些數據源cp到子服務器上

#方法一： 
scp HDP-2.6.3.0-centos7-rpm.tar.gz [email protected]:/onstardata
#方法二：
# 各個服務器上執行命令，利用 rz 和 sz 進行導入導出
 1、yum -y install lrzsz

九）安裝數據庫 只需要在主服務器上操作

#1、下載mysql rpm源
wget http://repo.mysql.com/mysql57-community-release-el7-8.noarch.rpm
#如果提示-bash: wget: command not found
yum install wget

#2、安裝rpm源
rpm -ivh mysql57-community-release-el7-8.noarch.rpm

#3、安裝mysql-server
yum install mysql-server

#4、連接服務器 
mysql -uroot -p root
#錯誤如下：ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
#錯誤分析以及解決 /var/lib/mysql權限問題，修改MySQL權限爲當前用戶
sudo chown -R xxx:xxx /var/lib/mysql
#如上xxx:xxx當前的用戶名以及所屬組 我這裏是root:root
#重啓MySQL服務

#5、重新登陸
mysql -u root -p root
#出現如下問題 
#ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES)
#解決方案(獲取初始密碼)
cat /var/log/mysqld.log  | grep password

#如上圖所示爲初始密碼_獲得後繼續登錄
# Mysql登錄成功之後
set global validate_password_policy=0;
set global validate_password_length=4;
#設置密碼
alter user 'root'@'localhost' identified by 'root';

#部署數據庫執行
yum –y install ambari
CREATE DATABASE ambari;  
use ambari;  
CREATE USER 'ambari'@'%' IDENTIFIED BY 'ambarizk123';  
GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'%';
FLUSH PRIVILEGES;  
source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql  
show tables;
use mysql;  
select Host User Password from user where user='ambari';  

CREATE DATABASE hive;  
use hive;  
CREATE USER 'hive'@'%' IDENTIFIED BY 'hive';  
GRANT ALL PRIVILEGES ON *.* TO 'hive'@'%';
FLUSH PRIVILEGES;  

CREATE DATABASE oozie;  
use oozie;  
CREATE USER 'oozie'@'%' IDENTIFIED BY 'oozie';  
GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'%';
FLUSH PRIVILEGES;

十）建立Mysql和Ambari-Server建立連接

# 自行下載 mysql-connector-java.jar 5.7版本的 
mkdir /usr/share/java
cp /root/mysql-connector-java-5.1.40.jar /usr/share/java/mysql-connector-java.jar
cp /usr/share/java/mysql-connector-java.jar /var/lib/ambari-server/resources/mysql-jdbc-driver.jar
vi /etc/ambari-server/conf/ambari.properties
#添加server.jdbc.driver.path=/usr/share/java/mysql-connector-java.jar

十一）初始化ambari-server並啓動

[root@master ~]# ambari-server setup
下面是配置執行流程，按照提示操作
（1） 提示是否自定義設置。輸入：y
Customize user account for ambari-server daemon [y/n] (n)? y
（2）ambari-server 賬號。
Enter user account for ambari-server daemon (root):
如果直接回車就是默認選擇root用戶
如果輸入已經創建的用戶就會顯示：
Enter user account for ambari-server daemon (root):ambari
Adjusting ambari-server permissions and ownership...
（3）檢查防火牆是否關閉
Adjusting ambari-server permissions and ownership...
Checking firewall...
WARNING: iptables is running. Confirm the necessary Ambari ports are accessible. Refer to the Ambari documentation for more details on ports.
OK to continue [y/n] (y)?
直接回車
（4）設置JDK。輸入：3
Checking JDK...
Do you want to change Oracle JDK [y/n] (n)? y
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7
[3] Custom JDK
============================================================================
Enter choice (1): 3
如果上面選擇3自定義JDK,則需要設置JAVA_HOME。輸入：/usr/java/jdk1.8.0_161
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /usr/java/jdk1.8.0_131
Validating JDK on Ambari Server...done.
Completing setup...
（5）數據庫配置。選擇：y
Configuring database...
Enter advanced database configuration [y/n] (n)? y
（6）選擇數據庫類型。輸入：3
Configuring database...
============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
============================================================================
Enter choice (3): 3
（7）設置數據庫的具體配置信息，根據實際情況輸入，如果和括號內相同，則可以直接回車。如果想重命名，就輸入。
Hostname (localhost):
Port (3306):
Database name (ambari):
Username (ambari):
Enter Database Password (bigdata):ambarizk123
Re-Enter password: ambarizk123
（8）將Ambari數據庫腳本導入到數據庫
WARNING: Before starting Ambari Server, you must run the following DDL against the database to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql 
Proceed with configuring remote database connection properties [y/n] (y)?  
[root@master ~]# ambari-server start

這裏出問題一定看日誌，因爲問題各有個的不同。/var/log/ambari-server/ambari-server.log