HDP2.6.3.0+Ambari2.6.1大數據平臺

一、配置流程

http://www.cnblogs.com/zhang-ke/p/8944240.html
https://www.it610.com/article/1170280322634420224.htm
https://blog.csdn.net/weixin_34005042/article/details/92335346

四個包下載鏈接地址:
http://59.80.44.47/public-repo-1.hortonworks.com/HDP-GPL/centos7/2.x/updates/2.6.4.0/HDP-GPL-2.6.4.0-centos7-rpm.tar.gz
http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.3.0/HDP-2.6.3.0-centos7-rpm.tar.gz
http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.1.0/ambari-2.6.1.0-centos7.tar.gz
http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7/HDP-UTILS-1.1.0.22-centos7.tar.gz

參照此博主的博客搭建,本人是3臺虛擬機,按要求配置好hostname以及hosts,ssh等基礎工作。1、虛擬機配置爲:4核8G(定義爲低配,有條件的多加還是比較好)
2、在主服務器上配置HDP的壓縮包,從服務器上部署Mysql或者Maria數據庫,也可以不配置,默認選擇HDP的PostgreSQL數據庫(生產環境不推薦)。
3、儘量選擇離線安裝,HDP壓縮包太大。下載的包放到/var/www/html中。
4、選擇了MySQL數據庫就要在裏面創建表,hive/Oz/superset等在後續安裝中需要database。
5、如果是因爲MySQL的原因,參照此博主博客輸入安裝mysql驅動的命令解決。
6、在安裝過程中出現SSL問題,參考此劉大神博客解決,或次博主https://blog.csdn.net/qq_41805514/article/details/88871477,本人在前兩步完成後解決ssl問題。如果主機安裝還是出現問題,報錯節點找/usr/lib64/python2.7/ssl.py,deafult改成這個PROTOCOL_TLSv1_
在這裏插入圖片描述
在這裏插入圖片描述
在這裏插入圖片描述

二、成功圖片

在這裏插入圖片描述
在這裏插入圖片描述
在這裏插入圖片描述
在這裏插入圖片描述
![在這裏插入圖片描述](https://img-blog.csdnimg.cn/20190509140106193.jpg?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzMTQ3MTM2,size_16,color_FFFFFF,t_70在這裏插入圖片描述在這裏插入圖片描述在這裏插入圖片描述在這裏插入圖片描述在這裏插入圖片描述
在這裏插入圖片描述

報錯解決

問題:

DB configs consistency check: no errors and warnings were found.
ERROR: Exiting with exit code 1. 
REASON: Server not yet listening on http port 8080 after 50 seconds. Exiting.

方法:

vim  /etc/ambari-server/conf/ambari.properties
將以下屬性值增加到120或150秒
server.startup.web.timeout = 120

問題:

部分:
Metastore on vmocdp124.test.com failed (Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_metastore.py", line 183, in execute
timeout=int(check_command_timeout) )
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 154, in __init__
self.env.run()
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 158, in run
self.run_action(resource, action)
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 121, in run_action
provider_action()
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 238, in action_run
tries=self.resource.tries, try_sleep=self.resource.try_sleep)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner
result = function(command, **kwargs)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call
tries=tries, try_sleep=try_sleep)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in _call_wrapper
result = _call(command, **kwargs_copy)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 291, in _call
標準錯誤輸出: /var/lib/ambari-agent/data/errors-1312.txt

方法:

Metastore connection URL: jdbc:mysql://hdp001/hive?createDatabaseIfNotExist=true

問題出在URL,hive用戶通過hdp001這臺機器連接mysql需要設置密碼,

前面已經執行過:GRANT ALL PRIVILEGES ON hive.* TO 'hive'@'localhost' IDENTIFIED BY 'hive'; (創建hive用戶)
GRANT ALL PRIVILEGES ON hive.* TO 'hive'@'%' IDENTIFIED BY 'hive';
FLUSH PRIVILEGES; (刷新權限)

已經設置過hive用戶連接的密碼,沒想到還會連接不上

最後執行GRANT ALL PRIVILEGES ON hive.* TO 'hive'@'hdp001' IDENTIFIED BY 'hive',問題解決

問題:Hbase在Ambari中啓動後隨即自動關閉
方法:

ntpd時間,幾臺機器date驗證時間是否一致。

問題:

NameNode Last Checkpoint script alert definition does not trigger based on uncommitted transactions。
報錯誤[Checkpoint Critical]

方法:

sudo su hdfs -l -c 'hdfs dfsadmin -safemode enter'

sudo su hdfs -l -c 'hdfs dfsadmin -saveNamespace'
After that restarting both NameNodes cleared the Alerts.

問題:

Ambari2.4.2 Hbase啓動失敗
HBase Master Process
Connection failed: [Errno 111] Connection refused to bi.corp.mama.cn2:16000

啓動過程是沒有報錯的,都是Success,但是alter提示這個,過一會兒本來Start的HBase Master就自動Stop

辦法:

yum -y install ntp(安裝過的可以直接下一步)
cd /usr/share/zoneinfo/Asia/
ntpdate pool.ntp.org
systemctl start ntpd && systemctl enable ntpd
**SSH上的問題解決辦法要想到各節點之間的免密配置,以及自身0.0.0.0的免密配置。**
**端口上的問題,自建虛擬機不存在,若是阿里雲服務器要在其安全組裏添加端口**

如何在CDH中安裝CFM
https://cloud.tencent.com/developer/article/1426077

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章