點擊上方藍色“大數據實戰演練”,選擇“設爲星標”或“置頂”
回覆“資料”領取獨家整理的學習資料!
每一個成功人士的背後,必定曾經做出過勇敢而又孤獨的決定。
放棄不難,但堅持很酷~
一、前言
如何將服務一勞永逸的集成到 ambari 中,方便新環境部署,無需二次拷貝呢?
通常,我們在自定義服務集成開發中,會將集成服務的代碼項目放置到 /var/lib/ambari-server/resources/stacks/HDP/<version>/services/ 目錄下。這樣子的前提就是 ambari 與 hdp 集羣必須提前搭建好。
其實這種方式是適合開發模式的。但是當集成服務開發、測試好之後,封版的話,再用這種方式就不太好了。假如有幾十個自定義服務,你都得挨個手動拷貝,麻煩還容易犯錯不是。並且萬一要部署的環境再一多,顯然這種並不是很好的部署模式。
那怎麼辦呢?
我們將集成服務的源碼整合到 ambari-server rpm 包中不就行了?
我們需要將集成服務的源碼放置在 ambari-server 模塊的 resources 的對應目錄下,然後再通過操作得到新的 ambari-server 的 rpm 即可。這裏針對 rpm 包的製作提供了兩種方式:
方式一:整體編譯 ambari 源碼(比較複雜,但是正規)
方式二:重新構建 rpm 包
二、方式一:整體編譯 ambari 源碼
以 ELASTICSEARCH 服務爲例,將 ELASTICSEARCH 相關項目代碼拷貝到 hdp 對應的 version 中,然後整體編譯,得到新的 ambari-server rpm 包。
整體編譯命令:
mvn -B -X -e install package rpm:rpm -DnewVersion=2.7.3.0.0 -DskipTests -Dpython.ver="python >= 2.6" -Drat.skip=true -Preplaceurl
詳細可參考我的博客:《Ambari2.7.1整體編譯+安裝使用》 。
三、方式二:重新構建 rpm 包
如果嫌整體編譯 ambari 太繁瑣,也可以基於已有的 ambari-server rpm 包,重新構建 rpm 包,這樣相對簡單一些。
1、首先安裝 rpmrebuild 工具
鏈接:https://pan.baidu.com/s/1SkO2SlLmrl4vWqbgrRuzHg 提取碼:m1ti
將 tar 包拷貝到 /opt 下面,編譯 rpmrebuild ,貌似 rpm 包與 rpmrebuild 版本有關係。
cd /opt
tar zxvf rpmrebuild-2.11.tar.gz
make
make install
2、安裝工具 rpm-build
yum install rpm-build
3、安裝 ambari-server
如果環境中沒有 ambari-server 服務,可以先安裝官方提供的 ambari-server 的 rpm 包
yum install ambari-server
4、將集成服務拷貝到指定目錄
將自定義的集成服務拷貝到
/var/lib/ambari-server/resources/stacks/HDP/<version>/services/
目錄下即可。
1)比如我要將 ELASTICSEARCH 集成到 hdp 3.1 裏面,所以我需要將 ELASTICSEARCH 服務代碼拷貝到 /var/lib/ambari-server/resources/stacks/HDP/3.1/services/ 目錄下。
5、rebuild ambari-server 的 rpm 包
rpmrebuild -e ambari-server-2.7.3.0-139.x86_64
在執行 rpmrebuild -e 命令的交互過程中,可修改 .spec 文件,增加 ELASTICSEARCH 的相關文件:
%dir %attr(0755, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH"
%dir %attr(0755, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/configuration"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/configuration/elastic-config.xml"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/configuration/elastic-env.xml"
%dir %attr(0755, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package"
%dir %attr(0755, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/alerts"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/alerts/alert_check_dir.py"
%dir %attr(0755, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/scripts"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/scripts/cfgNodejs.sh"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/scripts/changeOsConfToES.sh"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/scripts/createUser.sh"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/scripts/elastic_common.py"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/scripts/elasticsearch_client.py"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/scripts/es_metrics.py"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/scripts/head.py"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/scripts/master.py"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/scripts/params.py"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/scripts/service_check.py"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/scripts/status_params.py"
%dir %attr(0755, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/templates"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/templates/changeHostName.sh.j2"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/templates/data_to_metrics.sh.j2"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/templates/es_metrics.py.j2"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/package/templates/install_requests.sh.j2"
%dir %attr(0755, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/quicklinks"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/quicklinks/quicklinks-es.json"
%dir %attr(0755, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/themes"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/themes/theme_elasticsearch_version_4.json"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/metainfo.xml"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/metrics.json"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/role_command_order.json"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/widgets.json"
%attr(0644, root, root) "/var/lib/ambari-server/resources/stacks/HDP/3.1/services/ELASTICSEARCH/alerts.json"
大概需要 3、4 分鐘,rpm 包就重新構建好了。
6、檢查新構建的 rpm 包內是否有新添加的文件
rpm -qpl ambari-server-2.7.3.0-139.x86_64.rpm
7、安裝新構建的 rpm 包
安裝下新構建的 rpm 包,測試新集成的 ELASTICSEARCH 服務功能是否正常。
1)停止存在的 ambari-server 進程:ambari-server stop
2)卸載 ambari-server 服務:yum remove ambari-server
3) 安裝新構建的 rpm 包:yum install /root/ambari-server-2.7.3.0-139.x86_64.rpm
4)重新設置 ambari-server 配置:ambari-server setup
5)啓動 ambari-server 進程
6)重裝 ELASTICSEARCH 服務,測試功能。
本文對應的視頻已上傳至 bilibili 平臺,可點擊原文鏈接或拷貝以下鏈接觀看:
https://www.bilibili.com/video/BV1xz4y117K4/
關於對 Ambari 自定義服務集成視頻十八講的詳細介紹,可以拷貝以下鏈接觀看:
https://mp.weixin.qq.com/s/2nlDgiJj4-AJIsZpEIwZIQ
往期推薦
最後說一句(求關注,別白嫖我)
掃一掃,我們的故事就開始了。
文章有用,點贊、轉發、在看都是一種支持,求三連!
另外公衆號改變了推送規則,大家看文章不要忘記點擊最下方的在看,點贊按鈕,這樣微信自動識別爲常看公衆號,否則很可能推送的文章可能淹沒在別的文章找不到,謝謝大家。
讓我知道你在看
本文分享自微信公衆號 - 大數據實戰演練(gh_f942bfc92d26)。
如有侵權,請聯繫 [email protected] 刪除。
本文參與“OSC源創計劃”,歡迎正在閱讀的你也加入,一起分享。