使用docker搭建部署hadoop分佈式集羣
在網上找了很長時間都沒有找到使用docker搭建hadoop分佈式集羣的文檔，沒辦法，只能自己寫一個了。
一：環境準備：
1：首先要有一個Centos7操作系統，可以在虛擬機中安裝。
a、root賬戶登錄，查看內核版本
[root@admin01 ~]# uname -a
Linux admin01 3.10.0-862.el7.x86_64 #1 SMP Fri Apr 20 16:44:24 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
b、更新yum源
[root@admin01 ~]# yum update
c、安裝一些必要的系統工具
[root@admin01 ~]# yum install -y yum-utils device-mapper-persistent-data lvm2
d、添加軟件源信息：
[root@admin01 ~]# yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
e、 更新 yum 緩存：
[root@admin01 ~]# yum makecache fast
f、 可以查看所有倉庫中所有docker版本，並選擇特定版本安裝
[root@admin01 ~]# yum list docker-ce --showduplicates | sort -r
g、安裝Docker，命令：yum install docker-ce-版本號，我選的是17.12.1.ce
[root@admin01 ~]# yum install docker-ce-17.12.1.ce
h、驗證安裝是否成功(有client和service兩部分表示docker安裝啓動都成功了)
[root@admin01 ~]# docker version
Client:
Version: 17.12.0-ce
API version: 1.35
Go version: go1.9.2
Git commit: c97c6d6
Built: Wed Dec 27 20:10:14 2017
OS/Arch: linux/amd64
Server:
Engine:
Version: 17.12.0-ce
API version: 1.35 (minimum version 1.12)
Go version: go1.9.2
Git commit: c97c6d6
Built: Wed Dec 27 20:12:46 2017
OS/Arch: linux/amd64
Experimental: false
i、啓動Docker，命令：systemctl start docker，然後加入開機啓動
[root@admin01 ~]# systemctl start docker
[root@admin01 ~]# systemctl enable docker
Created symlink from /etc/systemd/system/multi-user.target.wants/docker.service to /usr/lib/systemd/system/docker.service.
2：需要先構建一個hadoop的基礎鏡像，使用dockerfile文件方式進行構建。
先構建一個具備ssh功能的鏡像，方便後期使用。（但是這樣對於容器的安全性會有影響）
注意：這個鏡像中的root用戶的密碼是root
[root@admin01 centos-ssh-root]# mkdir centos-ssh-root
[root@admin01 centos-ssh-root]# cd centos-ssh-root
[root@admin01 centos-ssh-root]# vi Dockerfile
其中Dockerfile裏面的內容爲：
[root@admin01 centos-ssh-root]# vi Dockerfile
#添加
# 選擇一個已有的os鏡像作爲基礎
FROM centos
# 鏡像的作者
MAINTAINER crxy
# 安裝openssh-server和sudo軟件包，並且將sshd的UsePAM參數設置成no
RUN yum install -y openssh-server sudo
RUN sed -i 's/UsePAM yes/UsePAM no/g' /etc/ssh/sshd_config
#安裝openssh-clients
RUN yum install -y openssh-clients
# 添加測試用戶root，密碼root，並且將此用戶添加到sudoers裏
RUN echo "root:root" | chpasswd
RUN echo "root ALL=(ALL) ALL" >> /etc/sudoers
# 下面這兩句比較特殊，在centos6上必須要有，否則創建出來的容器sshd不能登錄
RUN ssh-keygen -t dsa -f /etc/ssh/ssh_host_dsa_key
RUN ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key
# 啓動sshd服務並且暴露22端口
RUN mkdir /var/run/sshd
EXPOSE 22
CMD ["/usr/sbin/sshd", "-D"]
構建命令：
[root@admin01 centos-ssh-root]# docker build -t="admin/centos-ssh-root" .
Sending build context to Docker daemon 2.56kB
Step 1/12 : FROM centos
---> 67fa590cfc1c
Step 2/12 : MAINTAINER crxy
---> Running in 6a6f50fca5b5
Removing intermediate container 6a6f50fca5b5
---> 5ab81450a24c
Step 3/12 : RUN yum install -y openssh-server sudo
---> Running in 7e2893515277
Loaded plugins: fastestmirror, ovl
Determining fastest mirrors
* base: mirror.lzu.edu.cn
* extras: mirrors.huaweicloud.com
* updates: mirrors.cn99.com
Resolving Dependencies
--> Running transaction check
---> Package openssh-server.x86_64 0:7.4p1-16.el7 will be installed
--> Processing Dependency: openssh = 7.4p1-16.el7 for package: openssh-server-7.4p1-16.el7.x86_64
--> Processing Dependency: fipscheck-lib(x86-64) >= 1.3.0 for package: openssh-server-7.4p1-16.el7.x86_64
--> Processing Dependency: libwrap.so.0()(64bit) for package: openssh-server-7.4p1-16.el7.x86_64
--> Processing Dependency: libfipscheck.so.1()(64bit) for package: openssh-server-7.4p1-16.el7.x86_64
---> Package sudo.x86_64 0:1.8.23-3.el7 will be installed
--> Running transaction check
---> Package fipscheck-lib.x86_64 0:1.4.1-6.el7 will be installed
--> Processing Dependency: /usr/bin/fipscheck for package: fipscheck-lib-1.4.1-6.el7.x86_64
---> Package openssh.x86_64 0:7.4p1-16.el7 will be installed
---> Package tcp_wrappers-libs.x86_64 0:7.6-77.el7 will be installed
--> Running transaction check
---> Package fipscheck.x86_64 0:1.4.1-6.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
================================================================================
Package Arch Version Repository Size
================================================================================
Installing:
openssh-server x86_64 7.4p1-16.el7 base 458 k
sudo x86_64 1.8.23-3.el7 base 841 k
Installing for dependencies:
fipscheck x86_64 1.4.1-6.el7 base 21 k
fipscheck-lib x86_64 1.4.1-6.el7 base 11 k
openssh x86_64 7.4p1-16.el7 base 510 k
tcp_wrappers-libs x86_64 7.6-77.el7 base 66 k
Transaction Summary
================================================================================
Install 2 Packages (+4 Dependent packages)
Total download size: 1.9 M
Installed size: 6.1 M
Downloading packages:
warning: /var/cache/yum/x86_64/7/base/packages/fipscheck-1.4.1-6.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY
Public key for fipscheck-1.4.1-6.el7.x86_64.rpm is not installed
--------------------------------------------------------------------------------
Total 1.4 MB/s | 1.9 MB 00:01
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
Importing GPG key 0xF4A80EB5:
Userid : "CentOS-7 Key (CentOS 7 Official Signing Key) <[email protected]>"
Fingerprint: 6341 ab27 53d7 8a78 a7c2 7bb1 24c6 a8a7 f4a8 0eb5
Package : centos-release-7-6.1810.2.el7.centos.x86_64 (@CentOS)
From : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : fipscheck-1.4.1-6.el7.x86_64 1/6
Installing : fipscheck-lib-1.4.1-6.el7.x86_64 2/6
Installing : openssh-7.4p1-16.el7.x86_64 3/6
Installing : tcp_wrappers-libs-7.6-77.el7.x86_64 4/6
Installing : openssh-server-7.4p1-16.el7.x86_64 5/6
Installing : sudo-1.8.23-3.el7.x86_64 6/6
Verifying : fipscheck-lib-1.4.1-6.el7.x86_64 1/6
Verifying : sudo-1.8.23-3.el7.x86_64 2/6
Verifying : openssh-7.4p1-16.el7.x86_64 3/6
Verifying : fipscheck-1.4.1-6.el7.x86_64 4/6
Verifying : tcp_wrappers-libs-7.6-77.el7.x86_64 5/6
Verifying : openssh-server-7.4p1-16.el7.x86_64 6/6
Installed:
openssh-server.x86_64 0:7.4p1-16.el7 sudo.x86_64 0:1.8.23-3.el7
Dependency Installed:
fipscheck.x86_64 0:1.4.1-6.el7 fipscheck-lib.x86_64 0:1.4.1-6.el7
openssh.x86_64 0:7.4p1-16.el7 tcp_wrappers-libs.x86_64 0:7.6-77.el7
Complete!
Removing intermediate container 7e2893515277
---> a23041fa059e
Step 4/12 : RUN sed -i 's/UsePAM yes/UsePAM no/g' /etc/ssh/sshd_config
---> Running in d5b7eb666c82
Removing intermediate container d5b7eb666c82
---> 8171ca2f7545
Step 5/12 : RUN yum install -y openssh-clients
---> Running in 8d49b5c61e8b
Loaded plugins: fastestmirror, ovl
Loading mirror speeds from cached hostfile
* base: mirror.lzu.edu.cn
* extras: mirrors.huaweicloud.com
* updates: mirrors.cn99.com
Resolving Dependencies
--> Running transaction check
---> Package openssh-clients.x86_64 0:7.4p1-16.el7 will be installed
--> Processing Dependency: libedit.so.0()(64bit) for package: openssh-clients-7.4p1-16.el7.x86_64
--> Running transaction check
---> Package libedit.x86_64 0:3.0-12.20121213cvs.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
================================================================================
Package Arch Version Repository
Size
================================================================================
Installing:
openssh-clients x86_64 7.4p1-16.el7 base 655 k
Installing for dependencies:
libedit x86_64 3.0-12.20121213cvs.el7 base 92 k
Transaction Summary
================================================================================
Install 1 Package (+1 Dependent package)
Total download size: 747 k
Installed size: 2.8 M
Downloading packages:
--------------------------------------------------------------------------------
Total 940 kB/s | 747 kB 00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : libedit-3.0-12.20121213cvs.el7.x86_64 1/2
Installing : openssh-clients-7.4p1-16.el7.x86_64 2/2
Verifying : openssh-clients-7.4p1-16.el7.x86_64 1/2
Verifying : libedit-3.0-12.20121213cvs.el7.x86_64 2/2
Installed:
openssh-clients.x86_64 0:7.4p1-16.el7
Dependency Installed:
libedit.x86_64 0:3.0-12.20121213cvs.el7
Complete!
Removing intermediate container 8d49b5c61e8b
---> 6d7599dc2e56
Step 6/12 : RUN echo "root:root" | chpasswd
---> Running in 466cd81a8a2f
Removing intermediate container 466cd81a8a2f
---> 1d84010bd0d0
Step 7/12 : RUN echo "root ALL=(ALL) ALL" >> /etc/sudoers
---> Running in 238de2007a1e
Removing intermediate container 238de2007a1e
---> bdd3f6da541e
Step 8/12 : RUN ssh-keygen -t dsa -f /etc/ssh/ssh_host_dsa_key
---> Running in 369328c8512b
Enter passphrase (empty for no passphrase): Enter same passphrase again: Generating public/private dsa key pair.
Your identification has been saved in /etc/ssh/ssh_host_dsa_key.
Your public key has been saved in /etc/ssh/ssh_host_dsa_key.pub.
The key fingerprint is:
SHA256:aba5zQjMo9orNV5Jym8HyyQ+3pgWb74oy04bmEz5MuE root@369328c8512b
The key's randomart image is:
+---[DSA 1024]----+
| |
| |
| |
| . . . |
| + . o .S |
|+oo Bo=o o |
|oEo= X=oo |
| o=+=*Bo.= |
| .*BO*+oo o |
+----[SHA256]-----+
Removing intermediate container 369328c8512b
---> 41ee2c8915b2
Step 9/12 : RUN ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key
---> Running in d820dfa82fb6
Enter passphrase (empty for no passphrase): Enter same passphrase again: Generating public/private rsa key pair.
Your identification has been saved in /etc/ssh/ssh_host_rsa_key.
Your public key has been saved in /etc/ssh/ssh_host_rsa_key.pub.
The key fingerprint is:
SHA256:qF+kJk9RQ0htp+MqzqhB/0+7z1S/LKaHoELbON2xjAk root@d820dfa82fb6
The key's randomart image is:
+---[RSA 2048]----+
| ..o. |
| ..o . |
| .oo |
| oo. |
| . o.S. . |
|. .E . *. . . |
|. ..O Xo=.. . |
| . Bo%o=+. +. . |
|....=o++o++ .o |
+----[SHA256]-----+
Removing intermediate container d820dfa82fb6
---> 4a3db8fa011f
Step 10/12 : RUN mkdir /var/run/sshd
---> Running in a8fd545374b4
Removing intermediate container a8fd545374b4
---> fe17d8a2889a
Step 11/12 : EXPOSE 22
---> Running in db2ad3e4383f
Removing intermediate container db2ad3e4383f
---> c3714cf390af
Step 12/12 : CMD ["/usr/sbin/sshd", "-D"]
---> Running in fbb7e94e8ef4
Removing intermediate container fbb7e94e8ef4
---> f957172bef4b
Successfully built f957172bef4b
Successfully tagged admin/centos-ssh-root:latest
查詢剛纔構建成功的鏡像
3：基於這個鏡像再構建一個帶有jdk的鏡像
注意：jdk使用的是1.8版本的
[root@admin01 ~]# mkdir centos-ssh-root-jdk
[root@admin01 ~]# cd centos-ssh-root-jdk/
[root@admin01 centos-ssh-root-jdk]# cp ../jdk-8u162-linux-x64.tar.gz .
其中Dockerfile裏面的內容爲：
[root@admin01 centos-ssh-root-jdk]# vi Dockerfile
##添加
FROM admin/centos-ssh-root
ADD jdk-8u162-linux-x64.tar.gz /usr/local/
RUN mv /usr/local/jdk1.8.0_162 /usr/local/jdk1.8
ENV JAVA_HOME /usr/local/jdk1.8
ENV PATH $JAVA_HOME/bin:$PATH
構建命令：
[root@admin01 centos-ssh-root-jdk]# docker build -t="admin/centos-ssh-root-jdk" .
Sending build context to Docker daemon 189.8MB
Step 1/5 : FROM admin/centos-ssh-root
---> f957172bef4b
Step 2/5 : ADD jdk-8u162-linux-x64.tar.gz /usr/local/
---> 4c6b1cb6fea0
Step 3/5 : RUN mv /usr/local/jdk1.8.0_162 /usr/local/jdk1.8
---> Running in c9cde9060bc4
Removing intermediate container c9cde9060bc4
---> 83a08f6c8444
Step 4/5 : ENV JAVA_HOME /usr/local/jdk1.8
---> Running in faabe30cc8af
Removing intermediate container faabe30cc8af
---> 8ce4aeda5c6e
Step 5/5 : ENV PATH $JAVA_HOME/bin:$PATH
---> Running in a6695cfd7862
Removing intermediate container a6695cfd7862
---> 39bab29b31af
Successfully built 39bab29b31af
Successfully tagged admin/centos-ssh-root-jdk:latest
查詢構建成功的鏡像
4：基於這個jdk鏡像再構建一個帶有hadoop的鏡像
注意：hadoop使用的是2.7.3版本的。
[root@admin01 ~]#
[root@admin01 ~]# mkdir centos-ssh-root-jdk-hadoop
[root@admin01 ~]# cd centos-ssh-root-jdk-hadoop/
[root@admin01 centos-ssh-root-jdk-hadoop]# cp ../hadoop-2.7.3.tar.gz .
[root@admin01 centos-ssh-root-jdk-hadoop]# vi Dockerfile
其中，Dockerfile裏面的內容爲：
[root@admin01 centos-ssh-root-jdk-hadoop]# vi Dockerfile
## 添加
FROM admin/centos-ssh-root-jdk
ADD hadoop-2.7.3.tar.gz /usr/local
RUN mv /usr/local/hadoop-2.7.3 /usr/local/hadoop
ENV HADOOP_HOME /usr/local/hadoop
ENV PATH $HADOOP_HOME/bin:$PATH
構建命令：
[root@admin01 centos-ssh-root-jdk-hadoop]# docker build -t="admin/centos-ssh-root-jdk-hadoop" .
Sending build context to Docker daemon 214.1MB
Step 1/5 : FROM admin/centos-ssh-root-jdk
---> 39bab29b31af
Step 2/5 : ADD hadoop-2.7.3.tar.gz /usr/local
---> a565f87323c1
Step 3/5 : RUN mv /usr/local/hadoop-2.7.3 /usr/local/hadoop
---> Running in 5f07e5a1d490
Removing intermediate container 5f07e5a1d490
---> ff2197105cde
Step 4/5 : ENV HADOOP_HOME /usr/local/hadoop
---> Running in 11d8a893b137
Removing intermediate container 11d8a893b137
---> 46061a36967e
Step 5/5 : ENV PATH $HADOOP_HOME/bin:$PATH
---> Running in acfd517cb020
Removing intermediate container acfd517cb020
---> bc32364422f6
Successfully built bc32364422f6
Successfully tagged admin/centos-ssh-root-jdk-hadoop:latest
查詢構建成功的鏡像
二、搭建hadoop分佈式集羣
1：集羣規劃
準備搭建一個具有三個節點的集羣，一主兩從
主節點：admin01 ip：192.168.47.101
從節點1：admin02 ip：192.168.47.102
從節點2：admin03 ip：192.168.47.103
但是由於docker容器重新啓動之後ip會發生變化，所以需要我們給docker設置固定ip。使用pipework給docker容器設置固定ip
2：啓動三個容器，分別作爲admin01、admin02、admin03
在宿主機上執行下面命令，給容器設置主機名和容器的名稱，並且在admin01中對外開放端口50070 和8088
[root@admin01 ~]# docker run --name admin01 --hostname admin01 -d -P -p 50070:50070 -p 8088:8088 admin/centos-ssh-root-jdk-hadoop
[root@admin01 ~]# docker run --name admin02 --hostname admin02 -d -P admin/centos-ssh-root-jdk-hadoop
[root@admin01 ~]# docker run --name admin03 --hostname admin03 -d -P admin/centos-ssh-root-jdk-hadoop
查看運行的實例：
[root@admin01 ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
4083a6ab0808 admin/centos-ssh-root-jdk-hadoop "/usr/sbin/sshd -D" 49 seconds ago Up 49 seconds 0.0.0.0:32770->22/tcp admin03
6e2e2f0a40fb admin/centos-ssh-root-jdk-hadoop "/usr/sbin/sshd -D" About a minute ago Up About a minute 0.0.0.0:32769->22/tcp admin02
d4f382e8579c admin/centos-ssh-root-jdk-hadoop "/usr/sbin/sshd -D" 2 hours ago Up 2 hours 0.0.0.0:8088->8088/tcp, 0.0.0.0:50070->50070/tcp, 0.0.0.0:32768->22/tcp admin01
3：給這三臺容器設置固定IP
1：下載pipework
下載地址：https://github.com/jpetazzo/pipework.git
2：把下載的zip包上傳到宿主機服務器上，解壓,並且修改名字
[root@admin01 ~]# unzip pipework-master.zip
Archive: pipework-master.zip
ae42f1b5fef82b3bc23fe93c95c345e7af65fef3
creating: pipework-master/
extracting: pipework-master/.gitignore
inflating: pipework-master/LICENSE
inflating: pipework-master/README.md
inflating: pipework-master/docker-compose.yml
creating: pipework-master/doctoc/
inflating: pipework-master/doctoc/Dockerfile
inflating: pipework-master/pipework
inflating: pipework-master/pipework.spec
[root@admin01 ~]# mv pipework-master pipework
[root@admin01 ~]# cp -rp pipework/pipework /usr/local/bin/
3：安裝bridge-utils
[root@admin01 ~]# yum -y install bridge-utils
4:創建網絡
[root@admin01 ~]# brctl addbr br0
[root@admin01 ~]# ip link set dev br0 up
[root@admin01 ~]# ip addr add 192.168.47.101/24 dev br0
5：給容器設置固定ip
[root@admin01 ~]# pipework br0 admin01 192.168.47.101/24
[root@admin01 ~]# pipework br0 admin02 192.168.47.102/24
[root@admin01 ~]# pipework br0 admin03 192.168.47.103/24
驗證一下，分別ping三個ip，能ping通就說明沒問題
4: 配置Hadoop集羣
先連接到admin01上，使用命令
[root@admin01 ~]# docker exec -it admin01 /bin/bash
下面的步驟就是hadoop集羣的配置過程
1：設置主機名與ip的映射，修改三臺容器：vi /etc/hosts
添加下面配置
192.168.47.101 admin01
192.168.47.102 admin02
192.168.47.103 admin03
2：設置ssh免密碼登錄
在admin01上執行下面操作