基於hadoop2.6.5搭建5個節點的分佈式集羣—（三）shell腳本實現多節點批量處理

在上個章節中我們要在5個節點上均要創建/home/hadoop/app、/home/hadoop/tools、/home/data目錄，我們是在每個節點上執行mkdir命令，這樣是不是很囉嗦，效率很低，於是採用shell腳本實現多節點批處理是明智之舉。

在這裏，我寫1個配置文件與2個shell腳本（deploy.conf、deploy.sh、runRomteCmd.sh）；

一、deploy.conf的代碼如下：

#### NOTES
# There is crontab job using this config file which would compact log files and remove old log file.
# please be  carefully while modifying this file until you know what crontab exactly do
#hdp
hadoop001,all,namenode,zookeeper,resourcemanager,

hadoop002,all,slave,namenode,zookeeper,resourcemanager,

hadoop003,all,slave,datanode,zookeeper,

hadoop004,all,slave,datanode,zookeeper,

hadoop005,all,slave,datanode,zookeeper                                                             

二、deploy.sh的代碼如下：

#!/bin/bash
#set -x

if [ $# -lt 3 ]
then
  echo "Usage: ./deply.sh srcFile(or Dir) descFile(or Dir) MachineTag"
  echo "Usage: ./deply.sh srcFile(or Dir) descFile(or Dir) MachineTag confFile"
  exit
fi

src=$1
dest=$2
tag=$3
if [ 'a'$4'a' == 'aa' ]
then
  confFile=/usr/hadoop/tools/deploy.conf
else
  confFile=$4
fi

if [ -f $confFile ]
then
  if [ -f $src ]
  then
    for server in `cat $confFile|grep -v '^#'|grep ','$tag','|awk -F',' '{print $1}'`
    do
       scp $src $server":"${dest}
    done
  elif [ -d $src ]
  then
    for server in `cat $confFile|grep -v '^#'|grep ','$tag','|awk -F',' '{print $1}'`
    do
       scp -r $src $server":"${dest}
    done
  else
      echo "Error: No source file exist"
  fi

else
  echo "Error: Please assign config file or run deploy.sh command with deploy.conf in same directory"
fi


三、runRomteCmd.sh的代碼如下：

#!/bin/bash
#set -x

if [ $# -lt 2 ]
then
  echo "Usage: ./runRemoteCmd.sh Command MachineTag"
  echo "Usage: ./runRemoteCmd.sh Command MachineTag confFile"
  exit
fi

cmd=$1
tag=$2
if [ 'a'$3'a'=='aa' ]
then
 
  confFile=/usr/hadoop/tools/deploy.conf
else
  confFile=$3
fi

if [ -f $confFile ]
then
    for server in `cat $confFile|grep -v '^#'|grep ','$tag','|awk -F',' '{print $1}'`
    do
       echo "*******************$server***************************"
       ssh $server "source /etc/profile; $cmd"
    done
else
  echo "Error: Please assign config file or run deploy.sh command with deploy.conf in same directory"
fi