顯然單機和多機部署的區別在於網絡配置,也就是怎麼進行docker容器之間的通信,所以多機部署的關鍵其實在於docker的網絡通信。
安裝HLF
五臺server,系統是Ubuntu16.04,已經配置好了ssh證書登錄,假設IP分別是
10.22.1.12 # orderer.example.com, cli
10.22.1.13 # peer0.org1.example.com
10.22.1.14 # peer1.org1.example.com
10.22.1.15 # peer0.org2.example.com
10.22.1.16 # peer1.org2.example.com
如果沒有特殊說明,以下命令均在10.22.1.12
機器上/home/myuser/
目錄下進行。
創建一個host.txt
文件,方便動態修改部署到其它機器:
vim hosts.txt
10.22.1.12 # orderer.example.com, cli
10.22.1.13 # peer0.org1.example.com
10.22.1.14 # peer1.org1.example.com
10.22.1.15 # peer0.org2.example.com
10.22.1.16 # peer1.org2.example.com
安裝curl、docker、docker-composer、HLF鏡像(1.3的版本,如需其它版本自行替換腳本中的版本號)的腳本install.sh
,需要用戶密碼作爲第一個參數:
#!/bin/bash
if [ $# -eq 0 ]; then
echo "please input password..."
exit 1
fi
cat hosts.txt | while read host
do
echo ' '
echo ' '
echo ' '
echo '################################'
echo '##########' $host '##########'
echo '################################'
echo ' '
echo ' '
echo ' '
ssh myuser@$host << end
echo $1 | sudo apt install curl -y
sudo -S apt-get remove docker docker-engine docker.io containerd runc -y
docker -v
sudo apt-get update -y
sudo apt-get install \
apt-transport-https \
ca-certificates \
curl \
gnupg2 \
software-properties-common -y
curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
sudo add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/debian \
$(lsb_release -cs) \
stable"
sudo apt-get update -y
sudo apt-get install docker-ce docker-ce-cli containerd.io -y
docker run hello-world
docker -v
if [ $? -ne 0 ]; then
echo "##################################################"
echo "some wrong when install docker!"
echo "##################################################"
exit 1
fi
echo $1 | sudo -S curl -L "https://github.com/docker/compose/releases/download/1.25.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
sudo rm /usr/bin/docker-compose
sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
docker-compose -v
if [ $? -ne 0 ]; then
echo "##################################################"
echo "some wrong when install docker-compose!"
echo "##################################################"
exit 1
fi
curl -sSL http://bit.ly/2ysbOFE | bash -s 1.3.0
if [ $? -ne 0 ]; then
echo "##################################################"
echo "some wrong when pull the images"
echo "##################################################"
exit 1
fi
end
done
echo done!
到這裏不出意外的話就算是配置好了,可以到/home/myuser/fabric-samples/first-network/
目錄下測試是否安裝成功:
啓動網絡:
./byfn.sh up
如果看到下面的輸出就是安裝成功了:
90
===================== Query successful on peer1.org2 on channel 'mychannel' =====================
========= All GOOD, BYFN execution completed ===========
_____ _ _ ____
| ____| | \ | | | _ \
| _| | \| | | | | |
| |___ | |\ | | |_| |
|_____| |_| \_| |____/
關閉並清理網絡(每次重啓網絡前都要記得清理,否則會報錯):
./byfn.sh down
配置docker網絡
據說可以使用
host
模式進行部署,簡單很多,但是我使用host模式時在實例化chaincode的時候總是報錯,沒找到解決方案,於是採用了overlay
模式。
docker swarm init --advertise-addr=10.22.1.12 # 在10.22.1.12上初始化swarm
docker swarm join-token manager # 這裏會輸出一個命令docker swarm join --token ... ,複製下來,去其他機器上執行
ssh [email protected]
docker swarm join --token ...
ssh [email protected]
docker swarm join --token ...
ssh [email protected]
docker swarm join --token ...
ssh [email protected]
docker swarm join --token ...
# 回到10.22.1.12
docker network create --attachable --driver overlay HLF # 創建一個overlay網絡
這時查看docker的網絡docker network ls
,大概會是這樣:
NETWORK ID NAME DRIVER SCOPE
ntubodu3k0fp HLF overlay swarm
31158df52877 bridge bridge local
54ad61772123 docker_gwbridge bridge local
d8b38ea6fbed host host local
rlzokyfpla8r ingress overlay swarm
e97a070185d2 none null local
其他機器上也是一樣,它們會自動創建。
創建啓動文件
在/home/myuser/fabric-samples/first-network
目錄下有一個docker-compose-cli.yaml
文件,主要根據這個文件進行修改。
將這個文件分成5個,一個包含orderer和cli,另外四個分別是四個peer。
.
├── base
│ ├── docker-compose-base.yaml
│ └── peer-base.yaml
├── docker-compose-cli.yaml
├── orderer_cli.yaml
├── peer01.yaml
├── peer02.yaml
├── peer11.yaml
└── peer12.yaml
主要修改內容是將networks修改爲我們之前創建的網絡HLF,不過需要注意指定external爲true,否則docker會自動重新創建一個網絡。下面是全部的文件:
orderer_cli.yaml:
# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#
version: '2'
volumes:
orderer.example.com:
networks:
HLF:
external: true ## important!!!!!
services:
orderer.example.com:
extends:
file: base/docker-compose-base.yaml
service: orderer.example.com
container_name: orderer.example.com
#network_mode: overlay
networks:
- HLF
# extra_hosts:
# - "orderer.example.com:10.22.1.12"
# - "peer0.org1.example.com:10.22.1.13"
# - "peer1.org1.example.com:10.22.1.14"
# - "peer0.org2.example.com:10.22.1.15"
# - "peer1.org2.example.com:10.22.1.16"
cli:
container_name: cli
image: hyperledger/fabric-tools:$IMAGE_TAG
tty: true
stdin_open: true
environment:
- GOPATH=/opt/gopath
- CORE_VM_ENDPOINT=unix:///host/var/run/docker.sock
#- CORE_LOGGING_LEVEL=DEBUG
- CORE_LOGGING_LEVEL=INFO
- CORE_PEER_ID=cli
- CORE_PEER_ADDRESS=peer0.org1.example.com:7051
- CORE_PEER_LOCALMSPID=Org1MSP
- CORE_PEER_TLS_ENABLED=true
- CORE_PEER_TLS_CERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/server.crt
- CORE_PEER_TLS_KEY_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/server.key
- CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt
- CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/users/[email protected]/msp
working_dir: /opt/gopath/src/github.com/hyperledger/fabric/peer
command: /bin/bash
volumes:
- /var/run/:/host/var/run/
- ./../chaincode/:/opt/gopath/src/github.com/chaincode
- ./crypto-config:/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/
- ./scripts:/opt/gopath/src/github.com/hyperledger/fabric/peer/scripts/
- ./channel-artifacts:/opt/gopath/src/github.com/hyperledger/fabric/peer/channel-artifacts
networks:
- HLF
# extra_hosts:
# - "orderer.example.com:10.22.1.12"
# - "peer0.org1.example.com:10.22.1.13"
# - "peer1.org1.example.com:10.22.1.14"
# - "peer0.org2.example.com:10.22.1.15"
# - "peer1.org2.example.com:10.22.1.16"
peer01.yaml:
# Copyright IBM Corp. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0
#
version: '2'
volumes:
peer0.org1.example.com:
networks:
HLF:
external: true
services:
peer0.org1.example.com:
container_name: peer0.org1.example.com
extends:
file: base/docker-compose-base.yaml
service: peer0.org1.example.com
#network_mode: overlay
#networks:
# - byfn
networks:
- HLF
peer02.yaml:
# Copyright IBM Corp. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0
#
version: '2'
volumes:
peer0.org1.example.com:
networks:
HLF:
external: true
services:
peer0.org1.example.com:
container_name: peer0.org1.example.com
extends:
file: base/docker-compose-base.yaml
service: peer0.org1.example.com
#network_mode: overlay
#networks:
# - byfn
networks:
- HLF
peer11.yaml:
# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#
version: '2'
volumes:
peer0.org2.example.com:
networks:
HLF:
external: true
services:
peer0.org2.example.com:
container_name: peer0.org2.example.com
extends:
file: base/docker-compose-base.yaml
service: peer0.org2.example.com
#network_mode: overlay
#networks:
# - byfn
networks:
- HLF
# extra_hosts:
# - "orderer.example.com:10.22.1.12"
# - "peer0.org1.example.com:10.22.1.13"
# - "peer1.org1.example.com:10.22.1.14"
# - "peer0.org2.example.com:10.22.1.15"
# - "peer1.org2.example.com:10.22.1.16"
peer12.yaml:
# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
version: '2'
volumes:
peer1.org2.example.com:
networks:
HLF:
external: true
services:
peer1.org2.example.com:
container_name: peer1.org2.example.com
extends:
file: base/docker-compose-base.yaml
service: peer1.org2.example.com
#network_mode: overlay
# networks:
# - byfn
networks:
- HLF
# extra_hosts:
# - "orderer.example.com:10.22.1.12"
# - "peer0.org1.example.com:10.22.1.13"
# - "peer1.org1.example.com:10.22.1.14"
# - "peer0.org2.example.com:10.22.1.15"
# - "peer1.org2.example.com:10.22.1.16"
除此之外還需要修改/home/myuser/fabric-samples/first-network
目錄下的兩個文件:
base
├── docker-compose-base.yaml
└── peer-base.yaml
peer-base.yaml中的- CORE_VM_DOCKER_HOSTCONFIG_NETWORKMODE=${COMPOSE_PROJECT_NAME}_byfn
修改爲- CORE_VM_DOCKER_HOSTCONFIG_NETWORKMODE=HLF
。
docker-compose-base.yaml中的端口映射部分:
- 7051:7051
- 7053:7053
- 8051:7051
- 8053:7053
- 9051:7051
- 9053:7053
- 10051:7051
- 10053:7053
修改爲
- 7051:7051
- 7053:7053
- 7051:7051
- 7053:7053
- 7051:7051
- 7053:7053
- 7051:7051
- 7053:7053
到此修改完成。
這時可以參考官網的命令去生成證書文件和通道配置文件,腳本如下:
#!/bin/zsh
cd /home/myuser/fabric-samples/first-network
# clear the old X.509
rm -rf crypto-config
rm -rf channel-artifacts/*
# generate X.509 using the default configuration
cryptogen generate --config=crypto-config.yaml
# tell the tool where to look for the configtx.yaml
export FABRIC_CFG_PATH=$PWD
export CHANNEL_NAME=mychannel
# create the orderer genesisi block
configtxgen -profile TwoOrgsOrdererGenesis -outputBlock ./channel-artifacts/genesis.block #-channelID $CHANNEL_NAME
# create a channel configuration transaction
configtxgen -profile TwoOrgsChannel -outputCreateChannelTx ./channel-artifacts/channel.tx -channelID $CHANNEL_NAME
# define the anchor peer for Org1
configtxgen -profile TwoOrgsChannel -outputAnchorPeersUpdate ./channel-artifacts/Org1MSPanchors.tx -channelID $CHANNEL_NAME -asOrg Org1MSP
# define the anchor peer for Org2
configtxgen -profile TwoOrgsChannel -outputAnchorPeersUpdate ./channel-artifacts/Org2MSPanchors.tx -channelID $CHANNEL_NAME -asOrg Org2MSP
將修改的7個yaml文件和證書crypto-config
通過scp傳送到對應的機器,對應的位置。
運行
啓動腳本:./run.sh
#!/bin/bash
peer01=10.22.1.13
peer11=10.22.1.14
peer02=10.22.1.15
peer12=10.22.1.16
cd ~/fabric-samples/first-network
docker-compose -f orderer.yaml up -d
ssh myuser@$peer01 "cd fabric-samples/first-network; docker-compose -f peer01.yml up -d"
ssh myuser@$peer11 "cd fabric-samples/first-network; docker-compose -f peer11.yml up -d"
ssh myuser@$peer02 "cd fabric-samples/first-network; docker-compose -f peer02.yml up -d"
ssh myuser@$peer12 "cd fabric-samples/first-network; docker-compose -f peer12.yml up -d"
進入cli容器
:docker exec -it cli bash
在cli容器
中執行腳本進行測試: bash scripts/script.sh
看到如下輸出說明成功:
90
===================== Query successful on peer1.org2 on channel 'mychannel' =====================
========= All GOOD, BYFN execution completed ===========
_____ _ _ ____
| ____| | \ | | | _ \
| _| | \| | | | | |
| |___ | |\ | | |_| |
|_____| |_| \_| |____/
可能出現的錯誤
- 拉取鏡像的時候,可能會報
curl: (56) recv failure: connection reset by peer
:將install.sh
中的curl -sSL http://bit.ly/2ysbOFE | bash -s 1.3.0
改爲curl -sSL https://bit.ly/2ysbOFE | bash -s 1.3.0
即可 - 在
cli容器
中執行腳本進行測試的時候可能會有各種情況,這個時候就先檢查網絡是否連通,然後檢查證書文件是否有問題,…。