Docker部署redis-sentinel高可用性(HA)解決方案

Redis-Sentinel是Redis官方推薦的高可用性(HA)解決方案，當用Redis做Master-slave的高可用方案時，假如master宕機了，Redis本身(包括它的很多客戶端)都沒有實現自動進行主備切換，而Redis-sentinel本身也是一個獨立運行的進程，它能監控多個master-slave集羣，發現master宕機後能進行自動切換。

一、封裝Docker鏡像

1、創建redis Dockerfile

操作系統鏡像採用debian:jessie-slim瘦身版
redis源碼採用最新的stable版本, http://download.redis.io/releases/ ，端口採用6379

FROM debian:jessie-slim

# add our user and group first to make sure their IDs get assigned consistently, regardless of whatever dependencies get added
RUN groupadd -r redis && useradd -r -g redis redis

# grab gosu for easy step-down from root
# https://github.com/tianon/gosu/releases
ENV GOSU_VERSION 1.11
RUN set -ex; \
	\
	fetchDeps='ca-certificates wget'; \
	apt-get update; \
	apt-get install -y --no-install-recommends $fetchDeps; \
	rm -rf /var/lib/apt/lists/*; \
	\
	dpkgArch="$(dpkg --print-architecture | awk -F- '{ print $NF }')"; \
	wget -O /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$dpkgArch"; \
	wget -O /usr/local/bin/gosu.asc "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$dpkgArch.asc"; \
	export GNUPGHOME="$(mktemp -d)"; \
	gpg --keyserver ha.pool.sks-keyservers.net --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4; \
	gpg --batch --verify /usr/local/bin/gosu.asc /usr/local/bin/gosu; \
	rm -r "$GNUPGHOME" /usr/local/bin/gosu.asc; \
	chmod +x /usr/local/bin/gosu; \
	gosu nobody true; \
	\
	apt-get purge -y --auto-remove $fetchDeps

ENV REDIS_VERSION stable
ENV REDIS_DOWNLOAD_URL http://download.redis.io/releases/redis-stable.tar.gz

# for redis-sentinel see: http://redis.io/topics/sentinel
RUN set -ex; \
	\
	buildDeps=' \
		wget \
		\
		gcc \
		libc6-dev \
		make \
	'; \
	apt-get update; \
	apt-get install -y $buildDeps --no-install-recommends; \
	rm -rf /var/lib/apt/lists/*; \
	\
	wget -O redis.tar.gz "$REDIS_DOWNLOAD_URL"; \
	mkdir -p /usr/src/redis; \
	tar -xzf redis.tar.gz -C /usr/src/redis --strip-components=1; \
	rm redis.tar.gz; \
	\
# disable Redis protected mode [1] as it is unnecessary in context of Docker
# (ports are not automatically exposed when running inside Docker, but rather explicitly by specifying -p / -P)
# [1]: https://github.com/antirez/redis/commit/edd4d555df57dc84265fdfb4ef59a4678832f6da
	grep -q '^#define CONFIG_DEFAULT_PROTECTED_MODE 1$' /usr/src/redis/src/server.h; \
	sed -ri 's!^(#define CONFIG_DEFAULT_PROTECTED_MODE) 1$!\1 0!' /usr/src/redis/src/server.h; \
	grep -q '^#define CONFIG_DEFAULT_PROTECTED_MODE 0$' /usr/src/redis/src/server.h; \
# for future reference, we modify this directly in the source instead of just supplying a default configuration flag because apparently "if you specify any argument to redis-server, [it assumes] you are going to specify everything"
# see also https://github.com/docker-library/redis/issues/4#issuecomment-50780840
# (more exactly, this makes sure the default behavior of "save on SIGTERM" stays functional by default)
	\
	make -C /usr/src/redis -j "$(nproc)"; \
	make -C /usr/src/redis install; \
	\
	rm -r /usr/src/redis; \
	\
	apt-get purge -y --auto-remove $buildDeps

RUN apt-get update && apt-get install -y vim

RUN mkdir /data && chown redis:redis /data
VOLUME /data
WORKDIR /data

RUN mkdir /etc/redis
ADD redis.conf /etc/redis/
COPY docker-entrypoint.sh /usr/local/bin/
ENTRYPOINT ["docker-entrypoint.sh"]

EXPOSE 6379
CMD ["redis-server"]

2、創建redis.conf

在redis源代碼中提供的redis.conf的基礎上進行修改

masterauth <master-password> 當master服務設置了密碼保護時，slave服務連接master的密碼
requirepass<password> 設置Redis連接密碼

masterauth redis-secret-0123passw0rd
requirepass redis-secret-0123passw0rd

關閉高危命令

# Command renaming.
#
# It is possible to change the name of dangerous commands in a shared
# environment. For instance the CONFIG command may be renamed into something
# hard to guess so that it will still be available for internal-use tools
# but not available for general clients.
#
# Example:
#
# rename-command CONFIG b840fc02d524045429941cc15f59e41cb7be6c52
#
# It is also possible to completely kill a command by renaming it into
# an empty string:
#
# 刪除所有數據庫的所有key
rename-command FLUSHALL ""

# Lua解釋器執行腳本
rename-command EVAL     ""

在dockerfile中增加如下命令：

RUN mkdir /etc/redis
ADD redis.conf /etc/redis/

替換redis源代碼中的配置文件

3、創建docker-entrypoint.sh

#!/bin/sh
set -e

# first arg is `-f` or `--some-option`
# or first arg is `something.conf`
if [ "${1#-}" != "$1" ] || [ "${1%.conf}" != "$1" ]; then
	set -- redis-server "$@"
fi

# allow the container to be started with `--user`
if [ "$1" = 'redis-server' -a "$(id -u)" = '0' ]; then
	chown -R redis .
	exec gosu redis "$0" "$@"
fi

exec "$@"

4、創建redis鏡像

按照自定義的tag創建redis鏡像

docker build -t wxsc/redis:stable .

查看鏡像信息

二、封裝redis-sentinel容器

一主兩從三哨兵，可以，如果兩個哨兵投了不一樣的，就再投一遍。

1、創建redis-sentinel Dockerfile

FROM wxsc/redis:stable

EXPOSE 26379
ADD sentinel.conf /etc/redis/sentinel.conf
RUN chown redis:redis /etc/redis/sentinel.conf
ENV SENTINEL_QUORUM 2
ENV SENTINEL_DOWN_AFTER 30000
ENV SENTINEL_FAILOVER 180000
COPY sentinel-entrypoint.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/sentinel-entrypoint.sh
ENTRYPOINT ["sentinel-entrypoint.sh"]

2、創建 sentinel-entrypoint.sh

#!/bin/sh

sed -i "s/\$SENTINEL_QUORUM/$SENTINEL_QUORUM/g" /etc/redis/sentinel.conf
sed -i "s/\$SENTINEL_DOWN_AFTER/$SENTINEL_DOWN_AFTER/g" /etc/redis/sentinel.conf
sed -i "s/\$SENTINEL_FAILOVER/$SENTINEL_FAILOVER/g" /etc/redis/sentinel.conf

exec docker-entrypoint.sh redis-server /etc/redis/sentinel.conf --sentinel

3、創建 sentinel.conf

# Example sentinel.conf can be downloaded from http://download.redis.io/redis-stable/sentinel.conf

port 26379

dir /tmp

sentinel monitor mymaster redis-master 6379 $SENTINEL_QUORUM

sentinel auth-pass mymaster redis-secret-0123passw0rd

sentinel down-after-milliseconds mymaster $SENTINEL_DOWN_AFTER

sentinel parallel-syncs mymaster 1

sentinel failover-timeout mymaster $SENTINEL_FAILOVER

4、創建docker-compose.yml

master:
  image: wxsc/redis:stable
  container_name: redis-cluster_master
  command: redis-server /etc/redis/redis.conf
  ports:
    - "6379:6379"
slave_1:
  image: wxsc/redis:stable
  container_name: redis-cluster_slave_1
  command: redis-server /etc/redis/redis.conf --slaveof redis-master 6379
  links:
    - master:redis-master
  ports:
    - "6380:6379"
slave_2:
  image: wxsc/redis:stable
  container_name: redis-cluster_slave_2
  command: redis-server /etc/redis/redis.conf --slaveof redis-master 6379
  links:
    - master:redis-master
  ports:
    - "6381:6379"
sentinel_1:
  build: sentinel
  container_name: redis-cluster_sentinel_1
  command: redis-sentinel /etc/redis/sentinel.conf
  environment:
    - SENTINEL_DOWN_AFTER=5000
    - SENTINEL_FAILOVER=5000
  links:
    - master:redis-master
    - slave_1
    - slave_2
  ports:
    - "6390:6379"
    - "26390:26379"
sentinel_2:
  build: sentinel
  container_name: redis-cluster_sentinel_2
  environment:
    - SENTINEL_DOWN_AFTER=5000
    - SENTINEL_FAILOVER=5000
  links:
    - master:redis-master
    - slave_1
    - slave_2
  ports:
    - "6391:6379"
    - "26391:26379"
sentinel_3:
  build: sentinel
  container_name: redis-cluster_sentinel_3
  environment:
    - SENTINEL_DOWN_AFTER=5000
    - SENTINEL_FAILOVER=5000
  links:
    - master:redis-master
    - slave_1
    - slave_2
  ports:
    - "6392:6379"
    - "26392:26379"

5、安裝Docker Compose

yum -y install epel-release
yum -y install python-pip
pip install -U docker-compose

錯誤： AttributeError: ‘module’ object has no attribute 'GSSException’

pip install paramiko==2.0.2

6、創建redis-sentinel鏡像

docker-compose build

三、創建redis集羣

docker-compose up -d

查看容器