交付dubbo 微服務到k8s 集羣

dubbo 微服務結構簡介

上圖中，藍色的表示與業務有交互，綠色的表示只對Dubbo內部交互。上述圖所描述的調用流程如下：

服務提供方發佈服務到服務註冊中心；
服務消費方從服務註冊中心訂閱服務；
服務消費方調用已經註冊的可用服務

1. 部署maven 並改成國內源

在 7.200 上：
tar xf apache-maven-3.6.0-bin.tar.gz -C  /data/nfs-volume/jenkins_home/maven-3.6.3

修改maven 倉庫爲阿里雲倉庫

 <mirrors>
  
   <mirror>
     <id>nexus-aliyun</id>
     <mirrorOf>*</mirrorOf>
     <name>Nexus aliyun</name>
     <url>http://maven.aliyun.com/nexus/content/groups/public</url>
   </mirror>
  </mirrors>

#修改成阿里雲鏡像源

2. 準備 dubbo 底層鏡像

docker pull docker.io/stanleyws/jre8:8u112  #這個是jdk8的原始鏡像
 docker tag  docker.io/stanleyws/jre8:8u112 harbor.od.com/public/jre:8u112
 docker push  harbor.od.com/public/jre:8u112

cat <<EOF >/data/dockerfile/jre8/Dockerfile
FROM harbor.od.com/public/jre:8u112   
RUN /bin/cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime &&\
    echo 'Asia/Shanghai' >/etc/timezone
ADD config.yml /opt/prom/config.yml
ADD jmx_javaagent-0.3.1.jar /opt/prom/
WORKDIR /opt/project_dir
ADD entrypoint.sh /entrypoint.sh
CMD ["/entrypoint.sh"]
EOF

cat <<EOF>config.yml
---
rules:
  - pattern: '.*'
EOF 

#下載java-agent 用來給普羅米修斯監控
wget https://repo1.maven.org/maven2/io/prometheus/jmx/jmx_prometheus_javaagent/0.3.1/jmx_prometheus_javaagent-0.3.1.jar -O jmx_javaagent-0.3.1.jar


vim entrypoint.sh
#!/bin/sh
M_OPTS="-Duser.timezone=Asia/Shanghai-javaagent:/opt/prom/jmx_javaagent-0.3.1.jar=$(hostname-i):${M_PORT:-"12346"}:/opt/prom/config.yml"
C_OPTS=${C_OPTS}
JAR_BALL=${JAR_BALL}
exec java -jar ${M_OPTS} ${C_OPTS} ${JAR_BALL}

chmod +x entrypoint.sh
#登錄harbor 創建base 倉庫，權限是public
]# docker build . -t harbor.od.com/base/jre8:8u112 && docker push  harbor.od.com/base/jre8:8u112

3.交付dubbo server 到k8s 集羣

在 jenkins 中新建一個pipline的項目，名字自定義，選擇參數構建

#然後到Jenkins 新建pipline 工程，在general ，選擇 this project isparameterized。

選擇string parameter 。勾選 trim is string ,分別是：

app_name image_name git_repo git_ver add_tag target_dir (./target) mvn_dir (./)

mvn_cmd (mvn clean package -Dmaven.test.skip=true)

# Choice parameter

base_image (base/jre8:8u112)

maven (3.6.3) #執行編譯的軟件包版本,要根據實際mvn 寫。

以此填入自定義的參數，我這裏放個填好後的圖，就是 app_name image_name 那一列：

然後在jenkins pipeline 下面填入pipeline 腳本

pipeline {
  agent any 
    stages {
      stage('pull') { //get project code from repo 
        steps {
          sh "git clone ${params.git_repo} ${params.app_name}/${env.BUILD_NUMBER} && cd ${params.app_name}/${env.BUILD_NUMBER} && git checkout ${params.git_ver}"
        }
      }
      stage('build') { //exec mvn cmd
        steps {
          sh "cd ${params.app_name}/${env.BUILD_NUMBER}  && /var/jenkins_home/maven-${params.maven}/bin/${params.mvn_cmd}"
        }
      }
      stage('package') { //move jar file into project_dir
        steps {
          sh "cd ${params.app_name}/${env.BUILD_NUMBER} && cd ${params.target_dir} && mkdir project_dir && mv *.jar ./project_dir"
        }
      }
      stage('image') { //build image and push to registry
        steps {
          writeFile file: "${params.app_name}/${env.BUILD_NUMBER}/Dockerfile", text: """FROM harbor.od.com/${params.base_image}
ADD ${params.target_dir}/project_dir /opt/project_dir"""#這一步是把 FROM harbor.od.com/${params.base_image}
#ADD ${params.target_dir}/project_dir /opt/project_dir  具體內容寫到jenkins 裏面dockerfile 裏面 然後在jenkins 裏面構建有了新的jar 包的鏡像 並推送到
#habor 上，鏡像的名字後綴是剛纔的add_tag 也就是時間
           sh "cd  ${params.app_name}/${env.BUILD_NUMBER} && docker build -t harbor.od.com/${params.image_name}:${params.git_ver}_${params.add_tag} . && docker push harbor.od.com/${params.image_name}:${params.git_ver}_${params.add_tag}"
        }
      }
    }
}

　保存，之後再項目項目裏面選擇參數構建：

具體的參數如下：

　git_repo 要填dubbo-server 的git 倉庫地址，git_ver 就是代碼分支，這裏拉取的 apolo 分支的代碼。

add_tag 建議用時間日期 base_image 就是上一步構建的dockerfile 基礎鏡像 maven 就是之前部署的，名字要和 7.200 /data/nfs-volume/jenkins_home/ 裏面 maven 的目錄名字一致，要不jenkins 構建時候會找不到，類似如下報錯

+ /var/jenkins_home/maven-3.6.3/bin/mvn clean package -Dmaven.test.skip=true
/var/jenkins_home/workspace/dubboproject1@tmp/durable-700da7c2/script.sh: 1: /var/jenkins_home/workspace/dubboproject1@tmp/durable-700da7c2/script.sh: /var/jenkins_home/maven-3.6.3/bin/mvn: not found

　　然後點擊構建，成功後結果如下，這時候登陸harbor 就會發現多了一個

app/dubbo-demo-service:apollo_20200517_1600 的鏡像了！

4. 準備dubbo server 的yaml 文件

[root@hdss7-200 dubbo-server]# cat dp.yaml 
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: dubbo-demo-service
  namespace: app
  labels: 
    name: dubbo-demo-service
spec:
  replicas: 1
  selector:
    matchLabels: 
      name: dubbo-demo-service
  template:
    metadata:
      labels: 
        app: dubbo-demo-service
        name: dubbo-demo-service
    spec:
      containers:
      - name: dubbo-demo-service
        image: harbor.od.com/app/dubbo-demo-service:apollo_20200517_1600 #就是剛纔新構建的鏡像
        ports:
        - containerPort: 20880
          protocol: TCP
        env:
        - name: JAR_BALL
          value: dubbo-server.jar
        - name: C_OPTS
          value: -Denv=dev -Dapollo.meta=http://config.od.com    #這裏結合apolo 配置中心來做的
        imagePullPolicy: IfNotPresent
      imagePullSecrets:
      - name: harbor
      restartPolicy: Always
      terminationGracePeriodSeconds: 30
      securityContext: 
        runAsUser: 0
      schedulerName: default-scheduler
  strategy:
    type: RollingUpdate
    rollingUpdate: 
      maxUnavailable: 1
      maxSurge: 1
  revisionHistoryLimit: 7
  progressDeadlineSeconds: 600

5 交付dubbo-monitor 到k8s 集羣

#先下載dubbo monitor 包

https://github.com/Jeromefromcn/dubbo-monitor#dubbo-monitor ，下載後解壓，

修改配置

cat /opt/src/dubbo-monitor/dubbo-monitor-simple/conf/dubbo_origin.properties
dubbo.container=log4j,spring,registry,jetty
dubbo.application.name=simple-monitor
dubbo.application.owner=
dubbo.registry.address=zookeeper://10.4.7.11:2181 # 改成實際zk 地址
dubbo.protocol.port=20880
dubbo.jetty.port=8080
dubbo.jetty.directory=/dubbo-monitor-simple/monitor
dubbo.statistics.directory=/dubbo-monitor-simple/statistics
dubbo.charts.directory=/dubbo-monitor-simple/charts
dubbo.log4j.file=logs/dubbo-monitor.log

　準備dockerfile

vim /data/dockerfile/dubbo-monitor/Dockerfile
FROM jeromefromcn/docker-alpine-java-bash
MAINTAINER Jerome Jiang
COPY dubbo-monitor-simple/ /dubbo-monitor-simple/   #dubbo-monitor-simple/ 就是dubbo-monitor 包裏面的文件夾
CMD /dubbo-monitor-simple/bin/start.sh

docker build . -t harbor.od.com/infra/dubbo-monitor:latest #構建鏡像

因爲github 上 start.sh 腳本在容器環境中啓動有問題需要修改，下面是修改後的。

[root@hdss7-200 dubbo-monitor]# cat dubbo-monitor-simple/bin/start.sh 
#!/bin/bash
sed -e "s/{ZOOKEEPER_ADDRESS}/$ZOOKEEPER_ADDRESS/g" /dubbo-monitor-simple/conf/dubbo_origin.properties > /dubbo-monitor-simple/conf/dubbo.properties
cd `dirname $0`
BIN_DIR=`pwd`
cd ..
DEPLOY_DIR=`pwd`
CONF_DIR=$DEPLOY_DIR/conf

SERVER_NAME=`sed '/dubbo.application.name/!d;s/.*=//' conf/dubbo.properties | tr -d '\r'`
SERVER_PROTOCOL=`sed '/dubbo.protocol.name/!d;s/.*=//' conf/dubbo.properties | tr -d '\r'`
SERVER_PORT=`sed '/dubbo.protocol.port/!d;s/.*=//' conf/dubbo.properties | tr -d '\r'`
LOGS_FILE=`sed '/dubbo.log4j.file/!d;s/.*=//' conf/dubbo.properties | tr -d '\r'`

if [ -z "$SERVER_NAME" ]; then
    SERVER_NAME=`hostname`
fi

PIDS=`ps -f | grep java | grep "$CONF_DIR" |awk '{print $2}'`
if [ -n "$PIDS" ]; then
    echo "ERROR: The $SERVER_NAME already started!"
    echo "PID: $PIDS"
    exit 1
fi

if [ -n "$SERVER_PORT" ]; then
    SERVER_PORT_COUNT=`netstat -tln | grep $SERVER_PORT | wc -l`
    if [ $SERVER_PORT_COUNT -gt 0 ]; then
        echo "ERROR: The $SERVER_NAME port $SERVER_PORT already used!"
        exit 1
    fi
fi

LOGS_DIR=""
if [ -n "$LOGS_FILE" ]; then
    LOGS_DIR=`dirname $LOGS_FILE`
else
    LOGS_DIR=$DEPLOY_DIR/logs
fi
if [ ! -d $LOGS_DIR ]; then
    mkdir $LOGS_DIR
fi
STDOUT_FILE=$LOGS_DIR/stdout.log

LIB_DIR=$DEPLOY_DIR/lib
LIB_JARS=`ls $LIB_DIR|grep .jar|awk '{print "'$LIB_DIR'/"$0}'|tr "\n" ":"`

JAVA_OPTS=" -Djava.awt.headless=true -Djava.net.preferIPv4Stack=true "
JAVA_DEBUG_OPTS=""
if [ "$1" = "debug" ]; then
    JAVA_DEBUG_OPTS=" -Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=n "
fi
JAVA_JMX_OPTS=""
if [ "$1" = "jmx" ]; then
    JAVA_JMX_OPTS=" -Dcom.sun.management.jmxremote.port=1099 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false "
fi
JAVA_MEM_OPTS=""
BITS=`java -version 2>&1 | grep -i 64-bit`
if [ -n "$BITS" ]; then
    JAVA_MEM_OPTS=" -server -Xmx128m -Xms128m -Xmn32m -XX:PermSize=32m -Xss256k -XX:+DisableExplicitGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:+UseCMSCompactAtFullCollection -XX:LargePageSizeInBytes=128m -XX:+UseFastAccessorMethods -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 "
else
    JAVA_MEM_OPTS=" -server -Xms128m -Xmx128m -XX:PermSize=16m -XX:SurvivorRatio=2 -XX:+UseParallelGC "
fi

echo -e "Starting the $SERVER_NAME ...\c"
exec java $JAVA_OPTS $JAVA_MEM_OPTS $JAVA_DEBUG_OPTS $JAVA_JMX_OPTS -classpath $CONF_DIR:$LIB_JARS com.alibaba.dubbo.container.Main > $STDOUT_FILE 2>&1

6. 準備dobbo monitor 文件

cat deployment.yaml 
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: dubbo-monitor
  namespace: infra
  labels: 
    name: dubbo-monitor
spec:
  replicas: 1
  selector:
    matchLabels: 
      name: dubbo-monitor
  template:
    metadata:
      labels: 
        app: dubbo-monitor
        name: dubbo-monitor
    spec:
      containers:
      - name: dubbo-monitor
        image: harbor.od.com/infra/dubbo-monitor:latest
        ports:
        - containerPort: 8080
          protocol: TCP
        - containerPort: 20880
          protocol: TCP
        imagePullPolicy: IfNotPresent
      imagePullSecrets:
      - name: harbor
      restartPolicy: Always
      terminationGracePeriodSeconds: 30
      securityContext: 
        runAsUser: 0
      schedulerName: default-scheduler
  strategy:
    type: RollingUpdate
    rollingUpdate: 
      maxUnavailable: 1
      maxSurge: 1
  revisionHistoryLimit: 7
  progressDeadlineSeconds: 600

        
[root@hdss7-200 dubbo-monitor]# cat cm.yaml 
apiVersion: v1
kind: ConfigMap
metadata:
  name: dubbo-monitor-cm
  namespace: infra
data:
  dubbo.properties:        |
    dubbo.container=log4j,spring,registry,jetty
    dubbo.application.name=simple-monitor
    dubbo.application.owner=simplemonitor
    dubbo.registry.address=zookeeper://10.4.7.11:2181  #zk 的地址一定要和代碼裏面保持一致！
    dubbo.protocol.port=20880
    dubbo.jetty.port=8080
    dubbo.jetty.directory=/dubbo-monitor-simple/monitor
    dubbo.charts.directory=/dubbo-monitor-simple/charts
    dubbo.statistics.directory=/dubbo-monitor-simple/statistics
    dubbo.log4j.file=/dubbo-monitor-simple/logs/dubbo-monitor.log
    dubbo.log4j.level=WARN
[root@hdss7-200 dubbo-monitor]# cat ingress.yaml 
kind: Ingress
apiVersion: extensions/v1beta1
metadata: 
  name: dubbo-monitor
  namespace: infra
spec:
  rules:
  - host: dubbo-monitor.od.com
    http:
      paths:
      - path: /
        backend: 
          serviceName: dubbo-monitor
          servicePort: 8080
[root@hdss7-200 dubbo-monitor]# cat svc.yaml 
kind: Service
apiVersion: v1
metadata: 
  name: dubbo-monitor
  namespace: infra
spec:
  ports:
  - protocol: TCP
    port: 8080
    targetPort: 8080
  selector: 
    app: dubbo-monitor

添加A 記錄

dubbo-monitor.od.com IN A  10.4.7.10

測試訪問 
http://dubbo-monitor.od.com

7.交付 dubbo服務消費者到K8S 集羣

jenkins 上新構建一個pipeline 項目，添加的參數和dubbo-serve 一樣,，可以直接把dubbo-server 複製過來，然後填入參數開始構建，參數如下圖：

看下 console output

Step 1/2 : FROM harbor.od.com/base/jre8:8u112
 ---> 621a01f7f85a
Step 2/2 : ADD ./dubbo-client/target/project_dir /opt/project_dir
 ---> 7d5577992ecd
Successfully built 7d5577992ecd
Successfully tagged harbor.od.com/app/dubbo-demo-consumer:apollo_20200517_1650
+ docker push harbor.od.com/app/dubbo-demo-consumer:apollo_20200517_1650
The push refers to repository [harbor.od.com/app/dubbo-demo-consumer]

會看到harbor 上多了  app/dubbo-demo-consumer:apollo_20200517_1650 的鏡像

8. 準備consumer 的yaml 文件

# dubbo-consumer 資源配置清單

dp.yaml
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: dubbo-demo-consumer
  namespace: app
  labels: 
    name: dubbo-demo-consumer
spec:
  replicas: 1
  selector:
    matchLabels: 
      name: dubbo-demo-consumer
  template:
    metadata:
      labels: 
        app: dubbo-demo-consumer
        name: dubbo-demo-consumer
    spec:
      containers:
      - name: dubbo-demo-consumer
        image: harbor.od.com/app/dubbo-demo-consumer:apollo_20200517_1650 #就是剛纔新構建的consumer 鏡像
        ports:
        - containerPort: 8080
          protocol: TCP
        - containerPort: 20880
          protocol: TCP
        env:
        - name: JAR_BALL
          value: dubbo-client.jar
        imagePullPolicy: IfNotPresent
      imagePullSecrets:
      - name: harbor
      restartPolicy: Always
      terminationGracePeriodSeconds: 30
      securityContext: 
        runAsUser: 0
      schedulerName: default-scheduler
  strategy:
    type: RollingUpdate
    rollingUpdate: 
      maxUnavailable: 1
      maxSurge: 1
  revisionHistoryLimit: 7
  progressDeadlineSeconds: 600

cat svc.yaml

kind: Service
apiVersion: v1
metadata: 
  name: dubbo-demo-consumer
  namespace: app
spec:
  ports:
  - protocol: TCP
    port: 8080
    targetPort: 8080
  selector: 
    app: dubbo-demo-consumer
	
cat ingress.yaml

kind: Ingress
apiVersion: extensions/v1beta1
metadata: 
  name: dubbo-demo-consumer
  namespace: app
spec:
  rules:
  - host: demo.od.com
    http:
      paths:
      - path: /
        backend: 
          serviceName: dubbo-demo-consumer
          servicePort: 8080

kubectl apply -f 這幾個文件，然後 訪問 http://demo.od.com/hello?name=hello 看是否有返回結果，有返回就是正常的。

登錄monitor 查看：可以看到 server 和 consumer 都有了。

交付dubbo 微服務到k8s 集羣

dubbo 微服務結構簡介

1. 部署maven 並改成國內源

2. 準備 dubbo 底層鏡像

3.交付dubbo server 到k8s 集羣

4. 準備dubbo server 的yaml 文件

5 交付dubbo-monitor 到k8s 集羣

6. 準備dobbo monitor 文件

7.交付 dubbo服務消費者到K8S 集羣

8. 準備consumer 的yaml 文件

AI模型 Llama 3體驗筆記

【面試準備】又一次失敗的面試經歷，題目離譜～資深軟件測試工程師

dotnet 8 版本與銀河麒麟V10和UOS系統的 glibc 兼容性

用django 寫個用戶管理頁面

K8S flannel 網絡

交付redis 到k8s 集羣

交付dubbo 微服務到k8s 集羣

交付Jenkins 到k8s集羣

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結