KubeShare-DevOps实战部署尚医通项目

最近在学习k8s，部署一个项目练练手，记录使用DevOps|KubeShare部署尚医通项目的过程，这个项目用到的中间件较多，采用微服务+分布式架构，可以很好巩固这段时间学到的东西

项目架构图如下所示：

部署需要的中间件

为了熟悉部署过程，大部分中间会手动进行部署，只有小部分中间件会选择使用应用仓库部署。以下的部署都是单节点部署，做不到高可用户，等后面有时间再做高可用

部署ElasticSearch

1.docker下启动ElasticSearch指令如下，分析指令可知

需要两个环境变量ES_JAVA_OPTS、discovery.type
/usr/share/elasticsearch/data目录下的数据需要挂载到外部存储
/usr/share/elasticsearch/config 下的配置文件挂载

# 创建数据目录
mkdir -p /mydata/es-01 && chmod 777 -R /mydata/es-01

# 容器启动
docker run --restart=always -d -p 9200:9200 -p 9300:9300 \
-e "discovery.type=single-node" \
-e ES_JAVA_OPTS="-Xms512m -Xmx512m" \
-v es-config:/usr/share/elasticsearch/config \
-v /mydata/es-01/data:/usr/share/elasticsearch/data \
--name es-01 \
elasticsearch:7.13.4

2.在ElasticSearch官网进行下载，查看需要挂载的elasticsearch.yml、jvm.options配置文件

elasticsearch.yml中需要修改配置

cluster.name: docker-cluster
network.host: 0.0.0.0

3.创建配置字典

4.添加环境变量和启动命令

5.挂载配置和外部存储，因为/usr/share/elasticsearch/config目录下还有其它配置文件，所以配置字典需要使用子路径进行挂载

6.验证部署，服务开放9200端口后，游览器访问ip地址＋分配的端口出现以下界面说明部署成功

部署Sentinel

1.在docker hub上查找Sentinel的镜像，这里选择leifengyang/sentinel:1.8.2的镜像

2.在项目中选择创建服务，选择创建有状态服务，搜索镜像，选择使用默认端口，然后一路点击下一步即可

3.验证部署，修改服务的访问方式为nodePort，然后在游览器上访问，看到登录页面，说明部署成功

账号密码都为sentinel

部署MongoDB

1.在部署前需要配置企业空间的应用仓库，这里选用bitnami的仓库，地址是：https://charts.bitnami.com/bitnami

2.在项目内创建MongoDB应用，在bitnami应用仓库搜索MongoDB

3.安装MongoDB，配置选择单节点，去掉账号密码验证，然后点击下一步即可

4.创建服务指定MongoDB的工作负载，使MongoDB可以外部访问，使Navicat连接验证是否安装成功

部署RabbitMQ

RabbitMQ和MongoDB一样，从应用仓库下载即可，密码设置为123456。

部署MySQL

1.使用docker启动MySQL的指令如下所示

docker run -p 3306:3306 --name mysql-01 \
-v /mydata/mysql/log:/var/log/mysql \
-v /mydata/mysql/data:/var/lib/mysql \
-v /mydata/mysql/conf:/etc/mysql/conf.d \
-e MYSQL_ROOT_PASSWORD=root \
--restart=always \
-d mysql:5.7

mysql配置文件内容

[client]
default-character-set=utf8mb4
 
[mysql]
default-character-set=utf8mb4
 
[mysqld]
init_connect='SET collation_connection = utf8mb4_unicode_ci'
init_connect='SET NAMES utf8mb4'
character-set-server=utf8mb4
collation-server=utf8mb4_unicode_ci
skip-character-set-client-handshake
skip-name-resolve

对指令进行分析

2.创建mysql的配置字典，建可以理解为配置文件的文件名，值可以理解为配置文件内容

3.通过有状态的副本集创建MySQL工作负载，需要通过环境变量来设置root用户的密码

4.需要将mysql的配置数据挂载到外部存储

5.配置文件挂载，选择提前配置好的字典

6.创建完成后，配置服务，使用Navicat连接验证，并且导入sql，创建项目需要的表和数据，一共有五个数据库

部署Redis

使用docker部署redis指令如下所示，分析可以得知，redis的数据需要挂载外部，配置文件redis.cnof需要配置字典，启动时需要添加指令 redis-server /etc/redis/redis.conf

#创建配置文件
## 1、准备redis配置文件内容
mkdir -p /mydata/redis/conf && vim /mydata/redis/conf/redis.conf


##配置示例
appendonly yes
port 6379
bind 0.0.0.0


#docker启动redis
docker run -d -p 6379:6379 --restart=always \
-v /mydata/redis/conf/redis.conf:/etc/redis/redis.conf \
-v  /mydata/redis-01/data:/data \
 --name redis-01 redis:6.2.5 \
 redis-server /etc/redis/redis.conf

部署Nacos

以下部署都是单节点部署，假如需要多节点部署

那么需要创建配置字典，键为cluster.conf，值为nacos容器的域名，多个容器用换行符分割

1.在mysql创建nacos需要的表和数据，可以在nacos官网下载最新的nacos的zip包，解压后将nacos\conf目录下的sql文件，使用navicat将sql文件导入mysql

2.创建配置字典，键是application.properties，值是nacos解压nacos\conf目录下application.properties文件中的内容，需要修改配置文件中数据库参数

3.创建nacos的服务，根据nacos官方文档单节点部署

需要添加环境变量，键MODE，值为standalone
配置字典要子路径挂载，路径为/home/nacos/config/application.properties
创建监控检查http请求，路径/nacos，端口8848

4.验证部署，需要开放服务端口，游览器访问http://{ip地址}:30376/nacos/#/login，默认账号nacos，默认密码nacos

中间件部署完成

中间件部署完后，创建表格保存基本信息，方便后续的部署

中间件名称	内网地址	外网地址	账号/密码
Nacos	his-nacos.syt:8848	192.168.64.128:30376	nacos/nacos
MySQL	his-mysql-node.syt:3306	192.168.64.128:30414	root/root
Redis	his-redis-node.syt:6379	192.168.64.128:30418
Sentinel	his-sentinel-node.syt:8080	192.168.64.128:30456
MongoDB	his-mongo-node.syt:27017	192.168.64.128:30593
RabbitMQ	rabbitm-4n3fjk-rabbitmq.syt:5672	192.168.64.128:31776	user/123456
ElasticSearch	his-es-node.syt:9200	192.168.64.128:32597

部署后端微服务

该项目的微服务的注册中心和配置中心都是nacos

1.在Nacos创建配置文件

配置文件名按照【微服务名+环境+.yml】规则命名，配置内容从项目代码可以拿到，注意其中一些中间件的参数需要替换

将所有的配置文件配置完后进入进入下一步

2.DevOps

开启DevOps

由于之前部署没有开启devops，现在进行安装，参考文档，安装流程：

以 admin 身份登录控制台。点击左上角的平台管理 ，然后选择集群管理。
点击定制资源定义，然后在搜索栏中输入 clusterconfiguration，点击搜索结果进入其详情页面。
在自定义资源中，点击 ks-installer 右侧的三个点，然后选择编辑 YAML。
在该配置文件中，将对应组件 enabled 的 false 更改为 true，以启用要安装的组件。完成后，点击确定以保存配置。

配置maven仓库的镜像地址

使用admin登陆ks–进入集群管理–进入配置中心–找到配置ks-devops-agent–添加阿里的镜像仓库

<!--阿里云的maven仓库地址--> 
<mirror>
      <id>aliyunmaven</id>
      <mirrorOf>*</mirrorOf>
      <name>阿里云公共仓库</name>
      <url>https://maven.aliyun.com/repository/public</url>
 </mirror>

使用可视化界面编辑流水线

得到的Jenkinsfile如下

pipeline {
  agent {
    node {
      label 'maven'
    }

  }
  stages {
    stage('拉取代码') {
      agent none
      steps {
        container('maven') {
          git(url: 'https://gitee.com/cao-yu-1208/yygh-parent.git', credentialsId: 'gitee-id', branch: 'master', changelog: true, poll: false)
          sh 'ls'
        }

      }
    }

    stage('项目编译') {
      agent none
      steps {
        container('maven') {
          sh 'ls'
          sh 'mvn clean package -Dmaven.test.skip=true'
          sh 'ls hospital-manage/target'
        }

      }
    }

    stage('default-2') {
      parallel {
        stage('构建hospital-manage镜像') {
          agent none
          steps {
            container('maven') {
              sh 'ls hospital-manage/target'
              sh 'docker build -t hospital-manage:latest -f hospital-manage/Dockerfile  ./hospital-manage/'
            }

          }
        }

        stage('构建server-gateway镜像') {
          agent none
          steps {
            container('maven') {
              sh 'ls server-gateway/target'
              sh 'docker build -t server-gateway:latest -f server-gateway/Dockerfile  ./server-gateway/'
            }

          }
        }

        stage('构建service-cmn镜像') {
          agent none
          steps {
            container('maven') {
              sh 'ls service/service-cmn/target'
              sh 'docker build -t service-cmn:latest -f service/service-cmn/Dockerfile  ./service/service-cmn/'
            }

          }
        }

        stage('构建service-hosp镜像') {
          agent none
          steps {
            container('maven') {
              sh 'ls service/service-hosp/target'
              sh 'docker build -t service-hosp:latest -f service/service-hosp/Dockerfile  ./service/service-hosp/'
            }

          }
        }

        stage('构建service-order镜像') {
          agent none
          steps {
            container('maven') {
              sh 'ls service/service-order/target'
              sh 'docker build -t service-order:latest -f service/service-order/Dockerfile  ./service/service-order/'
            }

          }
        }

        stage('构建service-oss镜像') {
          agent none
          steps {
            container('maven') {
              sh 'ls service/service-oss/target'
              sh 'docker build -t service-oss:latest -f service/service-oss/Dockerfile  ./service/service-oss/'
            }

          }
        }

        stage('构建service-sms镜像') {
          agent none
          steps {
            container('maven') {
              sh 'ls service/service-sms/target'
              sh 'docker build -t service-sms:latest -f service/service-sms/Dockerfile  ./service/service-sms/'
            }

          }
        }

        stage('构建service-statistics镜像') {
          agent none
          steps {
            container('maven') {
              sh 'ls service/service-statistics/target'
              sh 'docker build -t service-statistics:latest -f service/service-statistics/Dockerfile  ./service/service-statistics/'
            }

          }
        }

        stage('构建service-task镜像') {
          agent none
          steps {
            container('maven') {
              sh 'ls service/service-task/target'
              sh 'docker build -t service-task:latest -f service/service-task/Dockerfile  ./service/service-task/'
            }

          }
        }

        stage('构建service-user镜像') {
          agent none
          steps {
            container('maven') {
              sh 'ls service/service-user/target'
              sh 'docker build -t service-user:latest -f service/service-user/Dockerfile  ./service/service-user/'
            }

          }
        }

      }
    }

    stage('default-3') {
      parallel {
        stage('推送hospital-manage镜像') {
          agent none
          steps {
            container('maven') {
              withCredentials([usernamePassword(credentialsId : 'aliyun' ,usernameVariable : 'DOCKER_USER_VAR' ,passwordVariable : 'DOCKER_PWD_VAR' ,)]) {
                sh 'echo "$DOCKER_PWD_VAR" | docker login $REGISTRY -u "$DOCKER_USER_VAR" --password-stdin'
                sh 'docker tag hospital-manage:latest $REGISTRY/$DOCKERHUB_NAMESPACE/hospital-manage:SNAPSHOT-$BUILD_NUMBER'
                sh 'docker push  $REGISTRY/$DOCKERHUB_NAMESPACE/hospital-manage:SNAPSHOT-$BUILD_NUMBER'
              }

            }

          }
        }

        stage('推送server-gateway镜像') {
          agent none
          steps {
            container('maven') {
              withCredentials([usernamePassword(credentialsId : 'aliyun' ,usernameVariable : 'DOCKER_USER_VAR' ,passwordVariable : 'DOCKER_PWD_VAR' ,)]) {
                sh 'echo "$DOCKER_PWD_VAR" | docker login $REGISTRY -u "$DOCKER_USER_VAR" --password-stdin'
                sh 'docker tag server-gateway:latest $REGISTRY/$DOCKERHUB_NAMESPACE/server-gateway:SNAPSHOT-$BUILD_NUMBER'
                sh 'docker push  $REGISTRY/$DOCKERHUB_NAMESPACE/server-gateway:SNAPSHOT-$BUILD_NUMBER'
              }

            }

          }
        }

        stage('推送service-cmn镜像') {
          agent none
          steps {
            container('maven') {
              withCredentials([usernamePassword(credentialsId : 'aliyun' ,usernameVariable : 'DOCKER_USER_VAR' ,passwordVariable : 'DOCKER_PWD_VAR' ,)]) {
                sh 'echo "$DOCKER_PWD_VAR" | docker login $REGISTRY -u "$DOCKER_USER_VAR" --password-stdin'
                sh 'docker tag service-cmn:latest $REGISTRY/$DOCKERHUB_NAMESPACE/service-cmn:SNAPSHOT-$BUILD_NUMBER'
                sh 'docker push  $REGISTRY/$DOCKERHUB_NAMESPACE/service-cmn:SNAPSHOT-$BUILD_NUMBER'
              }

            }

          }
        }

        stage('推送service-hosp镜像') {
          agent none
          steps {
            container('maven') {
              withCredentials([usernamePassword(credentialsId : 'aliyun' ,usernameVariable : 'DOCKER_USER_VAR' ,passwordVariable : 'DOCKER_PWD_VAR' ,)]) {
                sh 'echo "$DOCKER_PWD_VAR" | docker login $REGISTRY -u "$DOCKER_USER_VAR" --password-stdin'
                sh 'docker tag service-hosp:latest $REGISTRY/$DOCKERHUB_NAMESPACE/service-hosp:SNAPSHOT-$BUILD_NUMBER'
                sh 'docker push  $REGISTRY/$DOCKERHUB_NAMESPACE/service-hosp:SNAPSHOT-$BUILD_NUMBER'
              }

            }

          }
        }

        stage('推送service-order镜像') {
          agent none
          steps {
            container('maven') {
              withCredentials([usernamePassword(credentialsId : 'aliyun' ,usernameVariable : 'DOCKER_USER_VAR' ,passwordVariable : 'DOCKER_PWD_VAR' ,)]) {
                sh 'echo "$DOCKER_PWD_VAR" | docker login $REGISTRY -u "$DOCKER_USER_VAR" --password-stdin'
                sh 'docker tag service-order:latest $REGISTRY/$DOCKERHUB_NAMESPACE/service-order:SNAPSHOT-$BUILD_NUMBER'
                sh 'docker push  $REGISTRY/$DOCKERHUB_NAMESPACE/service-order:SNAPSHOT-$BUILD_NUMBER'
              }

            }

          }
        }

        stage('推送service-oss镜像') {
          agent none
          steps {
            container('maven') {
              withCredentials([usernamePassword(credentialsId : 'aliyun' ,usernameVariable : 'DOCKER_USER_VAR' ,passwordVariable : 'DOCKER_PWD_VAR' ,)]) {
                sh 'echo "$DOCKER_PWD_VAR" | docker login $REGISTRY -u "$DOCKER_USER_VAR" --password-stdin'
                sh 'docker tag service-oss:latest $REGISTRY/$DOCKERHUB_NAMESPACE/service-oss:SNAPSHOT-$BUILD_NUMBER'
                sh 'docker push  $REGISTRY/$DOCKERHUB_NAMESPACE/service-oss:SNAPSHOT-$BUILD_NUMBER'
              }

            }

          }
        }

        stage('推送service-sms镜像') {
          agent none
          steps {
            container('maven') {
              withCredentials([usernamePassword(credentialsId : 'aliyun' ,usernameVariable : 'DOCKER_USER_VAR' ,passwordVariable : 'DOCKER_PWD_VAR' ,)]) {
                sh 'echo "$DOCKER_PWD_VAR" | docker login $REGISTRY -u "$DOCKER_USER_VAR" --password-stdin'
                sh 'docker tag service-sms:latest $REGISTRY/$DOCKERHUB_NAMESPACE/service-sms:SNAPSHOT-$BUILD_NUMBER'
                sh 'docker push  $REGISTRY/$DOCKERHUB_NAMESPACE/service-sms:SNAPSHOT-$BUILD_NUMBER'
              }

            }

          }
        }

        stage('推送service-statistics镜像') {
          agent none
          steps {
            container('maven') {
              withCredentials([usernamePassword(credentialsId : 'aliyun' ,usernameVariable : 'DOCKER_USER_VAR' ,passwordVariable : 'DOCKER_PWD_VAR' ,)]) {
                sh 'echo "$DOCKER_PWD_VAR" | docker login $REGISTRY -u "$DOCKER_USER_VAR" --password-stdin'
                sh 'docker tag service-statistics:latest $REGISTRY/$DOCKERHUB_NAMESPACE/service-statistics:SNAPSHOT-$BUILD_NUMBER'
                sh 'docker push  $REGISTRY/$DOCKERHUB_NAMESPACE/service-statistics:SNAPSHOT-$BUILD_NUMBER'
              }

            }

          }
        }

        stage('推送service-task镜像') {
          agent none
          steps {
            container('maven') {
              withCredentials([usernamePassword(credentialsId : 'aliyun' ,usernameVariable : 'DOCKER_USER_VAR' ,passwordVariable : 'DOCKER_PWD_VAR' ,)]) {
                sh 'echo "$DOCKER_PWD_VAR" | docker login $REGISTRY -u "$DOCKER_USER_VAR" --password-stdin'
                sh 'docker tag service-task:latest $REGISTRY/$DOCKERHUB_NAMESPACE/service-task:SNAPSHOT-$BUILD_NUMBER'
                sh 'docker push  $REGISTRY/$DOCKERHUB_NAMESPACE/service-task:SNAPSHOT-$BUILD_NUMBER'
              }

            }

          }
        }

        stage('推送service-user镜像') {
          agent none
          steps {
            container('maven') {
              withCredentials([usernamePassword(credentialsId : 'aliyun' ,usernameVariable : 'DOCKER_USER_VAR' ,passwordVariable : 'DOCKER_PWD_VAR' ,)]) {
                sh 'echo "$DOCKER_PWD_VAR" | docker login $REGISTRY -u "$DOCKER_USER_VAR" --password-stdin'
                sh 'docker tag service-user:latest $REGISTRY/$DOCKERHUB_NAMESPACE/service-user:SNAPSHOT-$BUILD_NUMBER'
                sh 'docker push  $REGISTRY/$DOCKERHUB_NAMESPACE/service-user:SNAPSHOT-$BUILD_NUMBER'
              }

            }

          }
        }

      }
    }

    stage('default-4') {
      parallel {
        stage('hospital-manage部署到集群') {
          agent none
          steps {
            container('maven') {
              withCredentials([kubeconfigContent(credentialsId : 'demo-kubeconfig' ,variable : 'KUBECONFIG_CONFIG' ,)]) {
                sh 'mkdir -p ~/.kube/'
                sh 'echo "$KUBECONFIG_CONFIG" > ~/.kube/config'
                sh 'envsubst < hospital-manage/deploy/deploy.yml | kubectl apply -f -'
              }

            }

          }
        }

        stage('server-gateway部署到集群') {
          agent none
          steps {
            container('maven') {
              withCredentials([kubeconfigContent(credentialsId : 'demo-kubeconfig' ,variable : 'KUBECONFIG_CONFIG' ,)]) {
                sh 'mkdir -p ~/.kube/'
                sh 'echo "$KUBECONFIG_CONFIG" > ~/.kube/config'
                sh 'envsubst < server-gateway/deploy/deploy.yml | kubectl apply -f -'
              }

            }

          }
        }

        stage('service-cmn部署到集群') {
          agent none
          steps {
            container('maven') {
              withCredentials([kubeconfigContent(credentialsId : 'demo-kubeconfig' ,variable : 'KUBECONFIG_CONFIG' ,)]) {
                sh 'mkdir -p ~/.kube/'
                sh 'echo "$KUBECONFIG_CONFIG" > ~/.kube/config'
                sh 'envsubst < service/service-cmn/deploy/deploy.yml | kubectl apply -f -'
              }

            }

          }
        }

        stage('service-hosp部署到集群') {
          agent none
          steps {
            container('maven') {
              withCredentials([kubeconfigContent(credentialsId : 'demo-kubeconfig' ,variable : 'KUBECONFIG_CONFIG' ,)]) {
                sh 'mkdir -p ~/.kube/'
                sh 'echo "$KUBECONFIG_CONFIG" > ~/.kube/config'
                sh 'envsubst < service/service-hosp/deploy/deploy.yml | kubectl apply -f -'
              }

            }

          }
        }

        stage('service-order部署到集群') {
          agent none
          steps {
            container('maven') {
              withCredentials([kubeconfigContent(credentialsId : 'demo-kubeconfig' ,variable : 'KUBECONFIG_CONFIG' ,)]) {
                sh 'mkdir -p ~/.kube/'
                sh 'echo "$KUBECONFIG_CONFIG" > ~/.kube/config'
                sh 'envsubst < service/service-order/deploy/deploy.yml | kubectl apply -f -'
              }

            }

          }
        }

        stage('service-oss部署到集群') {
          agent none
          steps {
            container('maven') {
              withCredentials([kubeconfigContent(credentialsId : 'demo-kubeconfig' ,variable : 'KUBECONFIG_CONFIG' ,)]) {
                sh 'mkdir -p ~/.kube/'
                sh 'echo "$KUBECONFIG_CONFIG" > ~/.kube/config'
                sh 'envsubst < service/service-oss/deploy/deploy.yml | kubectl apply -f -'
              }

            }

          }
        }

        stage('service-sms部署到集群') {
          agent none
          steps {
            container('maven') {
              withCredentials([kubeconfigContent(credentialsId : 'demo-kubeconfig' ,variable : 'KUBECONFIG_CONFIG' ,)]) {
                sh 'mkdir -p ~/.kube/'
                sh 'echo "$KUBECONFIG_CONFIG" > ~/.kube/config'
                sh 'envsubst < service/service-sms/deploy/deploy.yml | kubectl apply -f -'
              }

            }

          }
        }

        stage('service-statistics部署到集群') {
          agent none
          steps {
            container('maven') {
              withCredentials([kubeconfigContent(credentialsId : 'demo-kubeconfig' ,variable : 'KUBECONFIG_CONFIG' ,)]) {
                sh 'mkdir -p ~/.kube/'
                sh 'echo "$KUBECONFIG_CONFIG" > ~/.kube/config'
                sh 'envsubst < service/service-statistics/deploy/deploy.yml | kubectl apply -f -'
              }

            }

          }
        }

        stage('service-task部署到集群') {
          agent none
          steps {
            container('maven') {
              withCredentials([kubeconfigContent(credentialsId : 'demo-kubeconfig' ,variable : 'KUBECONFIG_CONFIG' ,)]) {
                sh 'mkdir -p ~/.kube/'
                sh 'echo "$KUBECONFIG_CONFIG" > ~/.kube/config'
                sh 'envsubst < service/service-task/deploy/deploy.yml | kubectl apply -f -'
              }

            }

          }
        }

        stage('service-user部署到集群') {
          agent none
          steps {
            container('maven') {
              withCredentials([kubeconfigContent(credentialsId : 'demo-kubeconfig' ,variable : 'KUBECONFIG_CONFIG' ,)]) {
                sh 'mkdir -p ~/.kube/'
                sh 'echo "$KUBECONFIG_CONFIG" > ~/.kube/config'
                sh 'envsubst < service/service-user/deploy/deploy.yml | kubectl apply -f -'
              }

            }

          }
        }

      }
    }

    stage('发送确认邮件') {
      agent none
      steps {
        mail(to: '1170907841@qq.com', subject: '构建结构', body: '构建超过了 $BUILD_NUMBER')
      }
    }

  }
  environment {
    DOCKER_CREDENTIAL_ID = 'dockerhub-id'
    GITHUB_CREDENTIAL_ID = 'github-id'
    KUBECONFIG_CREDENTIAL_ID = 'demo-kubeconfig'
    REGISTRY = 'registry.cn-shanghai.aliyuncs.com'
    DOCKERHUB_NAMESPACE = 'cy-syt'
    GITHUB_ACCOUNT = 'kubesphere'
    APP_NAME = 'devops-java-sample'
    ALIYUNHUB_NAMESPACE = 'cy-syt'
  }
  parameters {
    string(name: 'TAG_NAME', defaultValue: '', description: '')
  }
}

部署前端

后台部署

后台前端采用的vue在客户端进行渲染，所以在部署前需要修改请求地址为servce-gateway的外网访问地址，Jenkinsfile文件如下

pipeline {
  agent {
    node {
      label 'nodejs'
    }

  }
  stages {
    stage('拉取代码') {
      agent none
      steps {
        container('nodejs') {
          git(url: 'https://gitee.com/cao-yu-1208/yygh-admin.git', credentialsId: 'gitee-id', branch: 'master', changelog: true, poll: false)
          sh 'ls'
        }

      }
    }

    stage('项目编译') {
      agent none
      steps {
        container('nodejs') {
          sh 'npm i node-sass --sass_binary_site=https://npm.taobao.org/mirrors/node-sass/'
          sh 'npm install --registry=https://registry.npm.taobao.org'
          sh 'npm run build'
          sh 'ls'
        }

      }
    }
    
    stage('构建镜像') {
            agent none
            steps {
                container('nodejs') {
                    sh 'ls'
                    sh 'docker build -t yygh-admin:latest -f Dockerfile  .'
                }

            }
    }
    
    stage('推送镜像') {
            agent none
            steps {
                container('nodejs') {
                    withCredentials([usernamePassword(credentialsId : 'aliyun' ,usernameVariable : 'DOCKER_USER_VAR' ,passwordVariable : 'DOCKER_PWD_VAR' ,)]) {
                        sh 'echo "$DOCKER_PWD_VAR" | docker login $REGISTRY -u "$DOCKER_USER_VAR" --password-stdin'
                        sh 'docker tag yygh-admin:latest $REGISTRY/$DOCKERHUB_NAMESPACE/yygh-admin:SNAPSHOT-$BUILD_NUMBER'
                        sh 'docker push  $REGISTRY/$DOCKERHUB_NAMESPACE/yygh-admin:SNAPSHOT-$BUILD_NUMBER'
                    }

                }

            }
        }
        
         stage('部署到集群') {
          agent none
          steps {
            container('nodejs') {
              withCredentials([kubeconfigContent(credentialsId : 'demo-kubeconfig' ,variable : 'KUBECONFIG_CONFIG' ,)]) {
                sh 'mkdir -p ~/.kube/'
                sh 'echo "$KUBECONFIG_CONFIG" > ~/.kube/config'
                sh 'envsubst < deploy/deploy.yml | kubectl apply -f -'
              }

            }

          }
        }
        

  }
  environment {
    DOCKER_CREDENTIAL_ID = 'dockerhub-id'
    GITHUB_CREDENTIAL_ID = 'github-id'
    KUBECONFIG_CREDENTIAL_ID = 'demo-kubeconfig'
    REGISTRY = 'registry.cn-shanghai.aliyuncs.com'
    DOCKERHUB_NAMESPACE = 'cy-syt'
    GITHUB_ACCOUNT = 'kubesphere'
    APP_NAME = 'devops-java-sample'
    ALIYUNHUB_NAMESPACE = 'cy-syt'
  }
}

前台部署

前台项目使用的nuxt，在服务断进行渲染，访问地址可以填写servce-gateway的内网地址，Jenkinsfile文件如下

pipeline {
  agent {
    node {
      label 'nodejs'
    }

  }
  stages {
    stage('拉取代码') {
      agent none
      steps {
        container('nodejs') {
          git(url: 'https://gitee.com/cao-yu-1208/yygh-site.git', credentialsId: 'gitee-id', branch: 'master', changelog: true, poll: false)
        }

      }
    }

    stage('代码构建') {
      agent none
      steps {
        container('nodejs') {
           sh 'ls'
           sh 'npm install --registry=https://registry.npm.taobao.org'
                     sh 'npm run build'
        }

      }
    }

    stage('镜像构建') {
      agent none
      steps {
        container('nodejs') {
          sh 'docker build -t yygh-site:latest -f Dockerfile  .'
        }

      }
    }

    stage('推送镜像') {
      agent none
      steps {
        container('nodejs') {
          withCredentials([usernamePassword(credentialsId : 'aliyun' ,passwordVariable : 'DOCKER_PWD_VAR' ,usernameVariable : 'DOCKER_USER_VAR' ,)]) {
           sh 'echo "$DOCKER_PWD_VAR" | docker login $REGISTRY -u "$DOCKER_USER_VAR" --password-stdin'
           sh 'docker tag yygh-site:latest $REGISTRY/$DOCKERHUB_NAMESPACE/yygh-site:SNAPSHOT-$BUILD_NUMBER'
           sh 'docker push  $REGISTRY/$DOCKERHUB_NAMESPACE/yygh-site:SNAPSHOT-$BUILD_NUMBER'
          }

        }

      }
    }
    
    stage('部署到集群') {
          agent none
          steps {
            container('nodejs') {
              withCredentials([kubeconfigContent(credentialsId : 'demo-kubeconfig' ,variable : 'KUBECONFIG_CONFIG' ,)]) {
                sh 'mkdir -p ~/.kube/'
                sh 'echo "$KUBECONFIG_CONFIG" > ~/.kube/config'
                sh 'envsubst < deploy/deploy.yml | kubectl apply -f -'
              }

            }

          }
        }

  }
  environment {
    DOCKER_CREDENTIAL_ID = 'dockerhub-id'
    GITHUB_CREDENTIAL_ID = 'github-id'
    KUBECONFIG_CREDENTIAL_ID = 'demo-kubeconfig'
    REGISTRY = 'registry.cn-shanghai.aliyuncs.com'
    DOCKERHUB_NAMESPACE = 'cy-syt'
    GITHUB_ACCOUNT = 'kubesphere'
    APP_NAME = 'devops-java-sample'
    ALIYUNHUB_NAMESPACE = 'cy-syt'
  }
}

问题处理

DevOps提示未运行BUG解决

应该是之前最小化安装时没有开启devops，先创建了几个用户，再开启devops后，jenkins不识别这几个老的用户，解决方法如下：

创建一个新用户，比如project-user
找到kubesphere-devops-system企业空间下devops-apiserver/devops-controller/devops-jenkins开头的三个工作负载，将起副本调到0删除pod后，然后再把副本数改回来，会重新启动pod。
使用project-user登录kubesphere，重新运行流水线，显示正常

Docker报错EOF解决

开了vpn，关掉就ok了

微服务报错解决

部署时出现报错Failed to configure a DataSource: 'url' attribute is not specified and no embedded datasource could be configured.

检查url和驱动均无问题，排查发现是没有指定命名空间，导致启动时没找到配置文件

在Dockerfile文件中，修改jar的启动命令，指定nacos的命名空间接口--spring.cloud.nacos.config.namespace=33cc65fd-0289-4860-9db2-88f6223708d3

参考文档

KubeSphere DevOps 3.0 流水线运维指南

DevOps 用户指南

KubeShare-DevOps实战部署尚医通项目

KubeShare-DevOps实战部署尚医通项目

部署需要的中间件

部署ElasticSearch

部署Sentinel

部署MongoDB

部署RabbitMQ

部署MySQL

部署Redis

部署Nacos

中间件部署完成

部署后端微服务

1.在Nacos创建配置文件

2.DevOps

开启DevOps

配置maven仓库的镜像地址

使用可视化界面编辑流水线

部署前端

后台部署

前台部署

问题处理

DevOps提示未运行BUG解决

Docker报错EOF解决

微服务报错解决

参考文档

推荐阅读

Kubernetes基本概念

虚拟机安装Kubernetes

Ingress安装

评论