搭建CI/CD
可以使用的工具不少,例如:Jenkins
、Gitlab CI
等等,这里会使用大家最新熟悉的Jenkins
来做CI/CD
的工具。不少人都是基于虚拟机来搭建的,基于Kubernetes
搭建的Jenkins Slave会有什么效果呢?
安装Jenkins Master
直接进入正题,新建一个Deployment:
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: jenkins
namespace: default
spec:
template:
metadata:
labels:
app: jenkins
spec:
terminationGracePeriodSeconds: 10
serviceAccountName: jenkins
containers:
- name: jenkins
image: ccr.ccs.tencentyun.com/my-registry/jenkins:lts-k8s
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8080
name: web
protocol: TCP
- containerPort: 50000
name: agent
protocol: TCP
resources:
limits:
cpu: 1000m
memory: 1Gi
requests:
cpu: 500m
memory: 512Mi
volumeMounts:
- name: jenkins-home
mountPath: /var/jenkins_home
- name: docker-socket
mountPath: /var/run/docker.sock
env:
- name: JAVA_OPTS
value: -Djenkins.install.runSetupWizard=false -Duser.timezone=Asia/Shanghai
securityContext:
fsGroup: 1000
volumes:
- name: jenkins-home
emptyDir: {}
- name: docker-socket
hostPath:
path: /var/run/docker.sock
注意:ccr.ccs.tencentyun.com/my-registry/jenkins:lts-k8s
是我基于jenkins/jenkins:lts镜像定制的,公开的镜像,内置了比较主流的插件,使用admin/password
就可以登录。
新建一个Service:
apiVersion: v1
kind: Service
metadata:
name: jenkins
namespace: default
labels:
app: jenkins
spec:
selector:
app: jenkins
type: LoadBalancer
ports:
- name: web
port: 80
targetPort: 8080
- name: agent
port: 50000
targetPort: 50000
注意:我这里使用的是LoadBalancer
,简单方便的话也可以使用NodePort
的形式来暴露Jenkins的web服务,需要指定固定的node端口。这里还需要暴露一个agent端口,主要用于Jenkins的master和slave之间的通信。我这里使用的腾讯云的TKE
服务,LoadBalancer
会自动分别一个外网IP地址。
新建角色权限:
apiVersion: v1
kind: ServiceAccount
metadata:
name: jenkins
namespace: default
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: jenkins
rules:
- apiGroups: ["extensions", "apps"]
resources: ["deployments"]
verbs: ["create", "delete", "get", "list", "watch", "patch", "update"]
- apiGroups: [""]
resources: ["services"]
verbs: ["create", "delete", "get", "list", "watch", "patch", "update"]
- apiGroups: [""]
resources: ["pods"]
verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
resources: ["pods/exec"]
verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
resources: ["pods/log"]
verbs: ["get","list","watch"]
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: jenkins
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: jenkins
subjects:
- kind: ServiceAccount
name: jenkins
namespace: default
之后执行安装命令:
kubectl apply -f .
等待服务的启动,可以通过kubectl get svc
获取到jenkins
service的外网IP地址,就可以访问Jenkins web服务了。
配置Jenknis Slave
接下来我们就需要来配置 Jenkins,让它能够动态的生成 Slave 的 Pod。
第一步:安装Kubernetes Plugin,点击 Manage Jenkins -> Manage Plugins -> Available -> Kubernetes plugin 勾选安装即可。
第二步:填写 Kubernetes Cloud Details,点击 Manage Jenkins —> Configure System —> (拖到最下方)Add a new cloud —> 选择 Kubernetes
这里需要点击Test Connection
去验证是否可以连接你的Kubernetes
集群。
第三步:填写Pod Template和Container Template
注意这里的名称最好都一致,例如我这边都使用jnlp-agent
,之前因为名字不同导致无法完美运行。
第四步:添加卷信息
可以看到这里分别挂载了主机的docker,kubectl等工具。如果碰到没有权限执行/var/run/docker.sock
,因为
在腾讯云TKE 集群中的每个 node 节点系统里都有一个 docker.sock 文件,slave pod 在执行 docker build 时将会连接该文件。在此之前,需逐个登录到每个节点上,依次执行以下命令对 docker build 进行授权:
chmod 666 /var/run/docker.sock
第五步:配置权限
这里填写jenkins
,是我们在jenkins master中创建的一个Service Account。如果不填的话,会导致执行kubectl
命令会出现权限不足。
测试
新建一个Freestyle的Item:
填写在Container Template中的Label:
添加一个Build Shell:
echo '=== 测试动态jenkins slave === '
echo '=== 打印docker信息 ==='
docker info
echo '=== 获取pods ==='
kubectl get pods
在Jenkins web中点击立即构建Build Now
,在控制观察kuberntes Pod的状态:
执行完毕后,控制台会有如下输出:
Building remotely on jnlp-agent-l58sk (jnlp-agent) in workspace /home/jenkins/agent/workspace/jnlp-demo
[jnlp-demo] $ /bin/sh -xe /tmp/jenkins4069627081916824161.sh
+ echo === 测试动态jenkins slave ===
=== 测试动态jenkins slave ===
+ echo === 打印docker信息 ===
=== 打印docker信息 ===
+ docker info
Containers: 20
Running: 19
Paused: 0
Stopped: 1
Images: 13
Server Version: 18.06.3-ce
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: e6b3f5632f50dbc4e9cb6288d911bf4f5e95b18e (expected: 468a545b9edcd5932818eb9de8e72413e616e86e)
runc version: f56b4cbeadc407e715d9b2ba49e62185bd81cef4 (expected: a592beb5bc4c4092b1b1bac971afed27687340c5)
init version: fec3683
Security Options:
seccomp
Profile: default
Kernel Version: 4.14.105-19-0008
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 3.476GiB
Name: VM_0_12_centos
ID: 5EKE:FP2N:QFR7:BHIV:5LLJ:OUUK:CDV6:PRPI:5TW2:GAKT:JCMH:OFF3
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Registry Mirrors:
https://mirror.ccs.tencentyun.com/
Live Restore Enabled: true
+ echo === 获取pods ===
=== 获取pods ===
+ kubectl get pods
NAME READY STATUS RESTARTS AGE
jenkins-587dc776c6-79hlg 1/1 Running 0 23m
jnlp-agent-l58sk 2/2 Running 0 17s
Finished: SUCCESS
到这里基于Kubernetes的动态Jenkins Slave就搭建完毕了。这里演示了一个简单的demo,如果在项目中有Jenkinsfile
,可以定义node属性,指定使用jnlp-agent
即可使用slave构建你的项目了。
参考
「真诚赞赏,手留余香」
请我喝杯咖啡?
使用微信扫描二维码完成支付
