K
Q

Jenkins container persistence on Kubernetes cluster - PersistentVolumeClaim (VMware/Vsphere)

7/4/2019

Trying to persist my jenkins jobs on to vsphere storage when I delete the deployments/services.

I've tried using the standard approach: used StorageClass, then made a PersistentVolumeClaim which is referenced in the .ayml file that will create the deployments.

storage-class.yml:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: mystorage
provisioner: kubernetes.io/vsphere-volume
parameters:
  diskformat: zeroedthick

persistent-volume-claim.yml:

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pvc0003
spec:
  storageClassName: mystorage
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 15Gi

jenkins.yml:

---
apiVersion: v1
kind: Service
metadata:
  name: jenkins-auto-ci
  labels:
    app: jenkins-auto-ci
spec:
  type: NodePort
  ports:
  - port: 80
    targetPort: 8080
  selector:
    app: jenkins-auto-ci
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: jenkins-auto-ci
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: jenkins-auto-ci
    spec:
      containers:
      - name: jenkins-auto-ci
        image: jenkins
        resources:
          requests:
            cpu: 100m
            memory: 100Mi
        env:
        - name: GET_HOSTS_FROM
          value: dns
        ports:
        - name: http-port
          containerPort: 80
        - name: jnlp-port
          containerPort: 50000
        volumeMounts:
        - name: jenkins-home
          mountPath: "/var"
      volumes:
        - name: jenkins-home
          persistentVolumeClaim:
            claimName: pvc0003

I expect the jenkins jobs to persist when I delete and recreate the deployments.

-- Alias
jenkins
kubernetes
vmware
volumes
vsphere

Similar Questions

How to change interval for kubectl top?
How to use Kubernetes endpoint object?
k3s cleanup of HelmChart?
JX Promote returns 404 and exits from the job

1 Answer

7/4/2019

You should create VMDK which is Virtual Machine Disk.

You can do that using govc which is vSphere CLI.

govc datastore.disk.create -ds datastore1 -size 2G volumes/myDisk.vmdk

Or using ESXi CLI by ssh into the host as root and executing:

vmkfstools -c 2G /vmfs/volumes/datastore1/volumes/myDisk.vmdk

Once this is done you should create your PV let's call it vsphere_pv.yaml which might look like the following:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv0001
spec:
  capacity:
    storage: 2Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  vsphereVolume:
    volumePath: "[datastore1] volumes/myDisk"
    fsType: ext4

The datastore1 in this example was created in root folder of vCenter, if you have it in a different location you need to change the volumePath. If it's located in DatastoreCluster then set volumePath to"[DatastoreCluster/datastore1] volumes/myDisk".

Apply the yaml to the Kubernetes by kubectl apply -f vsphere_pv.yaml

You can check if it was created by describing it kubectl describe pv pv0001

Now you need PVC let's call it vsphere_pvc.yaml to consume PV.

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pvc0001
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi

Apply the yaml to the Kubernetes by kubectl apply -f vsphere_pvc.yaml

You can check if it was created by describing it kubectl describe pvc pv0001

Once this is done your yaml might be looking like the following:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: jenkins-auto-ci
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: jenkins-auto-ci
    spec:
      containers:
      - name: jenkins-auto-ci
        image: jenkins
        resources:
          requests:
            cpu: 100m
            memory: 100Mi
        env:
        - name: GET_HOSTS_FROM
          value: dns
        ports:
        - name: http-port
          containerPort: 80
        - name: jnlp-port
          containerPort: 50000
        volumeMounts:
        - name: jenkins-home
          mountPath: "/var"
      volumes:
        - name: jenkins-home
          persistentVolumeClaim:
            claimName: pvc0001

All this is nicely explained on Vmware GitHub vsphere-storage-for-kubernetes.

-- Crou
Source: StackOverflow