Define jenkins tmp folder as persistence volume in stable/jenkins helm chart

6/12/2020

I am able to mount jenkins-home Volume as PersistentVolumeClaim

I am unable to mount the tmp Volume as Persistent volume from the values.yaml , it keeps appearing as EmptyDir and connected to directly to Host

I have tried a both the volume options and defining here

https://github.com/helm/charts/blob/77c2f8c632b939af76b4487e0d8032c542568445/stable/jenkins/values.yaml#L478

It still appears as EmptyDir and connected to Host.

https://github.com/helm/charts/blob/master/stable/jenkins/values.yaml

enter image description here

Values.yaml below

    clusterZone: "cluster.local"
    nameOverride: ""
    fullnameOverride: ""
    namespaceOverride: test-project
    master:
      componentName: "jenkins-master"
      image: "jenkins/jenkins"
      tag: "lts"
      imagePullPolicy: "Always"
      imagePullSecretName:
      lifecycle:
        postStart:
          exec:
            command: ["/bin/sh", "-c", "echo Script from the postStart handler to install jq and aws > /usr/share/message && apt-get upgrade -y && apt-get update -y && apt-get install vim -y && apt-get install jq -y && apt-get install awscli -y && apt-get install -y -qq groff && apt-get install -y -qq less"]
      numExecutors: 10
      customJenkinsLabels: []
      useSecurity: true
      enableXmlConfig: true
      securityRealm: |-
        <securityRealm class="hudson.security.LegacySecurityRealm"/>
      authorizationStrategy: |-
         <authorizationStrategy class="hudson.security.FullControlOnceLoggedInAuthorizationStrategy">
           <denyAnonymousReadAccess>true</denyAnonymousReadAccess>
         </authorizationStrategy>
      hostNetworking: false
      # login user for Jenkins
      adminUser: "ctjenkinsadmin"
      rollingUpdate: {}
      resources:
        requests:
          cpu: "50m"
          memory: "512Mi"
        limits:
          cpu: "2000m"
          memory: "4096Mi"
      usePodSecurityContext: true
      servicePort: 8080
      targetPort: 8080
      # Type NodePort for minikube
      serviceAnnotations: {}
      deploymentLabels: {}
      serviceLabels: {}
      podLabels: {}
      # NodePort for Jenkins Service
      healthProbes: true
      healthProbesLivenessTimeout: 5
      healthProbesReadinessTimeout: 5
      healthProbeLivenessPeriodSeconds: 10
      healthProbeReadinessPeriodSeconds: 10
      healthProbeLivenessFailureThreshold: 5
      healthProbeReadinessFailureThreshold: 3
      healthProbeLivenessInitialDelay: 90
      healthProbeReadinessInitialDelay: 60
      slaveListenerPort: 50000
      slaveHostPort:
      disabledAgentProtocols:
        - JNLP-connect
        - JNLP2-connect
      csrf:
        defaultCrumbIssuer:
          enabled: true
          proxyCompatability: true
      cli: false
      slaveListenerServiceType: "ClusterIP"
      slaveListenerServiceAnnotations: {}
      slaveKubernetesNamespace:
      loadBalancerSourceRanges:
      - 0.0.0.0/0
      extraPorts: []
      installPlugins:
        - configuration-as-code:latest
        - kubernetes:latest
        - workflow-aggregator:latest
        - workflow-job:latest
        - credentials-binding:latest
        - git:latest
        - git-client:latest
        - git-server:latest
        - greenballs:latest
        - blueocean:latest
        - strict-crumb-issuer:latest
        - http_request:latest
        - matrix-project:latest
        - jquery:latest
        - artifactory:latest
        - jdk-tool:latest
        - matrix-auth:latest
      enableRawHtmlMarkupFormatter: false
      scriptApproval: []
      initScripts:
        - |
          #!groovy    
          import hudson.model.*;
          import jenkins.model.*;
          import jenkins.security.*;
          import jenkins.security.apitoken.*;
          // script parameters
          def userName = 'user'
          def tokenName = 'token'
          def uploadscript =['/bin/sh', '/var/lib/jenkins/update_token.sh']
          def user = User.get(userName, false)
          def apiTokenProperty = user.getProperty(ApiTokenProperty.class)
          def result = apiTokenProperty.tokenStore.generateNewToken(tokenName)
          def file = new File("/tmp/token.txt")
          file.delete()
          file.write result.plainValue
          uploadscript.execute()
          uploadscript.waitForOrKill(100)
          user.save()
          return result.plainValue
          value = result.plainValue
      jobs:
        Test-Job: |-
          <?xml version='1.0' encoding='UTF-8'?>
          <project>
            <keepDependencies>false</keepDependencies>
            <properties/>
            <scm class="hudson.scm.NullSCM"/>
            <canRoam>false</canRoam>
            <disabled>false</disabled>
            <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>
            <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
            <triggers/>
            <concurrentBuild>false</concurrentBuild>
            <builders/>
            <publishers/>
            <buildWrappers/>
          </project>
      JCasC:
        enabled: true
        configScripts:
          welcome-message: |
            jenkins:
              systemMessage: Welcome to Jenkins Server.
      customInitContainers: []
      sidecars:
        configAutoReload:
          enabled: false
          image: kiwigrid/k8s-sidecar:0.1.20
          imagePullPolicy: IfNotPresent
          resources: {}
          sshTcpPort: 1044
          folder: "/var/jenkins_home/casc_configs"
        other: []
      nodeSelector: {}
      tolerations: []
        #- key: "node.kubernetes.io/disk-pressure"
        #  operator: "Equal"
        #  effect: "NoSchedule"
        #- key: "node.kubernetes.io/memory-pressure"
        #  operator: "Equal"
        #  effect: "NoSchedule"
        #- key: "node.kubernetes.io/pid-pressure"
        #  operator: "Equal"
        #  effect: "NoSchedule"
        #- key: "node.kubernetes.io/not-ready"
        #  operator: "Equal"
        #  effect: "NoSchedule"
        #- key: "node.kubernetes.io/unreachable"
        #  operator: "Equal"
        #  effect: "NoSchedule"
        #- key: "node.kubernetes.io/unschedulable"
        #  operator: "Equal"
        #  effect: "NoSchedule"
      podAnnotations: {}
      customConfigMap: false
      overwriteConfig: false
      overwriteJobs: false

      jenkinsUrlProtocol: "https"
      # If you set this prefix and use ingress controller then you might want to set the ingress path below
      #jenkinsUriPrefix: "/jenkins"

      ingress:
        enabled: true
        apiVersion: "extensions/v1beta1"
        labels: {}
        annotations: {}
        kubernetes.io/secure-backends: "true"
        kubernetes.io/ingress.class: nginx
        name: ""
        #service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:us-west-2:454211873573:certificate/a3146344-5888-48d5-900c-80a9d1532781 #replace this value
        #service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
        #kubernetes.io/ingress.class: nginx
        #kubernetes.io/tls-acme: "true"
        #path: "/jenkins"
        kubernetes.io/ssl-redirect: "true"
        #nginx.ingress.kubernetes.io/ssl-redirect: "true"
        hostName: ""
        tls:
         #- secretName: jenkins.cluster.local
          # hosts:
          #   - jenkins.cluster.local

      backendconfig:
        enabled: false
        apiVersion: "extensions/v1beta1"
        name:
        labels: {}
        annotations: {}
        spec: {}

      route:
        enabled: false
        labels: {}
        annotations: {}

      additionalConfig: {}
      hostAliases: []

      prometheus:
        enabled: false
        serviceMonitorAdditionalLabels: {}
        scrapeInterval: 60s
        scrapeEndpoint: /prometheus
        alertingRulesAdditionalLabels: {}
        alertingrules: []

      testEnabled: true

    agent:
      enabled: true
      image: "jenkins/jnlp-slave"
      tag: "latest"
      customJenkinsLabels: []
      imagePullSecretName:
      componentName: "jenkins-slave"
      privileged: false
      resources:
        requests:
          cpu: "1"
          memory: "1Gi"
        limits:
          cpu: "1"
          memory: "4Gi"
      alwaysPullImage: false
      podRetention: "Never"
      envVars: []
      # mount docker in agent pod
      volumes:
        - type: HostPath
          hostPath: /var/run/docker.sock
          mountPath: /var/run/docker.sock
      nodeSelector: {}
      command:
      args:
        - echo installing jq;
          apt-get update;
          apt-get install jq -y; 
          apt-get install -y git; 
          apt-get install -y java-1.8.0-openjdk;
          apt-get install awscli;
      sideContainerName: "jnlp"
      TTYEnabled: true
      containerCap: 10
      podName: "default"
      idleMinutes: 0
      yamlTemplate: ""

    persistence:
      enabled: true
      existingClaim: test-project-pvc
      storageClass: test-project-pv
      annotations: {}
      accessMode: "ReadWriteOnce"
      size: "20Gi"
      volumes:
      mounts:
     
    networkPolicy:
      enabled: false
      apiVersion: networking.k8s.io/v1

    rbac:
      create: true
      readSecrets: false

    serviceAccount:
      create: true
      name:
      annotations: {}
-- DeclanG
jenkins
kubernetes
kubernetes-helm

1 Answer

7/30/2020

Please create a PersistentVolumeClaim with following yaml file in the namespace for jenkins (by updating the namespace field):

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: jenkins-tmp-pvc
  namespace: test-project
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
     storage: "10Gi"
storageClassName: gp2

Then add persistence volume, mount and javaOpts as follows in Jenkins values yml file:

master
  ...
  javaOpts: "-Djava.io.tmpdir=/var/jenkins_tmp"
persistence:
  ...
  volumes:
    - name: jenkins-tmp
      persistentVolumeClaim:
        claimName: jenkins-tmp-pvc
  mounts:
    - mountPath: /var/jenkins_tmp
      name: jenkins-tmp

This will first create the persistent volume claim "jenkins-tmp-pvc" and underlying persistent volume and then Jenkins will use the claim mount path "/var/jenkins_tmp" as tmp directory. Also, make sure your "gp2" storageclass is created with "allowVolumeExpansion: true" attribute so that "jenkins-tmp-pvc" is expandable whenever you need to increase tmp disk space.

-- Subodh Hatkar
Source: StackOverflow