I am setting up Jenkins machine inside a K8S cluster and want cloud properties for my Kubernetes cluster preconfigured.

For this very reason, I want to load a customized config.xml file at startup.

My config.xml is currently inside a configMap called jenkins-config and contains the whole XML file with my edits.

Now with the Jenkins Image that I am using, it loads all configs under /var/jenkins_home/

Which means the config.xml file is under /var/jenkins_home. /var/jenkins_home is being persisted of course.

I am introducing my configMap as a VolumeMount.

My deployment.yaml file is :

spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: jenkins
        release: 1.1.1
    spec:
      containers:
        - name: jenkins
          image: jenkins-master:1.0
          env:
            - name: JAVA_OPTS
              value: -Djenkins.install.runSetupWizard=false
          ports:
            - name: http-port
              containerPort: 8080
            - name: jnlp-port
              containerPort: 54000
          volumeMounts:
            - name: jenkins-home
              mountPath: /var/jenkins_home
              readOnly: false
            - name: jenkins-config
              mountPath: /var/jenkins_home/config.xml
              subPath: config.xml
      volumes:
        - name: jenkins-home
          emptyDir: {}
        - name: jenkins-config
          configMap:
            name: jenkins-config

Now I can access my pod and verify that the new config is indeed there but my Jenkins is giving ERRORs like:

WARNING: Unable to move atomically, falling back to non-atomic move.
java.nio.file.FileSystemException: /var/jenkins_home/atomic1870316694682040724tmp -> /var/jenkins_home/config.xml: Device or resource busy
    at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
    at sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:396)
    at sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:262)
    at java.nio.file.Files.move(Files.java:1395)
    at hudson.util.AtomicFileWriter.commit(AtomicFileWriter.java:191)
    at hudson.XmlFile.write(XmlFile.java:198)
    at jenkins.model.Jenkins.save(Jenkins.java:3221)
    at jenkins.model.Jenkins.saveQuietly(Jenkins.java:3227)
    at jenkins.model.Jenkins.setSecurityRealm(Jenkins.java:2505)
    at jenkins.model.Jenkins$16.run(Jenkins.java:3188)
    at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169)
    at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296)
    at jenkins.model.Jenkins$5.runTask(Jenkins.java:1066)
    at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214)
    at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

Sep 15, 2018 10:06:23 PM hudson.util.AtomicFileWriter commit
INFO: The target file /var/jenkins_home/config.xml was already existing
Sep 15, 2018 10:06:23 PM hudson.util.AtomicFileWriter commit
WARNING: Unable to move /var/jenkins_home/atomic1870316694682040724tmp to /var/jenkins_home/config.xml. Attempting to delete /var/jenkins_home/atomic1870316694682040724tmp and abandoning.
Sep 15, 2018 10:06:23 PM jenkins.model.Jenkins saveQuietly
WARNING: null
java.nio.file.FileSystemException: /var/jenkins_home/config.xml: Device or resource busy
    at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
    at sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:447)
    at sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:262)
    at java.nio.file.Files.move(Files.java:1395)
    at hudson.util.AtomicFileWriter.commit(AtomicFileWriter.java:206)
    at hudson.XmlFile.write(XmlFile.java:198)
    at jenkins.model.Jenkins.save(Jenkins.java:3221)
    at jenkins.model.Jenkins.saveQuietly(Jenkins.java:3227)
    at jenkins.model.Jenkins.setSecurityRealm(Jenkins.java:2505)
    at jenkins.model.Jenkins$16.run(Jenkins.java:3188)
    at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169)
    at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296)
    at jenkins.model.Jenkins$5.runTask(Jenkins.java:1066)
    at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214)
    at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
    Suppressed: java.nio.file.FileSystemException: /var/jenkins_home/atomic1870316694682040724tmp -> /var/jenkins_home/config.xml: Device or resource busy
        at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91)
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
        at sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:396)
        at sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:262)
        at java.nio.file.Files.move(Files.java:1395)
        at hudson.util.AtomicFileWriter.commit(AtomicFileWriter.java:191)
        ... 13 more

Looks like Jenkins loads the default config.xml file and then overwrites it with the one I send which makes Jenkins freak out.

I could make this part of my Docker Image BUT I want to override using K8S rather than making the file in the Image.

Any ideas on how can I safely introduce a config.xml file at startup in Jenkins?

EDIT

Another Attempt ::

I even tried the below config :

volumeMounts:
        - name: jenkins-home
          mountPath: /var/jenkins_home
          readOnly: false
  volumes:
    - name: jenkins-home
      configMap:
        name: jenkins-config
        items:
        - key: config.xml
          path: config.xml

But that yields :

kubectl logs -n jenkins-pipeline jenkins-bc879c4df-m8nlc
touch: cannot touch '/var/jenkins_home/copy_reference_file.log': Read-only file system
Can not write to /var/jenkins_home/copy_reference_file.log. Wrong volume permissions?