I am building a a "custom" jenkins agent (based on the public jenkins/agent:jdk8 image) docker image using the below dockerfile:

FROM jenkins/agent:jdk8

USER root
RUN apt-get -qq update \
   && apt-get -qq -y install \
   curl

RUN curl -sSL https://get.docker.com/ | sh


RUN date > /home/jenkins/build-date-root.txt

USER jenkins

RUN date > build-date.txt

I build with:

docker build -t internal/jenkins-custom-agent docker

When I run the image on my local machine with

docker run -it --rm internal/jenkins-custom-agent bash

I can see the 2 added files as expected inside the container:

jenkins@local:~$ pwd
/home/jenkins
jenkins@local:~$ ls -la
total 48
drwxr-xr-x  1 jenkins jenkins 4096 Feb 20 11:35 .
drwxr-xr-x  1 root    root    4096 Feb 15 20:28 ..
-rw-r--r--  1 jenkins jenkins  220 Apr 18  2019 .bash_logout
-rw-r--r--  1 jenkins jenkins 3526 Apr 18  2019 .bashrc
drwxr-xr-x  2 jenkins jenkins 4096 Feb 20 09:38 .gradle
drwxr-xr-x  2 jenkins jenkins 4096 Feb 15 20:28 .jenkins
-rw-r--r--  1 jenkins jenkins  807 Apr 18  2019 .profile
drwxr-xr-x  2 jenkins jenkins 4096 Feb 15 20:28 agent
-rw-r--r--  1 root    root      29 Feb 20 09:38 build-date-root.txt
-rw-r--r--  1 jenkins jenkins   29 Feb 20 09:38 build-date.txt

But when I run a container from the exact same image (pushed to our internal registry) in jenkins running in Kubernetes those files are not there:

11:18:30  + ls -la /home/jenkins
11:18:30  total 40
11:18:30  drwxrwxrwx 10 root    root    4096 Feb 20 10:18 .
11:18:30  drwxr-xr-x  1 root    root    4096 Feb 15 20:28 ..
11:18:30  drwxr-xr-x  3 jenkins jenkins 4096 Feb 20 10:18 .cache
11:18:30  drwxr-xr-x  3 jenkins jenkins 4096 Feb 20 10:18 .config
11:18:30  drwxr-xr-x  2 jenkins jenkins 4096 Feb 15 20:28 .jenkins
11:18:30  drwx------  2 jenkins jenkins 4096 Feb 20 10:18 .ssh
11:18:30  drwxr-xr-x  2 jenkins jenkins 4096 Feb 15 20:28 agent
11:18:30  drwxr-xr-x  3 jenkins jenkins 4096 Feb 20 10:18 caches
11:18:30  drwxr-xr-x  4 jenkins jenkins 4096 Feb 20 10:18 remoting
11:18:30  drwxr-xr-x  3 jenkins jenkins 4096 Feb 20 10:18 workspace

I suspect it has something to do with how its started in kubernetes/jenkins and the cat command:

But I don't understand how that can prevent files that I know is in the image to suddenly disappear.

Is it not possible to "extend" jenkins/agent:jdk8 and add my own custom files/folders etc?

UPDATE:

I found this:

https://support.cloudbees.com/hc/en-us/articles/360031223512-What-you-need-to-know-when-using-Kaniko-from-Kubernetes-Jenkins-Agents?mobile_site=true&page=7

And based on that I have now changed to using

workingDir: /tmp/jenkins

for the two containers and now I can find the files in the image:

16:23:50  + ls -la /home/jenkins
16:23:50  total 40
16:23:50  drwxr-xr-x 1 jenkins jenkins 4096 Feb 20 09:38 .
16:23:50  drwxr-xr-x 1 root    root    4096 Feb 15 20:28 ..
16:23:50  -rw-r--r-- 1 jenkins jenkins  220 Apr 18  2019 .bash_logout
16:23:50  -rw-r--r-- 1 jenkins jenkins 3526 Apr 18  2019 .bashrc
16:23:50  drwxr-xr-x 2 jenkins jenkins 4096 Feb 20 09:38 .gradle
16:23:50  drwxr-xr-x 2 jenkins jenkins 4096 Feb 15 20:28 .jenkins
16:23:50  -rw-r--r-- 1 jenkins jenkins  807 Apr 18  2019 .profile
16:23:50  drwxr-xr-x 2 jenkins jenkins 4096 Feb 15 20:28 agent
16:23:50  -rw-r--r-- 1 root    root      29 Feb 20 09:38 build-date-root.txt
16:23:50  -rw-r--r-- 1 jenkins jenkins   29 Feb 20 09:38 build-date.txt

The do "explain" that in the article but it to low-level for me to understand.