When this command docker run --rm -v $(pwd)/api_tests.conf:/usr/config/api_tests.conf --name api-automation local.artifactory.swg-devops.com/api-automation is ran, api_tests.conf file is mounting as a directory in container instead of file.
I went through Single file volume mounted as directory in Docker and few other similar questions on stack overflow but unable to get the right solution.
I have tested the same code in local mac laptop and here file from local machine mounts to container as a file but locally i don't have docker-in-docker setup.
I have Dockerfile as below.
FROM alpine:latest
MAINTAINER Basavaraj
RUN apk add --no-cache python3 \
&& pip3 install --upgrade pip
WORKDIR /api-automation
COPY . /api-automation
RUN pip --no-cache-dir install .
ENTRYPOINT "some command"and I have the build.sh file as below,
#!/bin/bash
docker pull local.artifactory.swg-devops.com/api-automation
# creating file with name "api_tests.conf" by adding configuration data
echo "configuration data" > api_tests.conf
# it displays all the configuration data written to api_tests.conf
cat $(pwd)/api_tests.conf
docker run --rm -v $(pwd)/api_tests.conf:/usr/.aiops/config/api_tests.conf --name api-automation local.artifactory.swg-devops.com/api-automationNow we are calling build.sh file from gocd environment. Looks like docker run command executed in docker-in-docker(dind) and as a result client which spawns the docker container on a different host and the file (api_tests.conf) being created does not exist on that different host. because of this file (api_tests.conf) is mounting as empty directory in container.
what are the different solutions to mount the file in docker-in-docker environment? Can we share the file (api_tests.conf) which we created to host where the docker container is spawned?
I think the problem you're having is most likely because of using dind, although it's worth pointing out that this issue would also occur if you had mounted the docker socket into another container as well.
This is because when you ask the docker daemon to mount a directory, you're docker client (cli) actually mount the file/directory itself, it's just passing a request to the docker daemon to mount this location from its local file system. And this is where the problem is, because usually this isn't where you think it is, if you're using dind or sharing the docker.socket, and hence the file/directory doesn't exist from the daemons point of view.
So in your case the $(pwd) is possibly being expanded to some well known/existing directory path, and then the docker daemon is mounting this directory portion, since the file doesn't exist. That's my guess at least, as I've seen similar behaviour before when using dind/docker.socket sharing in other set ups.
One crazy solution to this would be to bind mount the files you want into the dind container at startup, and then you could try subsequently bind mounting those files from within the dind container into any subsequent containers. However bear in mind this is precisely the kind of file system usage that's warned against in the dind documentation because of instability and potential data loss, so be warned.
Hope this helps.