Can't build docker image with kubernetes-plugin for Jenkins

9/15/2019

I'm using kubernetes-plugin and I have some issue to build docker images on top of K8S pod:

I'm creating POD with:

podTemplate(containers: [
    containerTemplate(
        name: 'docker-build',
        image: 'docker',
        command: 'cat',
        ttyEnabled: true
    )
],
volumes: [
    hostPathVolume(mountPath: '/var/run/docker.sock', hostPath: '/var/run/docker.sock')
])
{
    node(POD_LABEL) {
        stage("Checkout") {
            dir("${env.WORKSPACE}/code") {
                script {
                    // Checkout - Works
                }
            }
        }

        stage ('Build docker images') {
            container('docker-build') {
                dir("${env.WORKSPACE}/code") {
                    sh """
                    ./build-images
                    """
                }
            }
        }
    }
}

But it fails on the docker build step:

Err:1 http://deb.debian.org/debian stretch InRelease
  Temporary failure resolving 'deb.debian.org'
Err:2 http://security.debian.org/debian-security stretch/updates InRelease
  Temporary failure resolving 'security.debian.org'
Err:3 http://deb.debian.org/debian stretch-updates InRelease
  Temporary failure resolving 'deb.debian.org'
Reading package lists...
[91mW: Failed to fetch http://deb.debian.org/debian/dists/stretch/InRelease  Temporary failure resolving 'deb.debian.org'
W: Failed to fetch http://security.debian.org/debian-security/dists/stretch/updates/InRelease  Temporary failure resolving 'security.debian.org'
W: Failed to fetch http://deb.debian.org/debian/dists/stretch-updates/InRelease  Temporary failure resolving 'deb.debian.org'
W: Some index files failed to download. They have been ignored, or old ones used instead.
[0mReading package lists...
Building dependency tree...
Reading state information...
[91mE: Unable to locate package libpam-dev
E: Unable to locate package libpcap0.8-dev
E: Couldn't find any package by glob 'libpcap0.8-dev'
E: Couldn't find any package by regex 'libpcap0.8-dev'
E: Unable to locate package libpq5[0m[91m
E: Unable to locate package libtins-dev
E: Unable to locate package openjdk-8-jdk-headless
E: Unable to locate package python3

When accessing the POD with kubectl exec and trying to build, it fails on the same error:

docker build -t my_test .

When trying to build the same with --network=host it works:

docker build --network=host -t my_test .

I'm trying to understand why it requires --network=host in order to work.

BTW - when I'm in the Jenkins slave pod and I'm trying to download any packages or access to the internet it all works just fine, it happens only when I'm trying to build docker image and trying to download packages during this process.

I suspect that the docker build is failing because of some network misconfiguration or maybe docker network is getting a bad state during this docker build on top of docker ..

So far I have tried:

  • Create the pod with hostNetwork: true and it didn't help.
  • Create the pod with privileged: true and it didn't help.
  • Many other hacks to make the pod running on different network and it did't help as well.

Please assist.

-- EddieM
docker
jenkins
kubernetes

1 Answer

9/15/2019

The latest versions of the AWS EKS-optimized AMI disable the docker bridge network by default. To enable it, add the bootstrap_extra_args parameter to your worker group template. Source.

-- EddieM
Source: StackOverflow