How to push a docker image to Google Container Registry using fabric8 maven plugin from Jenkins pipeline: unauthorized

8/9/2018

I have been struggling to push a docker image to a Google Container Registry using fabric8 maven plugin on a Jenkins pipeline. I have checked every question on stackoverflow but none of them solved my problem.

This is my setup:

Kubernetes Cluster running on Google Kubernetes Engine. I have deployed a pod with a Jenkins server that starts agents with the Kubernetes CI plugin, based on this custom image:

FROM openjdk:8

RUN apt-get update && \
    apt-get -y install \
    apt-transport-https \
    ca-certificates \
    curl \
    software-properties-common
RUN curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
RUN add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu xenial stable" && \
    apt-get update &&  \
    apt-get -y install docker-ce

RUN curl https://dl.google.com/dl/cloudsdk/release/google-cloud-sdk.tar.gz > /tmp/google-cloud-sdk.tar.gz

RUN mkdir -p /usr/local/gcloud \
  && tar -C /usr/local/gcloud -xvf /tmp/google-cloud-sdk.tar.gz \
  && /usr/local/gcloud/google-cloud-sdk/install.sh

ENV PATH $PATH:/usr/local/gcloud/google-cloud-sdk/bin

This agent has Java 8 and Docker.

I want to use fabric8 maven plugin to perform the docker image build and push to Google Container Registry. The point is no matter what I do I always end up facing this error (unauthorized):

Unable to push 'eu.gcr.io/myprojectid/gke-springboot-sample' from registry 'eu.gcr.io' : unauthorized: You don't have the needed permissions to perform this operation, and you may have invalid credentials. To authenticate your request, follow the steps in: https://cloud.google.com/container-registry/docs/advanced-authentication -> [Help 1]

NOTE:

  1. I have created a service account associated to my GC project and given it these permissions: project editor and storage admin.

  2. I have configured a Global Jenkins Secret (cloud-registry-credentials) associated to the key.json file belonging to that service account

There are all the things I have tried in my pipeline so far:

1: oauth2accesstoken

stage("Build docker image") {
    agent {
        label 'jenkins-slave'
    }
    steps {
        container('jenkins-slave'){
            script {
                sh "./mvnw clean build fabric8:build"
                withCredentials([file(credentialsId: 'cloud-registry-credentials', variable: 'crc')]) {
                    sh "gcloud auth print-access-token | docker login -u oauth2accesstoken --password-stdin https://eu.gcr.io"
                    sh "./mvnw fabric8:push"
                }
            }
        }
    }
}

OUTPUT:

+ gcloud auth print-access-token
+ docker login -u oauth2accesstoken --password-stdin https://eu.gcr.io

WARNING! Your password will be stored unencrypted in /home/jenkins/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

[Pipeline] sh
Running shell script
+ ./mvnw fabric8:push

[INFO] F8> The push refers to a repository [eu.gcr.io/myprojectid/gke-springboot-sample]
#
[ERROR] F8> Unable to push 'eu.gcr.io/projectid/gke-springboot-sample' from registry 'eu.gcr.io' : unauthorized: You don't have the needed permissions to perform this operation, and you may have invalid credentials. To authenticate your request, follow the steps in: https://cloud.google.com/container-registry/docs/advanced-authentication  [unauthorized: You don't have the needed permissions to perform this operation, and you may have invalid credentials. To authenticate your request, follow the steps in: https://cloud.google.com/container-registry/docs/advanced-authentication ]

2: key.json

stage("Build docker image") {
    agent {
        label 'jenkins-slave'
    }
    steps {
        container('jenkins-slave'){
            script {
                sh "./mvnw fabric8:build"
                withCredentials([file(credentialsId: 'cloud-registry-credentials', variable: 'crc')]) {
                    sh "docker login -u _json_key --password-stdin https://eu.gcr.io < ${crc}"
                    sh "gcloud auth activate-service-account --key-file=${crc}"
                    sh "./mvnw fabric8:push"
                }
            }
        }
    }
}

OUTPUT:

+ docker login -u _json_key --password-stdin https://eu.gcr.io
WARNING! Your password will be stored unencrypted in /home/jenkins/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

[Pipeline] sh
 Running shell script

+ gcloud auth activate-service-account --key-file=****
Activated service account credentials for: [jenkins-cr@myprojectid.iam.gserviceaccount.com]

[Pipeline] sh
 Running shell script
+ ./mvnw fabric8:push

[ERROR] F8> Unable to push 'eu.gcr.io/myprojectid/gke-springboot-sample' from registry 'eu.gcr.io' : unauthorized: You don't have the needed permissions to perform this operation, and you may have invalid credentials. To authenticate your request, follow the steps in: https://cloud.google.com/container-registry/docs/advanced-authentication  [unauthorized: You don't have the needed permissions to perform this operation, and you may have invalid credentials. To authenticate your request, follow the steps in: https://cloud.google.com/container-registry/docs/advanced-authentication ]
-- codependent
fabric8
google-cloud-platform
google-container-registry
jenkins
kubernetes

2 Answers

8/13/2018

After adding the -X flag to the maven ./mvnw fabric8:push command I found the following DEBUG line: AuthConfig: no credentials found.

I had to dig in the source code of the AuthConfigFactory class to find out that the problem was that it didn't find the previously generated /home/jenkins/.docker/config.json file.

The problem is that this container's home is /root so it was looking for it in /root/.docker/config.json and it wasn't there. As a workaround I added sh "cp /home/jenkins/.docker/config.json /root/.docker/config.json" and now it works like a charm:

[DEBUG] F8> AuthConfig: credentials from ~.docker/config.json
[INFO] F8> myprojectid/gke-springboot-sample:0.0.1-SNAPSHOTsnapshot-180813-080943-0179 in 5 seconds 
-- codependent
Source: StackOverflow

8/10/2018

you might be looking for the Jenkins Google Container Registry Auth Plugin.

another possible reason may be, that the user jenkins-cr does not have the required role storage.admin assigned, in Cloud IAM... here's the relevant documentation. just seen you've assigned it; maybe double-check that.

you might also be able to get support there, in case this is rather related to the fabric8 plugin.

also, hence it is a bucket, that bucket's ACL could eventually interfere.

-- Martin Zeitler
Source: StackOverflow