Make Kubernetes cluster search for image in multiple places

10/22/2018

I want to reference a container image by name and digest only rather than by the full URI.

I have strongly-hashed Kubernetes object spec files that mention docker images. We have several private container image repos (prod, staging, dev) with multiple sub-directories.

I need a way to be able to specify the docker image "search prefixes" without modifying the Kubernetes object spec files.

Example: I have a checked-in hashed file with the following line:

image: something@sha256:2635462354664526623546235645264

The image is first pushed to gcr.io/dev-bucket/commit-hash/something. Then it's copied to gcr.io/staging-bucket/commit-hash/something and finally to gcr.io/prod-bucket/something.

I'd like to be able to tell Kubernetes possible image search locations/prefixes so that I can use that object file without any changes. (Modifying the files becomes a big problem when they form a strongly-hashed tree.)

-- Ark-kun
containers
docker
image
kubernetes

1 Answer

10/23/2018

I think you can use imagePullSecrets.

You should create a docker-registry secret with the url and all authentication needed for each of the stages/buckets.

kubectl create secret docker-registry dev-bucket --docker-server=https://hub.docker.com --docker-username=user --docker-email=user@example.com --docker-password=password

Then while creating a POD you should do something as follows:

apiVersion: v1
kind: Pod
metadata:
  name: test-pod
spec:
  containers:
- name: test-container
  image: <registry_name>/<image_name>:<tagname>
  imagePullSecrets:
  - name: dev-bucket

Please do check the Google Cloud Registry (GCR) with external Kubernetes guide, and how to Pull an Image from a Private Registry

Do not forget about creating the service account in gcloud, with permissions to GCR and keys for the service account.

You should add tags to your images, that way you will be able to push just one image over to different tag.

EDIT:

Here is an example:

$ docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
d1725b59e92d: Pull complete
Digest: sha256:0add3ace90ecb4adbf7777e9aacf18357296e799f81cabc9fde470971e499788
Status: Downloaded newer image for hello-world:latest

Now we will add new tags to it

$ docker tag hello-world gcr.io/project-for-x/hello-world:dev_latest
$ docker push gcr.io/project-for-x/hello-world:dev_latest
The push refers to repository [gcr.io/project-for-x/hello-world]
428c97da766c: Layer already exists
dev_latest: digest: sha256:1a6fd470b9ce10849be79e99529a88371dff60c60aab424c077007f6979b4812 size: 524
$ docker tag hello-world gcr.io/project-for-x/hello-world:stage_latest                                                                                                                    
$ docker push gcr.io/project-for-x/hello-world:stage_latest                                                                                                                               
The push refers to repository [gcr.io/project-for-x/hello-world]
428c97da766c: Layer already exists
stage_latest: digest: sha256:1a6fd470b9ce10849be79e99529a88371dff60c60aab424c077007f6979b4812 size: 524
$ docker tag hello-world gcr.io/project-for-x/hello-world:prod_latest                                                                                                                     
$ docker push gcr.io/project-for-x/hello-world:prod_latest                                                                                                                                
The push refers to repository [gcr.io/project-for-x/hello-world]
428c97da766c: Layer already exists
prod_latest: digest: sha256:1a6fd470b9ce10849be79e99529a88371dff60c60aab424c077007f6979b4812 size: 524

This created a hello-world files inside GCR

General information
Image type  Docker Manifest, Schema 2
Media type  :application/vnd.docker.distribution.manifest.v2+json
Virtual size    :977 B
Uploaded time   :October 24, 2018 at 3:07:49 PM UTC+2
Build ID    :—
Container classification
Digest: sha256:1a6fd470b9ce10849be79e99529a88371dff60c60aab424c077007f6979b4812 
Tags:   dev_latest prod_latest stage_latest 
Repository: hello-world
Project:    project-for-x

That way you can have just one yaml file to deploy the same image on different environments.

-- Crou
Source: StackOverflow