Deployment not detecting change of container image tag in init-container

10/26/2016

I have been using init-containers since they became available and find them super useful. My core image (below as web-dev) does not change much, but my init-container image (below as web-data-dev) does change often.

The init-container uses a container image with a version number. I change this version number to the latest value, and then do kubectl apply -f deployment.yaml

For instance, i change eu.gcr.io/project/web-data-dev:187 to eu.gcr.io/project/web-data-dev:188 before running kubectl apply.

When I do this however, no deployment happens, if i make any changes to the image the init-container uses, the deployment will still not happen. I assume this is because the init-container changes are not being detected.

I then tried to just put some garbage in the image field, like this: "image": "thisIsNotAnImage" and run kubectl apply -f again, but the update is still not applied.

My question is - How do i make kubectl apply -f detect an image tag change in an init-container? am i doing something wrong, is this a bug, or is this simply not implemented yet because init-containers are Alpha?

The full deployment YAML is below.

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: web-deployment
spec:
  replicas: 1
  strategy:
    rollingUpdate:
      maxUnavailable: 0
  template:
    metadata:
      labels:
        app: web
        tier: frontend
      annotations:
        pod.alpha.kubernetes.io/init-containers: '[
            {
                "name": "initialiser1",
                "image": "eu.gcr.io/project/web-data-dev:187",
                "command": ["cp", "-r", "/data-in/", "/opt/"],
                "volumeMounts": [
                    {
                        "name": "file-share",
                        "mountPath": "/opt/"
                    }
                ]
            }
        ]'
    spec:
      containers:

        - image: eu.gcr.io/project/web-dev:20
          name: web
          resources:
            requests:
              cpu: 10m
              memory: 40Mi
          ports:
            - containerPort: 80
              name: http
            - containerPort: 443
              name: https
          volumeMounts:
            - name: file-share
              mountPath: /opt/

      volumes:
        - name: file-share
          emptyDir: {}
-- IaMaCuP
google-cloud-platform
google-kubernetes-engine
kubernetes

1 Answer

10/27/2016

If you are using Kubernetes 1.4, try to change pod.alpha.kubernetes.io/init-containers to pod.beta.kubernetes.io/init-containers.

I can't find a proper issue on GitHub, but behaviour of these two annotations is different. I can do kubectl apply -f with the second one and the deployment will be updated.

You can test it using the example below:

kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: nginx
spec:
  template:
    metadata:
      labels:
        app: nginx
      annotations:
        pod.beta.kubernetes.io/init-containers: '[
            {
                "name": "install",
                "image": "busybox",
                "command": ["/bin/sh", "-c", "echo foo > /work-dir/index.html"],
                "volumeMounts": [
                  {
                    "name": "workdir",
                    "mountPath": "/work-dir"
                    }
                ]
            }
        ]'
    spec:
      volumes:
        - name: workdir
          emptyDir: {}
      containers:
        - name: nginx
          image: nginx
          ports:
            - containerPort: 80
          volumeMounts:
            - name: workdir
              mountPath: /usr/share/nginx/html

Try to change foo to bar and see the result:

$ cat nginx.yaml | kubectl apply -f -
deployment "nginx" created
$ curl $(minikube service nginx --url)
Waiting, endpoint for service is not ready yet...
foo
$ cat nginx.yaml | sed -e 's/foo/bar/g' | kubectl apply -f -
deployment "nginx" configured
$ curl $(minikube service nginx --url)
Waiting, endpoint for service is not ready yet...
bar

The same thing using pod.alpha.kubernetes.io/init-containers:

$ curl $(minikube service nginx --url)
Waiting, endpoint for service is not ready yet...
foo
$ cat nginx.yaml | sed -e 's/foo/bar/g' | kubectl apply -f -
deployment "nginx" configured
$ curl $(minikube service nginx --url)
foo
-- Maciej Strzelecki
Source: StackOverflow