I recently evaluated Kubernetes with a simple test project and I was able to update image of StatefulSet with command like this:

kubectl set image statefulset/cloud-stateful-set cloud-stateful-container=ncccloud:v716

I'm now trying to get our real system to work in Kubernetes and the pods don't do anything when I try to update image, even though I'm using basically the same command.

It says:

statefulset.apps "cloud-stateful-set" image updated

And kubectl describe statefulset.apps/cloud-stateful-set says:

Image: ncccloud:v716"

But kubectl describe pod cloud-stateful-set-0 and kubectl describe pod cloud-stateful-set-1 say:

"Image: ncccloud:latest"

The ncccloud:latest is an image which doesn't work:

$ kubectl get pods
NAME                               READY     STATUS             RESTARTS   AGE
cloud-stateful-set-0               0/1       CrashLoopBackOff   7          13m
cloud-stateful-set-1               0/1       CrashLoopBackOff   7          13m
mssql-deployment-6cd4ff766-pzz99   1/1       Running            1          55m

Another strange thing is that every time I try to apply the StatefulSet it says configured instead of unchanged.

$ kubectl apply -f k8s/cloud-stateful-set.yaml
statefulset.apps "cloud-stateful-set" configured

Here is my cloud-stateful-set.yaml:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: cloud-stateful-set
  labels:
    app: cloud
    group: service
spec:
  replicas: 2
#  podManagementPolicy: Parallel
  serviceName: cloud-stateful-set
  selector:
    matchLabels:
      app: cloud
  template:
    metadata:
      labels:
        app: cloud
        group: service
    spec:
      containers:
        - name: cloud-stateful-container
          image: ncccloud:latest
          imagePullPolicy: Never
          ports:
            - containerPort: 80
          volumeMounts:
            - name: cloud-stateful-storage
              mountPath: /cloud-stateful-data
  volumeClaimTemplates:
    - metadata:
        name: cloud-stateful-storage
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 10Mi

Here is full output of kubectl describe pod/cloud-stateful-set-1:

Name:           cloud-stateful-set-1
Namespace:      default
Node:           docker-for-desktop/192.168.65.3
Start Time:     Tue, 02 Jul 2019 11:03:01 +0300
Labels:         app=cloud
                controller-revision-hash=cloud-stateful-set-5c9964c897
                group=service
                statefulset.kubernetes.io/pod-name=cloud-stateful-set-1
Annotations:    <none>
Status:         Running
IP:             10.1.0.20
Controlled By:  StatefulSet/cloud-stateful-set
Containers:
  cloud-stateful-container:
    Container ID:   docker://3ec26930c1a81caa39d5c5a16c4e25adf7584f90a71e0110c0b03ecb60dd9592
    Image:          ncccloud:latest
    Image ID:       docker://sha256:394427c40e964e34ca6c9db3ce1df1f8f6ce34c4ba8f3ab10e25da6e89678830
    Port:           80/TCP
    Host Port:      0/TCP
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    139
      Started:      Tue, 02 Jul 2019 11:19:03 +0300
      Finished:     Tue, 02 Jul 2019 11:19:03 +0300
    Ready:          False
    Restart Count:  8
    Environment:    <none>
    Mounts:
      /cloud-stateful-data from cloud-stateful-storage (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-gzxpx (ro)
Conditions:
  Type           Status
  Initialized    True
  Ready          False
  PodScheduled   True
Volumes:
  cloud-stateful-storage:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  cloud-stateful-storage-cloud-stateful-set-1
    ReadOnly:   false
  default-token-gzxpx:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-gzxpx
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason                 Age                From                         Message
  ----     ------                 ----               ----                         -------
  Normal   Scheduled              19m                default-scheduler            Successfully assigned cloud-stateful-set-1 to docker-for-desktop
  Normal   SuccessfulMountVolume  19m                kubelet, docker-for-desktop  MountVolume.SetUp succeeded for volume "pvc-4c9e1796-9c9a-11e9-998f-00155d64fa03"
  Normal   SuccessfulMountVolume  19m                kubelet, docker-for-desktop  MountVolume.SetUp succeeded for volume "default-token-gzxpx"
  Normal   Pulled                 17m (x5 over 19m)  kubelet, docker-for-desktop  Container image "ncccloud:latest" already present on machine
  Normal   Created                17m (x5 over 19m)  kubelet, docker-for-desktop  Created container
  Normal   Started                17m (x5 over 19m)  kubelet, docker-for-desktop  Started container
  Warning  BackOff                4m (x70 over 19m)  kubelet, docker-for-desktop  Back-off restarting failed container

Here is full output of kubectl describe statefulset.apps/cloud-stateful-set:

Name:               cloud-stateful-set
Namespace:          default
CreationTimestamp:  Tue, 02 Jul 2019 11:02:59 +0300
Selector:           app=cloud
Labels:             app=cloud
                    group=service
Annotations:        kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"apps/v1","kind":"StatefulSet","metadata":{"annotations":{},"labels":{"app":"cloud","group":"service"},"name":"cloud-stateful-set","names...
Replicas:           2 desired | 2 total
Pods Status:        2 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
  Labels:  app=cloud
           group=service
  Containers:
   cloud-stateful-container:
    Image:        ncccloud:v716
    Port:         80/TCP
    Host Port:    0/TCP
    Environment:  <none>
    Mounts:
      /cloud-stateful-data from cloud-stateful-storage (rw)
  Volumes:  <none>
Volume Claims:
  Name:          cloud-stateful-storage
  StorageClass:
  Labels:        <none>
  Annotations:   <none>
  Capacity:      10Mi
  Access Modes:  [ReadWriteOnce]
Events:
  Type    Reason            Age   From                    Message
  ----    ------            ----  ----                    -------
  Normal  SuccessfulCreate  25m   statefulset-controller  create Pod cloud-stateful-set-0 in StatefulSet cloud-stateful-set successful
  Normal  SuccessfulCreate  25m   statefulset-controller  create Pod cloud-stateful-set-1 in StatefulSet cloud-stateful-set successful

I'm using Docker Desktop on Windows, if it matters.