During image upgrade of pods few of the pods are stuck in ContainerCreating state.
kubectl get events has below error: FailedSync kubelet, 10.102.10.34 Error syncing pod, skipping: timeout expired waiting for volumes to attach/mount for pod "default"/"ob-service-1124355621-1th47". list of unattached/unmounted volumes=[timezone default-token-3x1x9]
Docker Logs :
^[[31mERRO^[[0m[240242] Handler for DELETE /v1.22/containers/749d05b355e2b80bffb90d207232d37e3ebc5ff57942c46ce0a2b4ca5950ed0e returned error: Driver devicemapper failed to remove root filesystem 749d05b355e2b80bffb90d207232d37e3ebc5ff57942c46ce0a2b4ca5950ed0e: Device is Busy
^[[31mERRO^[[0m[240242] Error saving dying container to disk: open /var/lib/docker/containers/5d01db2c31a3073cc7fb68f2be5acc45c34583d5f2ae0c0879ec064f90da6943/config.v2.json: no such file or directory
^[[31mERRO^[[0m[240263] Error removing mounted layer 5d01db2c31a3073cc7fb68f2be5acc45c34583d5f2ae0c0879ec064f90da6943: Device is Busyit's a bit hard to debug with just the information you provided, but the general direction you should be looking into is resources of your cluster.
failed to sync usually means the pods can't be fit into any of the workers (maybe adding more will help) or from your error seems like you're trying to "connect" to volumes that are busy and can't accept the connection which fails the pod.
Again lacking details, but let's assume you're on AWS and you have a volume that didn't dismount and now you're trying to connect to it again - the above would be the result pretty much, so you'll need to detach the volume so the new pod can connect to it. if you say there are some pods that are okay with the same image it means you don't have enough volumes and/or some of the current volumes are not available to accept new connection (maybe during the deletion of the old pods they didn't dismount properly)