I have a zookeeper pod that somehow went to Crashloopbackoff and now no matter what I do , it is not coming up even though there is enough space on the node.
Remaining space in node : 46G
Space requested by my pod : 2000Mi
I am able to see this in the logs of the pod
===> Configuring ... Errno 28 No space left on device Command /usr/local/bin/dub template /etc/confluent/docker/myid.template /var/lib/zookeeper/data/myid FAILED !
I have tried
kubectl delete pod zookeeperafter which it just goes back to crashloopbackoff
Is there anyway to recover the pod(statefulset) without terminating and redeploying ?
Please help.
This is most likely an issue with leftover docker images. You should check:
available space with df -h
available inodes with df -i
The most common solution for this issue is to remove unused docker images with docker image prune -a:
WARNING! This will remove all images without at least one container associated to them.
Are you sure you want to continue? [y/N] yYou can also use docker system prune that would delete all containers that were stopped as well as all volumes and networks and that are not used by any container. It will also remove all dangling images.