Apparently the GC of my Kubernetes cluster is failing to delete any image and the server is getting to full-disk.
Can you please guide me on where to find the logs for the ImageGC with the error trying to delete the images or to a reason of why this is happening?
3m 5d 1591 ip-xxx.internal Node Warning FreeDiskSpaceFailed {kubelet ip-xxx.internal} failed to garbage collect required amount of images. Wanted to free 6312950988, but freed 0
3m 5d 1591 ip-xxx.internal Node Warning ImageGCFailed {kubelet ip-xxx.internal} failed to garbage collect required amount of images. Wanted to free 6312950988, but freed 0
Thanks!
most likely your host file system is full, can you check /var file system usage. you can use docker-gc to cleanup old image.
https://github.com/spotify/docker-gc
Run it like this
docker run --rm --privileged -v /var/run/docker.sock:/var/run/docker.sock -v /etc:/etc:ro spotify/docker-gc