Analyze container file system on kubernetes after it exits/crashes

1/24/2020

Perhaps a silly question with no sense:

In a kubernetes deployment (or minikube), when a pod container crashes, i would like to analyze the file system at that moment. In this way, i could see core dumps or any other useful information.

I know that i could mount a volume or PVC to get core dumps from a host-defined core pattern location, and i also could get logs by mean a rsyslog sidecar or any other way, but i still would like to do "post-mortem" analysis if possible. I assume that kubernetes should provide (but i don't know how, that's the reason of my question) some mechanism to do this forensics tasks easing the life to all of us, because in a production system we could need to analyze killed/exited containers.

I tried playing directly with docker run without --rm option, but can't get nothing useful from inspection to get useful information or recreate the file system in last moment that had the container alive.

Thank u very much!

-- eramos
containers
crash
docker
kubernetes
postmortem-debugging

1 Answer

1/24/2020

When a pod container crashes, i would like to analyze the file system at that moment.

POD (Containers) natively use non-persistent storage. When a container exits/terminates, so does the container’s storage.

POD (Container) can be connected to storage that is external. This will allows for the storage of persistent data (you can configure volume mount as path to core dump etc..), since this external storage is not removed when a container is stopped/killed will help you with more flexibility to analysis the file system. Configuring container file system storage with commonly used file systems such as NFS .. etc ..

-- DT.
Source: StackOverflow