Uploading a crash report when an application terminates abnormally in a Docker container on Kubernetes

8/11/2021

I'm using Sentry's Native SDK with Google Breakpad (which uses in-process crash reporting) as the backend to report crashes for a game server written in C++. The game servers are run in Docker containers in a Kubernetes cluster. The container is running linux - specifically Ubuntu 20.04.

When my main game server process crashes the Docker container seemingly stops before Sentry / breakpad is able to send the report. Given that breakpad performs it's reporting in process I actually expected it would complete uploading before exit - but it seems that's not the case. Usually this wouldn't be a problem as Sentry checks for previously unsent reports (it persists them to disk) at (re)initialisation and sends them, however my Kubernetes cluster has no persistent storage volumes and I'd prefer to avoid adding them just for this. This means Sentry doesn't get the opportunity to submit any previously unsent reports that were persisted to disk when the application / container is restarted within the cluster - as there are none.

I considered adding a Kubernetes hostPath volume so that when a new game server pod is started on the node it will submit any unsent reports, but as mentioned - I'd prefer to avoid this if possible. I'm considering writing some supervising process that launches and monitors the game server process & when that process exits it will checks for unsent crash reports (they currently get written to ~/.sentry-native) - but perhaps there is a more sensible solution I'm overlooking?

-- dbotha
docker
kubernetes
kubernetes-pod
sentry

0 Answers