I am running a deployment which contains three containers the app, nginx and cloud sql instance. I have a lot of print statements in my python based app.
Every time a user interacts with the app, outputs are printed. I want to know if these logs are saved by default at any location.
I am worried that these logs might consume the space on the nodes in the cluster running it. Does this happen ? or Kubernetes deployments by default don't save any logs by default?
The applications run in containers usually under Docker and the stdout/stderr logs are saved for the lifetime of the container in the graph directory (usually /var/lib/docker)
You can look at the logs with either:
$ kubectl logs <pod-name> -c <container-in-pod>Or:
$ ssh <node>
$ docker logs <container>If you'd like to know more where they are stored you can go into the /var/lib/docker directory and see the logs stored in JSON format:
$ cd /var/lib/docker/containers
$ find . | grep json.log
./3454a0681100986248fd81856fadfe7cd95a1a6467eba32adb33da74c2c5443d/3454a0681100986248fd81856fadfe7cd95a1a6467eba32adb33da74c2c5443d-json.log
./80a87a9529a55f8d3fb9b814f0158dc91686704222e252b256455bcde48f56a5/80a87a9529a55f8d3fb9b814f0158dc91686704222e252b256455bcde48f56a5-json.log
...
If you'd like to do garbage collection on 'Exited' containers you can read more about it here.
Another way is to set up a cron job that runs periodically on your nodes that does this:
$ docker system prune -a --force