What is the best way for Logging in Kubernetes
Logging in K8S can be an issue as pods can get rescheduled. So as soon as a pod is rescheduled we may loose all the logs. I have read that Shipping logs is one of the options available. We ship the logs using a service (filebeat) to a search engine. My question is around this is it better to use log shipping or use PV for logs persistence?
Similar Questions
2 Answers
Implement the official architecture:
- https://kubernetes.io/docs/concepts/cluster-administration/logging/
- https://github.com/helm/charts/tree/master/stable/filebeat or https://github.com/helm/charts/tree/master/stable/fluent-bit
Yes, it is log shipping.
It's always better to gather logs on a centralized environment like Elasticsearch.
Doing so enables lots of capabilities like aggregations on all logs, searching between them, monitoring, SIEM, etc.
Also if the operators of your K8s cluster are different than your developers (who mainly need to read logs) shipping logs to a log management system allows you maintain security of your cluster, while giving developers fancy dashboards to search and watch their logs!