Why are my CloudWatch logs not appearing?

9/14/2018

This is my first time running a large number of pods in Kubernetes. I realized my logs aren't flowing into CloudWatch. Note that it was working well with no delay previously when there were fewer pods to log. The latest log in CloudWatch is 15 minutes ago. I have double the number of pods running now and in the Kubernetes dashboard the pod logs are showing.

Please advice:

  • Is this normal?
  • Will the logs appear in CloudWatch after awhile?
  • How can I check if fluentd logging container in kube-system namespace is doing ok?
  • Should I bump up the amount of resource limit/requested by fluentd container and are there good guidelines on this, e.g. 20 pods setting, 50 pods setting ?
-- cryanbhu
amazon-cloudwatchlogs
fluentd
kubernetes
kubernetes-dashboard

0 Answers