GKE, Stackdriver Kubernetes Engine Monitoring and custom log format

8/16/2019

I'm trying to use Stackdriver Kubernetes Engine Monitoring to monitor my GKE cluster (as opposed to Legacy Stackdriver), as this is recommended by https://cloud.google.com/monitoring/kubernetes-engine/ page (despite being beta).

However, I'd like to use custom log parses, as some of my containers use proprietary log formats.

My understanding:

  • it's a matter of customizing fluentd config map (in my case it's fluentd-gcp-config-v1.2.6 in kube-system namespace), however this config map is managed by GKE and can be replaced anytime.

  • with Legacy Stackdriver it was possible to disable the logging alone, and deploy fluentd-gcp manually - as described e.g. here: https://cloud.google.com/solutions/customizing-stackdriver-logs-fluentd . However, with new fancy Stackdriver Kubernetes Engine Monitoring it is not possible to disable logging anymore (and only leave metrics support).

Is there any other way to support custom log formats?

-- Code Painters
google-cloud-stackdriver
google-kubernetes-engine
kubernetes

0 Answers