I have docker containers writing logs in json format. When they run on GKE, the logs are shown in StackDriver fine, but when I run the same containers on some VM with kubernetes (not GKE) and use fluentd to route the logs to StackDriver, the log messages arrive escaped and under "log" key.
Example: {"stream":"stdout","log":"{\"time\":\"2019-07-25T09:55:18.2393210Z\", ....
How can I configure fluentd to get the logs in the same format as on GKE (without "log": and unescaped)?
There are few things to consider:
You can configure fluentd's log format with this guide.
You can try some reverse engineering. Fluentd config used by GKE can by studied at following path on fluend Pod: /etc/google-fluentd/config.d/containers.input.conf
You can directly check the GKE config in a ConfigMap called fluentd-gcp-config-v1.2.5. There is some useful information regarding how to config fluentd as non-managed. More details here.
Please let me know if that helped.