I've googled all day and it seems like I have no idea of what to google ;)
So what I want to do is to enhance the logging of my containers which run in the google cloud. The logs are being gathered by fluentd.
My specific yaml looks like this:
apiVersion: apps/v1beta2
kind: Deployment
metadata:
name: nginx-deployment
spec:
selector:
matchLabels:
app: nginx
replicas: 2
template:
metadata:
labels:
app: nginx
spec:
containers:
- env:
- name: TEST
value: "test123"
name: nginx
image: nginx:1.8
ports:
- containerPort: 80As you can see I have an environment variable called 'TEST' with the value 'test123'.
What I want to do now, is that fluentd appends this key/value pair to EACH log line... Is this so hard or am I not even able to do this?
Any help or links would really be appreciated.
Thanks in advance :)
If you are using the sample fluentd from kubernetes which comes with fluentd kubernetes metadata, then instead of adding it to the env alone, add it as a label. From there fluentd will by default add kube metadata on every log.
From what I can tell, I don't think there's an easy way (or any way, really) to have FluentD add environment variables defined within a pod's containers to the log messages sent by those containers.
Your best bet would be to actually change the Syslog configuration of the containers of that pod in such a way that it'd add those environment variables to the log messages. You can then easily filter those messages via a new ConfigMap for the default FluentD DaemonSet running by default on Kubernetes clusters hosted on Google Cloud IF you want to do any kind of additional processing to them.
By filter, I mean actually filtering messages according to their content. The default FluentD deployment uses a container that doesn't include fluent-plugin-kubernetes_metadata_filter, so you can't filter by pod name AFAIK.