Shipping logs from Kubernetes to Fluentd aggregator.
Is there a way to transform one of the key values into the tag value? For example there is key value for application_name. If this could be transformed into the tag value it would be possible to direct to different outputs.
Thanks,
There is no way to edit the tag once the record is created.
The way to do this is to re-emit the record with the rewrite tag filter
You could do something like this:
<match kubernetes_logs>
@type rewrite_tag_filter
<rule>
key application_name
pattern (.+)
tag $1
</rule>
</match>That said, this method makes fluentd to proccess twice as much records. If you only need to direct records to different endpoints, usually output plugins allow to filter records by key.
There is a fluentd record_transformer plugin. https://docs.fluentd.org/v1.0/articles/filter_record_transformer
<filter foo.bar>
@type record_transformer
<record>
hostname "#{Socket.gethostname}"
tag ${tag}
</record>
</filter>Example input: {"message":"hello world!"}
Example output : {"message":"hello world!", "hostname":"db001.internal.example.com", "tag":"foo.bar"}