Fluency with forward plugin: how to add kubernetes metadata to logs

9/24/2021

Hey i have a question.

Im using logback-more-appenders(fluency plugin) to send logs to EFK stack (fluent-bit) which is working in kubernetes cluster, but it lacks kubernetes metadata ( like node/pod names).

I know i can use <additionalField></additionalField> in logbck.xml to add Service name (because this is static), but i cannot do it to dynamic parts like node or pod name.

I tried to do it on fluent-bit side using kubernetes filter, but this works only with tail/systemd inputs not a forward one (it parses tag with filename which contains namespce and pod name). Im using forward plugin to send logs from java software to elasticsearch, and in logback.xml i cannot enter dynamic pod name (or i don't know if i can).

Any tips how i can do it? I prefer to send logs using fluency instead of sniffing host container logs.

-- Randers
fluent-bit
fluentd
kubernetes
logback
logging

1 Answer

12/20/2021

In my case, the best i could think of was to change from forward to tail plugin with structured logging (in json).

-- Randers
Source: StackOverflow