fluent-bit: can you use tail to parse docker logs that are not JSON

8/26/2021

I have a number of pods in my kubernetes cluster that are outputting logs to /var/log/containers.

Some of the *.log files are strictly JSON format, some are string format, and some are a mix.

In the fluent-bit configuration, I know I can use tail to process the log files and I'm specifying the "docker" Parser. This seems to only be picking up logs in files that are strictly JSON format.

Can I have multiple "tail" INPUT sections which would use the docker/json Parser to parse JSON logs but use some other parser (Regex to match anything) to get logs that are not in JSON format?

Most of the examples seem to assume that all logs are in the same format and mine aren't.

Maybe there is a way to make Kubernetes transform the non-JSON logs to JSON?

-- Becky McDermott
docker
fluent-bit
json
kubernetes
logging

0 Answers