Openshift 3.9 How to ADD plugin to fluentD

3/22/2019

I have the EFK stack and want to add nginx log parses to fluentD in Openshift. I read https://docs.fluentd.org/v1.0/articles/parser_nginx but I don't understand how to add a plugin in a daemon set. How is plugin added in fluentd to parse logs.

-- Danny
efk
fluentd
kibana
kubernetes
openshift

1 Answer

3/22/2019

Thanks for attaching configmap and deployment yaml. Can you try below config.

From Parser Plugin Overview. They provided a simple example to get nginx logs.

<source>
  @type tail
  path /path/to/input/file
  <parse>
    @type nginx
    keep_time_key true
  </parse>
</source>

Can you add above config in configmap and try. The nginx parser is in-built, the link you provided is explanation of parser internals. If you don't like the way it is selecting/parsing, you can write your own.

Anyways, by using @type we can define type of parser we want.

PS: Right now I dont have any setup to test this. Also I have little knowledge in fluentd

EDIT:

I believe you already knew how fluentd is getting logs... In your pastebin link, at volumeMounts section, it mounting host's /var/lib/docker/containers which has all container meta data and logs, etc

-- Veerendra Kakumanu
Source: StackOverflow