how to set tags in application(K8S deployment) for fluentd source type format

12/11/2020

Requirement is to receive application logs in port 24224(fluentd port) and store those logs in AWS S3.

Created fluent.conf file to receive the logs in 24224 port and to store in s3 based on matchings.

But not sure how to set tag for application logs in kubernetes Deployment YAML file.

We have multiple applications deployed using Deployment YAML. Now have to set tags for each applications/Deployments.

Fluent.conf

    <source>
      @type  forward
      port  242241
    </source>
    
    <match mongo.** rabbitmq.** ...>
      @type s3
     ...
     ...
    </match>

Probably we have docker-compose file which contains logging driver.

logging:
  driver: "fluentd"
  options:
	fluentd-address: localhost:24224
	tag: mongo.{{.Name}}

Need help to convert this logging/tag method in Kubernetes Deployment file.

-- Prakash26790
fluent-docker
fluentd
kubernetes
logging

1 Answer

12/12/2020

Kubernetes it's self can't ship the logs directly, you will have to configure a log forwarder for example (https://fluentbit.io/).

The below documentation will help you.

-- Tummala Dhanvi
Source: StackOverflow