Kubernetes pod logs using fluentd , with ELK

10/26/2018

We need to separate the pod logs for each k8s namespace to be indexed into separate Elasticsearch index. Currently all the logs are going to just one index pattern logstash-%DATE.

Is there a way on the fluntd side or in the logstash to separate the logs and redirect them to different indices based on the k8s namespace from which the logs are coming , what would be the better option in terms of performance , sacalability and operational ease.

Use case:

Elasticsearch is multi-tenant so we need to send the logs of each namespace to a different index pattern so that we can apply access control.

-- Ijaz Ahmad Khan
elasticsearch
fluentd
kubernetes
security

0 Answers