Fluent Bit With AWS EKS Cluster And ELK

9/9/2018

I'm new to Fluent Bit and I'd love to get help from you.

I have trying unsuccessfully so far to embed Fluent Bit tool within my AWS EKS cluster and collect service logs includes ingress-nginx on top of AWS ELB. In addition, I don't want to create a new ElasticSearch/Kibana cluster since I want to use my existing Apache Kafka & ELK stack.

What has been recently deployed on my Kubernetes cluster

Namespace

kubectl create namespace logging

Service Account

kubectl create -f https://raw.githubusercontent.com/fluent/fluent-bit-kubernetes-logging/master/fluent-bit-service-account.yaml

Role

kubectl create -f https://raw.githubusercontent.com/fluent/fluent-bit-kubernetes-logging/master/fluent-bit-role.yaml

kubectl create -f https://raw.githubusercontent.com/fluent/fluent-bit-kubernetes-logging/master/fluent-bit-role-binding.yaml

Fluent Bit to Kafka

kubectl create -f https://raw.githubusercontent.com/fluent/fluent-bit-kubernetes-logging/master/output/kafka/fluent-bit-configmap.yaml

kubectl create -f https://raw.githubusercontent.com/fluent/fluent-bit-kubernetes-logging/master/output/kafka/fluent-bit-ds.yaml

Even though I have deployed all of this, the logs have not yet reached the ELK's server. It's important to note that I've only changed the Brokers address within this filefluent-bit-configmap.yaml file to my Apache Kafka address (Communication between servers exists).

-- James
amazon-eks
amazon-web-services
fluent-bit
kubectl
kubernetes

0 Answers