Monitor the logs of pods from Multiple Kubernetes clusters(Using GCP & Kops) from a common Kibana UI

7/10/2018

I have configured a working EFK(Elasticesearch,Fluentd,Kibana) in one of my kubernetes cluster builded in GCP. I have two more clusters and installed the same EFK in remaining too. Now If I want to monitor the logs of each cluster environment,then I need to check all the three kibana console. Please let me know is it possible to centralize the all EFK builded in three clusters, So that I can manage to see the pod logs from all my clusters in a single Kibana console. Any help or suggestion will be helpful.

-- manu thankachan
efk
google-cloud-platform
kibana
kubernetes

1 Answer

7/10/2018

In fact Kibana only draws and allows to sort/manage data which exists in Elasticsearch. Let's say, you have 3 k8s clusters. Consequently, you have 3 DaemonSet of Fluentd. All you should do - is configure all Fluentd deployments to send data to the one and only Elasticsearch endpoint, to which the Kibana is connected.

-- Konstantin Vustin
Source: StackOverflow