ELK to monitor Kubernetes

3/7/2019

I have Kubernetes cluster runnning and created the ELK stack on different machine. Now I want to ship the logs from Kubernetes cluster to ELK how can I achieve it? The ELK stack is outside the cluster.

-- Manpreet
azure
docker
elastic-stack
kubernetes
performance

3 Answers

3/7/2019

We can use EFK stack for Kubernetes Logging and Monitoring. We need a Kubernetes cluster with following capabilities.

  1. Ability to run privileged containers.
  2. Helm and tiller enabled.
  3. Statefulsets and dynamic volume provisioning capability: Elasticsearch is deployed as stateful set on Kubernetes. It’s best to use latest version of Kubernetes (v 1.10 as of this writing)

Please refer https://platform9.com/blog/kubernetes-logging-and-monitoring-the-elasticsearch-fluentd-and-kibana-efk-stack-part-2-elasticsearch-configuration/ for step by step guide.

-- KZapagol
Source: StackOverflow

3/7/2019

You can run filebeat to collect logs from kubernetes. Follow the instruction of documentation on link:

After you download kubernetes.yaml change:

- name: ELASTICSEARCH_HOST
  value: [your elastic search domain]
- name: ELASTICSEARCH_PORT
  value: "9200"
- name: ELASTICSEARCH_USERNAME
  value: elastic
- name: ELASTICSEARCH_PASSWORD
  value: changeme

Pay attention! You need admin privileges for creating filebeat ServiceAccount

-- ozlevka
Source: StackOverflow

3/7/2019

Have you tried fluentd? Logging agent that collects logs and able to ship logs to Elastic search.

UPDATE

I just found some examples in kops repo. You can check here

-- Veerendra Kakumanu
Source: StackOverflow