I created a ElasticSearch service in AWS and using filebeat as deamonset in my kubernetes setup to forward the logs.
Below is my deamonset yaml:
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: filebeat
namespace: kube-system
labels:
k8s-app: filebeat
spec:
template:
metadata:
labels:
k8s-app: filebeat
spec:
serviceAccountName: filebeat
terminationGracePeriodSeconds: 30
containers:
- name: filebeat
image: docker.elastic.co/beats/filebeat:6.3.2
args: [
"-c", "/etc/filebeat.yml",
"-e",
]
env:
- name: ELASTICSEARCH_HOST
value: ${elasticsearch_host}
- name: ELASTICSEARCH_PORT
value: "443"
securityContext:
runAsUser: 0
resources:
limits:
memory: 256Mi
requests:
cpu: 100m
memory: 100Mi
volumeMounts:
- name: config
mountPath: /etc/filebeat.yml
readOnly: true
subPath: filebeat.yml
- name: inputs
mountPath: /usr/share/filebeat/inputs.d
readOnly: true
- name: data
mountPath: /usr/share/filebeat/data
- name: varlibdockercontainers
mountPath: /var/lib/docker/containers
readOnly: true
volumes:
- name: config
configMap:
defaultMode: 0600
name: filebeat-config
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containers
- name: inputs
configMap:
defaultMode: 0600
name: filebeat-inputs
# We set an `emptyDir` here to ensure the manifest will deploy correctly.
# It's recommended to change this to a `hostPath` folder, to ensure internal data
# files survive pod changes (ie: version upgrade)
- name: data
emptyDir: {}
Deamonset is running fine in K8s. I don't see any error but I am not getting any logs in ELK. while I am checking Kibana. I am getting this msg.
I was using the same filbeat yaml template for other environment its working but when I just created a new Elasticsearch service and using the new endpoint to forward the logs Its not working. I have also attached the description of deamonset. I don't see any events there. Could it might be the issue??