I am trying to run the filebeat daemon set to get the log for particular app. There are basically two nodegroups:- eai and eai-staging. eai nodgroup have only single namespace by the eai-staging have multiple namespace. I have following filebeat config:

apiVersion: v1
kind: ConfigMap
metadata:
  name: filebeat-config
  labels:
    app: filebeat
data:
  filebeat.yml: |-
    filebeat.inputs:
    - type: log
      fields:
        app_type: "${NAMESPACE}". <<<---- I want this app type to be different based on namespace
        log_type: secure
      fields_under_root: true

    output.logstash:
      hosts: ["${LOGSTASH_HOST}:${LOGSTASH_PORT}"]
      ttl: 1s
      pipelining: 0
    processors:
      - drop_fields:
          fields: ["beat", "host", "input", "offset", "source"]

Filebeat Daemon set

apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: filebeat
  labels:
    app: filebeat
spec:
  updateStrategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: filebeat
    spec:
      nodeSelector:
        nodegroup: eai
      priorityClassName: critical
      terminationGracePeriodSeconds: 30
      containers:
      - name: filebeat
        imagePullPolicy: Always
        image: docker.elastic.co/beats/filebeat:6.5.4
        args: [
          "-c", "/etc/filebeat.yml",
          "-e",
        ]
        env:
        - name: LOGSTASH_HOST
          value: "logstash-headless.etl.svc.cluster.local"
        - name: LOGSTASH_PORT
          value: "5046"
        - name: NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        securityContext:
          runAsUser: 0
          # If using Red Hat OpenShift uncomment this:
          #privileged: true
        resources:
          limits:
            cpu: 100m
            memory: 200Mi
          requests:
            cpu: 100m
            memory: 200Mi
        volumeMounts:
        - name: config
          mountPath: /etc/filebeat.yml
          readOnly: true
          subPath: filebeat.yml
        - name: data
          mountPath: /usr/share/filebeat/data
        - name: app-log
          mountPath: /var/log/app/
          readOnly: true
      volumes:
      - name: config
        configMap:
          name: filebeat-config
      - name: data
        hostPath:
          path: /var/lib/filebeat-data/eai/app-filebeat
          type: DirectoryOrCreate
      - name: app-log
        hostPath:
          path: /var/log/app/
          type: DirectoryOrCreate

Now, how can I get the particular namespace from where the app log is obtained by the filebeat. I tried to deploy one daemon set in the eai namespace in the eai nodegroup. So I can get the namespace for that using metadata.namespace.

But, if I deployed the daemon set in the eai-staging node group in the particular namespace. I will always get the same namespace value.

Is there any way around. Or should I deploy the daemon set in each namespace?

P.S. I could not use the filebeat in the same container because if filebeat is down due to some reason, the pod cannot receive the request for the app