I am shipping the Kubernetes logs to Elasticsearch with the help of filebeat. I am creating a custom index based on a selector name. That's working fine. But, I want to check for a particular string in the logs. If the keyword "Error" exist, then I want to create a new field (type) in the document and set the value "Error" .

  filebeat.yml: |-

    logging.level: debug

    # To enable hints based autodiscover, remove `filebeat.inputs` configuration and uncomment this:
    filebeat.autodiscover:
      providers:
        - type: kubernetes
          node: ${NODE_NAME}
          hints.enabled: true
          hints.default_config:
            type: container
            paths:
              - /var/log/containers/*${data.kubernetes.container.id}.log
          multiline.pattern: '^\['
          multiline.negate: true
          multiline.match: after

    processors:
      - add_cloud_metadata:
      - add_host_metadata:
    cloud.id: ${ELASTIC_CLOUD_ID}
    cloud.auth: ${ELASTIC_CLOUD_AUTH}
    output.elasticsearch:
      hosts: ['${ELASTICSEARCH_HOST:ES_HOST}:${ELASTICSEARCH_PORT:9200}']
      indices:
        - index: "%{[kubernetes.pod.labels.app]}-filebeat-%{[agent.version]}-%{+yyy.MM.dd}"
          equals:
            #status: OK
          has_fields: ['stream']
      number_of_shards: 1
      scan_frequency: 1s

    setup.template:
      name: 'access-logging'
      pattern: '%{[kubernetes.pod.labels.app]}-*'
      enabled: false

The new field, type should have a value Error if string matches. Else, it will be None .