I am running my Fluentd as a sidecar pod in the Kubernetes cluster. I'm trying to collect the logs from two sources:

1) Application pods 2) Various Kubernetes internal pods

I want to apply same filter on both sources, filter being adding Kubernetes metadata. This is that filter:

<filter kubernetes.**>
      @type kubernetes_metadata
      @id filter_kube_metadata
      kubernetes_url "#{ENV['FLUENT_FILTER_KUBERNETES_URL'] || 'https://' + ENV.fetch('KUBERNETES_SERVICE_HOST') + ':' + ENV.fetch('KUBERNETES_SERVICE_PORT') + '/api'}"
      verify_ssl "#{ENV['KUBERNETES_VERIFY_SSL'] || true}"
      ca_file "#{ENV['KUBERNETES_CA_FILE']}"
      skip_labels "#{ENV['FLUENT_KUBERNETES_METADATA_SKIP_LABELS'] || 'false'}"
      skip_container_metadata "#{ENV['FLUENT_KUBERNETES_METADATA_SKIP_CONTAINER_METADATA'] || 'false'}"
      skip_master_url "#{ENV['FLUENT_KUBERNETES_METADATA_SKIP_MASTER_URL'] || 'false'}"
      skip_namespace_metadata "#{ENV['FLUENT_KUBERNETES_METADATA_SKIP_NAMESPACE_METADATA'] || 'false'}"
    </filter>

These are my two sources:

Starting with Kubernetes pods

<source>
      @type tail
      read_from_head true
      tag kubernetes.*
      path /var/log/containers/*.log
      pos_file /var/log/fluentd-containers.log.pos
      exclude_path ["/var/log/containers/fluent*", "....etc"]
      <parse>
        @type kubernetes
        @type "#{ENV['FLUENT_CONTAINER_TAIL_PARSER_TYPE'] || 'json'}"
        time_format %Y-%m-%dT%H:%M:%S.%NZ
      </parse>
    </source>

This is the source definition for my application pods:

<source>
      @type tail
      read_from_head true
      tag application.*
      path /var/log/containers/*.log
      pos_file /var/log/application-containers.log.pos
      exclude_path ["/var/log/containers/fluent*", "...etc..."]
      <parse>
        @type regexp
        expression /(?<log_level>\w*)\s(?<source_ip>[a-zA-Z0-9:.]*[^\s]+)\s-\sUSER_ID:\s(?<user_id>\d*)\s\[(?<date_time>.*\s\+0000)\]\s(?<http_verb>\w*)\s(?<endpoint>\/[a-zA-Z0-9\/]*)\s(?<protocol_version>[a-zA-Z0-9\/\.]*)\s(?<response_code>\d*)\s(?<response_length>[0-9-]*)\s(?<referrer>[a-zA-Z0-9\-\/]*)\s(?<user_agent>[a-zA-Z0-9\/\.]*)\s(?<response_time>[0-9.]*\sms)/
        time_format %d/%b/%Y:%H:%M:%S %z
      </parse>
    </source>

The filter seems to be working per one source at time, either Kubernetes or Application. For ex if I change the filter from <filter kubernetes.**> to <filter application.**> my logs will be enhanced with Kubernetes metadata.

I've tried following this example, but in this example the tag is omitted from the source forward type and my Fluentd won't even boot up, indicating that tag is mandatory for source tail. As well as looking through other wasn't able to figure it out how to have one filter for multiple sources? Any ideas?