I have a Fluentd config that takes logs from a particular K8s namespace (app-ns), and uses a multi-parser to parse the output as JSON, or ignore any special parsing. It currently looks like this:

    <system>
      log_level info
    </system>

    <match kubernetes.**>
      @type rewrite_tag_filter
      <rule>
        key $['kubernetes']['namespace_name']
        pattern ^(.+)$
        tag $1.${tag}
      </rule>
    </match>


    <filter app-ns.**>
        @type parser
        format multi_format
        time_parse false
        suppress_parse_error_log true
        ignore_key_not_exist true
        key_name log
        reserve_data true
        hash_value_field app_log
        <parse>
            @type multi_format
            <pattern>
                format json
            </pattern>
            <pattern>
                format none
            </pattern>
        </parse>
    </filter>

    <filters for other namespaces>


    <match kubernetes.** kubelet.** docker.** kube-proxy.** kube-apiserver.** kube-controller-manager.** kube-scheduler.** default.** kube-system.** ap-ns.** <other namespaces> >
      @type kinesis_streams
      @id out_kinesis_streams
      <configs for kinesis>
    </match>

    <match **>
       @type null
    </match>

There are services in that namespace that use JSON logging, and cause a conflict in the fields in our ELK stack, causing dropped logs. What I hope to achieve is having one deployment be allowed to be parsed as JSON, but my attempts haven't worked.

I was thinking something like this:

    <match app-ns.**>
    @type rewrite_tag_filter
      <rule>
        key $['kubernetes']['container_name']
        pattern <regex for container to match on>
        tag app-ns-json
      </rule>
    </match>

    <filter app-ns-json.**>
        @type parser
        format multi_format
        time_parse false
        suppress_parse_error_log true
        ignore_key_not_exist true
        key_name log
        reserve_data true
        hash_value_field app_log
        <parse>
            @type multi_format
            <pattern>
                format json
            </pattern>
            <pattern>
                format none
            </pattern>
        </parse>
    </filter>

But I can't seem to get it to work. When I tried this, no logs made it out to our ELK stack.

Any suggestions would be appreciated!