I have a service, deployed into a kubernetes cluster, with fluentd set as a daemon set. And i need to diversify logs it receives so they end up in different s3 buckets. One bucket would be for all logs, generated by kubernetes and our debug/error handling code, and another bucket would be a subset of logs, generated by the service, parsed by structured logger and identified by a specific field in json. Think of it one bucket is for machine state and errors, another is for "user_id created resource image_id at ts" description of user actions

The service itself is ignorant of the fluentd, so i cannot manually set the tag for logs based on which s3 bucket i want them to end in. Now, the fluentd.conf i use sets s3 stuff like this:

<match **>
  # docs: https://docs.fluentd.org/v0.12/articles/out_s3
  # note: this configuration relies on the nodes have an IAM instance profile with access to your S3 bucket
  type copy
  <store>
    type s3
    log_level info
    s3_bucket "#{ENV['S3_BUCKET_NAME']}"
    s3_region "#{ENV['S3_BUCKET_REGION']}"
    aws_key_id "#{ENV['AWS_ACCESS_KEY_ID']}"
    aws_sec_key "#{ENV['AWS_SECRET_ACCESS_KEY']}"
    s3_object_key_format %{path}%{time_slice}/cluster-log-%{index}.%{file_extension}
    format json
    time_slice_format %Y/%m/%d
    time_slice_wait 1m
    flush_interval 10m
    utc
    include_time_key true
    include_tag_key true
    buffer_chunk_limit 128m
    buffer_path /var/log/fluentd-buffers/s3.buffer
  </store>
  <store>
  ...
  </store>
</match>

So, what i would like to do is to have something like a grep plugin

<store>
  type grep
  <regexp>
    key type
    pattern client-action
  </regexp>
</store>

Which would send logs into a separate s3 bucket to the one defined for all logs