fluentd full buffer cannot send log to elastic search

1/15/2020

I find issue, where my fluentd full buffer and cannot send log to elastic. is there a way to manually flush?

this is error log

-- ichsancode
elasticsearch
fluentd
kubernetes
logging
okd

2 Answers

1/23/2020

Arghya's suggestion is correct but there are more options that can help you.

  1. You can set flush_mode to immediate in order to force flush or set or set additional flush parameters in order to adjust it to your needs. You can read more about it here: Control Flushing.

  2. You can also consider using SIGUSR1 Signal:

Forces the buffered messages to be flushed and reopens Fluentd's log. Fluentd will try to flush the current buffer (both memory and file) immediately, and keep flushing at flush_interval.

Please let me know if that helped.

-- OhHiMark
Source: StackOverflow

1/15/2020

Increase flush_thread_count parameter

<match test>
  @type output_plugin
  <buffer ...>
    flush_thread_count 8
    ...
  </buffer>
  ...
</match>
-- Arghya Sadhu
Source: StackOverflow