Fluentd store files in azure blob of certain size only

5/10/2020

I am running Fluentd version 1.9.3 in Kubernetes. I have baked an image of fluentd with the azure plugin named "azure-storage-append-blob". Now I am able to write files to azure blob storage, but the issue is fluentd keeps writing logs of very small sizes(KB's or even bytes) continuously to the Blob storage. My requirement is fluentd should only write log files with a minimum size of 5MB to the azure blob storage and it must avoid writing these small log files. I tried using the buffer parameter "chunk_limit_size" within fluentd config, but it seems not to be working. Is there a workaround on this? Any help is appreciated.

-- Sujith M.S
azure
fluentd
kubernetes
logging

0 Answers