In the K8S environment, FluentBit outputs to Kafka, can establishes a connection and creates a topic, but generates data report error

2/5/2021

Ask a question. In the K8S environment, FluentBit outputs to Kafka, can establishes a connection and creates a topic, but generates data report error.

[error] [output:kafka:kafka.0] fluent-bit#producer-1: [thrd:data39:9092/176]: data39:9092/176: Failed to resolve 'data39:9092': Name or service not known (after 197ms in state CONNECT)

I tried FluentBit1.5 and 1.6 and had the same problem. I only changed the Kafka broker and topic in the FluentBit configuration file, everything else is the default configuration. I checked /etc/hosts and there is no problem.

The Kafka profile is set to: listeners=PLAINTEXT://0.0.0.0:9092 advertised.listeners=PLAINTEXT://x.x.x.x:9092

Also, FluentBit output to ElasticSearch is normal.

Ask everyone to help, thank you very much.

-- user3804623
apache-kafka
fluent-bit
kubernetes

1 Answer

2/17/2021

Posting this answer as a community wiki as original poster of this question provided a solution to his issue in the comments.

Feel free to edit and improve this answer.

it works. in fluent-bit-ds.yaml - mountPath: /etc/hosts name: hosts readOnly: true

The changes most probably were made to the fluent-bit-ds.yaml from this site:


Additional resources:

-- Dawid Kruk
Source: StackOverflow