kafkajs consumer lag, but messages commited

10/2/2020

need some help with kafkajs consumer lag

one kubernetes pod send 10 messages to queque, last offset in kafdrop is 10, all messages are succesfully delivered to 3rd party system and succesfully, commited

but, and its random, there is consumer lag in kafdrop, sometimes 1 message, sometimes 3.., state in kafdrop is for example: last offset 10, consumer offset 8, but all messages commited by kafkajs and delivered to 3rd party system

if we send another 10 messages to kafka, everythin OK if we restart consumers POD, messages with lag are send and commited twice

kafkajs is set to manual commit after 3rd party response

please some advice

-- Ondrej Basista
consumer
kafkajs
kubernetes
lag
node.js

0 Answers