I have 4 kafka brokers and 3 zookeepers deployed upon kubernetes. Out of 4 only 2 kafka brokers are working and rest 2 keep shutting down and restart with below error:
Exiting because log truncation is not allowed for partition byfn-sys-channel-0, current leader's latest offset 2 is less than replica's latest offset 21 (kafka.server.ReplicaFetcherThread)
Below is the config of kafka
KAFKA_ZOOKEEPER_CONNECT zookeeper0:2181,zookeeper1:2181,zookeeper2:2181
KAFKA_UNCLEAN_LEADER_ELECTION_ENABLE false
KAFKA_REPLICA_FETCH_MAX_BYTES 103809024
KAFKA_MIN_INSYNC_REPLICAS 1
KAFKA_MESSAGE_MAX_BYTES 103809024
KAFKA_LOG_RETENTION_MS -1
KAFKA_LOG_DIRS /var/kafkas/kafka2
KAFKA_DEFAULT_REPLICATION_FACTOR 3
KAFKA_BROKER_ID 2
KAFKA_ADVERTISED_LISTENERS PLAINTEXT://kafka2:9092
Please let me know how can i fix this ?
Kafka halting because log truncation is not allowed for topic error shuttng down kafka nodes
Above link shows if log truncation is not allowed for a topic