K
Q

What is the best and simple approach of Kafka microservice upgrade

9/28/2020

We have confluent kafka 4.0.0 used in our env as a microservice, we would like to get it upgraded to 5.5.1 version similarly the zookeeper and kafka Rest. We are running kafka broker and zookeeper as statefulsets and the kafka rest as deployment.

What is the best way of doing this upgrade with rollback procedures on?

  1. Upgrading the image tag to the version 5.5.1 on the running sts and deployments and change all the configurations under the same namespace.

or

  1. Creating all the kafka components in a new namespace with upgraded version and new configuration on it and simply route the traffic here by changing the ingress load balancer.

What would be the best way of upgrading out of these methods? also let me know if there is any other way of doing it.

-- Nishanth
apache-kafka
deployment
docker
kubernetes

Similar Questions

Create a base template for all future helm charts
Problem listing Google Container Registry images via Docker Registry HTTP API V2
RabbitMQ, .NET Core and Kubernetes (configuration)
How to host multiple applications in one Istio service mesh?
Is there a way to share a configMap in kubernetes between namespaces?

1 Answer

9/28/2020

We're doing the 2nd way:

  1. Spin up the new kafka
  2. Spin up the new set of workers listening to the new kafka
  3. Inject message to test workers-kafka integration
  4. Spin up the new app stack feeding the new kafka
  5. Inject traffic to test app-kafka-workers integration
  6. Switch traffic to the new stack

This is much safer than upgrading live as it lets you verify/debug on all the stages.

-- Max Lobur
Source: StackOverflow