cassandra node on k8s cannot discover other nodes after statefulset upgrade

3/19/2020

Recently I've discovered a very interesting thing, which I have a three node cassandra cluster running on kubernetes. One day I update the image of the cassandra statefulset and kubernetes upgrade the pod in reversed ordinal order of the pods. Since after the upgrade the ips of all pods will change, so I would expect cassandra-2 can re-discover cassandra-0(which is the seed) after the upgrade. But unfortunately cassandra-2 just cached other two's ips and marked them as down instead of performing a rediscovery.

Does anyone experienced that before?

-- disccip
cassandra
kubernetes

0 Answers