Failed to deploy Zookeeper 3.5.1 in kubernetes

11/24/2015

I'd like to setup 3-nodes zookeeper cluster with version 3.5.1. In order to deploy it into kubernetes, I have to make zookeeper node itself to be 0.0.0.0, so it can be started correctly. the configuration as below:

zk1: zoo.cfg

    standaloneEnabled=false
    dynamicConfigFile=/opt/zookeeper/conf/zoo.cfg.dynamic

zoo.cfg.dynamic

    server.1=0.0.0.0:2888:3888:participant;2181
    server.2=10.62.56.192:2889:3889:participant;2182
    server.3=10.62.56.192:2890:3890:participant;2183

zk2: zoo.cfg

    standaloneEnabled=false
    dynamicConfigFile=/opt/zookeeper/conf/zoo.cfg.dynamic

zoo.cfg.dynamic

    server.1=10.62.56.192:2888:3888:participant;2181
    server.2=0.0.0.0:2889:3889:participant;2182
    server.3=10.62.56.192:2890:3890:participant;2183

zk3: zoo.cfg

    standaloneEnabled=false
    dynamicConfigFile=/opt/zookeeper/conf/zoo.cfg.dynamic

zoo.cfg.dynamic

   server.1=10.62.56.192:2888:3888:participant;2181
   server.2=10.62.56.192:2889:3889:participant;2182
   server.3=0.0.0.0:2890:3890:participant;2183

The result is only zk1 can be started, the zk2 and zk3 fails to start. The errors are

   org.apache.zookeeper.server.quorum.QuorumPeerConfig$ConfigException: client address for this server (id = 2) in static config file is 0.0.0.0/0.0.0.0:2181 is different from client address found in dynamic file: /0.0.0.0:2182
    at org.apache.zookeeper.server.quorum.QuorumPeerConfig.setupClientPort(QuorumPeerConfig.java:608)
    at org.apache.zookeeper.server.quorum.QuorumPeerConfig.setupQuorumPeerConfig(QuorumPeerConfig.java:506)
    at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:157)
    at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:110)
    at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:79)

Invalid config, exiting abnormally

The point is that we have to configure the node itself as 0.0.0.0.

-- che yang
apache-zookeeper
kubernetes

1 Answer

12/1/2015

Try: https://hub.docker.com/r/elevy/zookeeper/ which is based on https://github.com/eliaslevy/docker-zookeeper

That is using 1 service per zk pod. Each service has its own "hostname", which can be resolved via DNS from within each zk pod. The names are zookeeper-1, zookeeper-2, and zookeeper-3.

So, instead of config lines like

server.1=0.0.0.0:2888:3888:participant;2181
server.2=10.62.56.192:2889:3889:participant;2182 
server.3=10.62.56.192:2890:3890:participant;2183

you have config lines like:

server.1=zookeeper-1:2888:3888
server.2=zookeeper-2:2888:3888
server.3=zookeeper-3:2888:3888

Note that since each Pod has its own IP, you don't need to use a range of ports, like 2888, 2889, 2890. You can just use 2888/3888 for each one.

-- Eric Tune
Source: StackOverflow