More than one zookeeper using docker compose - Issue

Hi Guys,

Am trying to set up a druid cluster using docker-compose given in the example given in https://github.com/apache/druid/tree/master/distribution/docker
I want 3 Zookeeper instances and have my environment file zookeeper host entry as below

druid_zk_service_host=zookeeper-1:2181,zookeeper-2:2182,zookeeper-3:2183

I have “Zookeeper-1”, “zookeeper-2” and “zookeeper-3” running in 3 different containers however, they seem to be running on port “2181” instead of 2183 and 2182 as I have given in the below compose file
Even after mentioning the correct ports, zookeper-2 and zookeeper-3 seem to be running in “2181”. What am i doing wrong? Any help is appreciated.

zookeeper-1:
container_name: zookeeper-1
image: zookeeper:3.5
ports:

  • “2181:2181”
    environment:
  • ZOO_MY_ID=1

zookeeper-2:
container_name: zookeeper-2
image: zookeeper:3.5
ports:

  • “2182:2182”
    environment:
  • ZOO_MY_ID=1

zookeeper-3:
container_name: zookeeper-3
image: zookeeper:3.5
ports:

  • “2183:2183”
    environment:
  • ZOO_MY_ID=1

Hi,

In the container, zookeeper still uses port 2181 by default.

So you should map port 2181 to whatever you want on your host

Here is what should work.

Also not sure what “ZOO_MY_ID” var is.

zookeeper-1:
container_name: zookeeper-1
image: zookeeper:3.5
ports:

  • “2181:2181”
    environment:
  • ZOO_MY_ID=1

zookeeper-2:
container_name: zookeeper-2
image: zookeeper:3.5
ports:

  • “2182:2181”
    environment:
  • ZOO_MY_ID=1

zookeeper-3:
container_name: zookeeper-3
image: zookeeper:3.5
ports:

  • “2183:2181”
    environment:
  • ZOO_MY_ID=1

Hi Guillaume,

Thanks for your reply. I did what you suggested.

zookeeper 1, 2 and 3 are on different containers. I started the services and zookeeper 1 was being used and everything was fine. But as soon as I stopped “zookeeper1” container , expecting the next ranked zookeeper to take over, it tried and I get the below error

2020-06-27T20:12:42,225 INFO [main-SendThread(zookeeper-2:2182)] org.apache.zookeeper.ClientCnxn - Opening socket connection to server zookeeper-2/172.19.0.9:2182. Will not attempt to authenticate using SASL (unknown error)

2020-06-27T20:12:42,227 INFO [main-SendThread(zookeeper-2:2182)] org.apache.zookeeper.ClientCnxn - Socket error occurred: zookeeper-2/172.19.0.9:2182: Connection refused

2020-06-27T20:12:42,328 INFO [main-SendThread(zookeeper-3:2183)] org.apache.zookeeper.ClientCnxn - Opening socket connection to server zookeeper-3/172.19.0.6:2183. Will not attempt to authenticate using SASL (unknown error)

2020-06-27T20:12:42,329 INFO [main-SendThread(zookeeper-3:2183)] org.apache.zookeeper.ClientCnxn - Socket error occurred: zookeeper-3/172.19.0.6:2183: Connection refused

This is my mapping according to your recommendation. ZOO_MY_ID is a environment variable used to specific zookeeper environment files i believe which I am not using.

zookeeper-1:
container_name: zookeeper-1
image: zookeeper:3.5
ports:

  • “2181:2181”
    environment:
  • ZOO_MY_ID=1

zookeeper-2:
container_name: zookeeper-3
image: zookeeper:3.5
ports:

  • “2182:2181”
    environment:
  • ZOO_MY_ID=1

zookeeper-3:
container_name: zookeeper-3
image: zookeeper:3.5
ports:

  • “2183:2181”
    environment:
  • ZOO_MY_ID=1

Top of my head zoo id is the id zooker uses to know which server he is. Try incrementing

Hi Marc,

Thanks a ton for your reply.

I changed the ZOO_MY_ID value for zookeeper-2 and zookeeper-3 to 2 and 3 respectively as you suggested.
The issue after that seemed to zoo keeper itself.

I found the below zookeeper log in zookeeper-3 after changing its ZOO_MY_ID to 3

2020-06-27 20:36:48,918 [myid:3] - ERROR [ContainerManagerTask:ContainerManager$1@84] - Error checking containers

java.lang.NullPointerException

at org.apache.zookeeper.server.ContainerManager.getCandidates(ContainerManager.java:143)

at org.apache.zookeeper.server.ContainerManager.checkContainers(ContainerManager.java:112)

at org.apache.zookeeper.server.ContainerManager$1.run(ContainerManager.java:78)

at java.base/java.util.TimerThread.mainLoop(Unknown Source)

at java.base/java.util.TimerThread.run(Unknown Source)

2020-06-27 20:37:48,910 [myid:3] - ERROR [ContainerManagerTask:ContainerManager$1@84] - Error checking containers

2020-06-27 20:38:48,910 [myid:3] - ERROR [ContainerManagerTask:ContainerManager$1@84] - Error checking containers

2020-06-27 20:39:48,910 [myid:3] - ERROR [ContainerManagerTask:ContainerManager$1@84] - Error checking containers

2020-06-27 20:40:48,910 [myid:3] - ERROR [ContainerManagerTask:ContainerManager$1@84] - Error checking containers

2020-06-27 20:41:48,910 [myid:3] - ERROR [ContainerManagerTask:ContainerManager$1@84] - Error checking containers

2020-06-27 20:42:48,911 [myid:3] - ERROR [ContainerManagerTask:ContainerManager$1@84] - Error checking containers

2020-06-27 20:43:48,911 [myid:3] - ERROR [ContainerManagerTask:ContainerManager$1@84] - Error checking containers