Zookeer closing client connections frequently

We are running a druid cluster. We are using zookeeper version 3.4.6 and Druid release 0.8.1. We have been seeing data loss during ingestion and whenever the data loss happens, we see the following exception in zookeeper logs.

2015-11-04 06:30:33,075 [myid:2] - WARN [NIOServerCxn.Factory:] - caught end of stream exception
EndOfStreamException: Unable to read additional data from client sessionid 0x350d2ab4ab40005, likely client has closed socket
at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
at java.lang.Thread.run(Thread.java:744)
2015-11-04 06:30:33,075 [myid:2] - INFO [NIOServerCxn.Factory:] - Closed socket connection for client / which had sessionid 0x350d2ab4ab40005

After googling, we found this problem to be common with ZK version 3.4.6. What is the recommended ZK version for use with Druid 0.8.1?


Most folks use 2.4.6 in production. How often do you guys see ZK disconnects? There’s some useful information online about keeping ZK stable:

Typo: 3.4.6