Tranquility fails to send data to druid if druid.host is set to localhost

I am using the example configuration (from the tutorial-example.tar.gz ) i.e. Druid config file: examples/conf/druid/_common/common.runtime.properties . The property druid.host is set to localhost in this config file.

Tranquility when used as a library is unable to send data to druid with the above configuration. It fails with the error:

Failed to send message: %sjava.lang.IllegalStateException: Failed to create merged beam: druid:overlord/pageviews

``

On setting druid.host to machine’s reachable ip address, data can be send successfully.

However when using tranquility as a server, data can be send while druid.host is set to localhost.

Why is the behaviour different when using tranquility as library and as server? How are druid services (like overlord) located by tranquility?

Got reply from Gian on how druid services are located.

Druid services are located by tranquility using zk. However it is still not clear why the behavior is different when using tranquility as server vs library. when using tranquility library one is not able to send data if druid.host is set to localhost (default).

Answer:

This behaviour is because druid uses zookeeper for service discovery. When druid starts up the overlord service would be registered/stored under a zookeeper node. The address stored under this node would be the one set in common.runtime.properties. This would be localhost by default.

So when tranquility finds the zookeeper node for overlord, the address it finds there would be localhost. tranquility of course will not able to use this to send data to druid (i.e. create realtime tasks using the tranquility configuration)

Reference: https://github.com/druid-io/tranquility/issues/186