Hello:
I’m new to Druid. I’m currently have a click stream project (running by docker). It already has kafka and zookeeper running. When I trying to run Druid (which I followed the quickstart demo from druid document), it seems like to run Druid to get Kafka data, it needs to fire another instance of ZooKeeper and Kafka. It ended up with error (see attachement).
I wonder whether I’m on the right path. What’s the best practice to connect Druid to Kafka as consumer on my local machine? I definitely need to run the click stream project to generate the stream data, so I need ZooKeeper and Kafka running from there too.
Thanks!
Julie
What do the logs that the error is referring to say?
Hi, Rachel:
When I run
./bin/start-micro-quickstart
For ZK, the logs said
2020-02-04T14:41:56,972 ERROR [main] org.apache.zookeeper.server.ZooKeeperServerMain - Unexpected exception, exiting abnormally
java.net.BindException: Address already in use
and for broker, coordinator-overlord, historical router and middleManager, the same error said
java.lang.reflect.InaccessibleObjectException: Unable to make public long
I think it’s because my click stream project is running ZK, Kafka Broker…etc already. If I terminate the click stream project, Druid can be run successfully. So I don’t know what is the best practice to run them both when they all required ZK…
Thanks
You can configure druid to use your existing ZK and Kafka. You don’t have to use the one it bundles. For Kafka, just shut down the instance and connect to your existing instance instead.
https://druid.apache.org/docs/latest/tutorials/cluster.html#configure-zookeeper-connection
