Druid Cluster setup

Hi folks, I managed to get druid/hive/llap up and running on a very small cluster in our private DC using 3 nodes, using HDP

I am now trying to get it running on much bigger cluster ~15 nodes in AWS, after fighting some networking issues, etc…I believe I have made progress, where HDP has been able to install everything.

but…it doesn’t seam to know exactly how to start stuff up in the right order…

I finally was able to manually start some services via the UI such as HDFS running by starting stuff in the ‘right order’…

but still I am unable to start druid…an no there is no error…on the server or on the client logs, it simply ‘never transitions’ into the started state…the client ambari logs regsiter the requests to start…but after that…nothing seems to happen.

I was able to manually ssh to one of our broker boxes, and start it from the cli, and the ambari-server now sees it as running…

I am in the process of considering dumping ambari/hdp, and trying to install druid manually…

What do most people do… The configuration stack of druid/hive/llap…seems like…ug…quite the nightmare trying to do it manually…but then again …using something ambari, also doesn’t seem to work.

Thanks for any advice/thoughts

I’m not familiar with running Druid within the HDP ecosystem with Hive/LLAP integration, you may want to check with the Hortonworks support channels.



Hi Dan. I am actually considering experimenting with Druid on hdp after creating a manually built Druid cluster on VM infrastructure.

I actually am running my file system on hdp for my druid cluster so I have a little bit of experience with both methodologies.

you’re posting definitely is not selling me on the ease of use of running everything on one large Hadoop cluster however.

Feel free to contact me directly and I might be able to give you some advice on how to get your cluster running.