Production Recommendation For Druid

HI Folks,

Can anyone help here about number of node needed for highly available production setup of druid?

Since its involve many component , every component should be ha.

We can share some hardware b/w some of the component independent to each other ie

Coordinator and Overlord
Historicals and MiddleManagers
Druid Brokers + UI

Please suggest a way forward for production setup on AWS.

Hi Vipin

For HA, you would need

2 nodes for “coordinator and overlord”

2 nodes for “Druid Brokers + UI”

Many nodes for “Historicals & MiddleManagers” (depending on data size)

Please review
http://druid.io/docs/latest/tutorials/cluster.html
and is a good starting point. Once you setup, you might have to observe the performance and modify Memory / node capacity accordingly.

Thanks & Rgds

Venkat

Hi Vipin,

To add to Venkat’s answer, You could have multiple Broker nodes as well.

regards,

Prathamesh

Thanks Guys.

I have one more question. Lets suppose i have current hot data of 100 GB , then what configuration you suggest for

Historicals and MiddleManagers

Druid Brokers

Thanks in advance.

You could start off with 4 nodes (64 or 128gb Ram; 8 vcpu +).

Broker, depends on the number of users / queries. Since you need HA, you could start with 2 and then progress further as need arises.