Druid cluster allocation

Hello. Everyone

I will build a druid cluster.

But I want to ask how to allocate broker , historical ,etc.

For example, the cluster use 3 historical and everyone is 52GB memory. just like it.

Today…

I have 6 TB data per month. How do I to decide these detail and count of component.

Thanks

Max

Hi Max

You would find general guidelines here -> http://druid.io/docs/latest/tutorials/cluster.html this is a good starting point for Capacity planning for the cluster you are trying to setup.

It is good to split the available nodes into 1. Master node (for co-ordinator and overlord) 2. Query node (broker / router / pivot) and 3. Data node (middle manager and historicals)

You would need at least one Master node and one Query node and several Data nodes. You need to have multiple Master / Query nodes to have high availability (2 should be good)

You also need to determine how much of the data loaded to druid is going to be used often and whatever is used often needs to be planned to be in Memory / tier-1 nodes. Based on that you could have a set of data nodes to manage not so frequently queried to be in tier-2.

Hope it helps!

Thanks & Rgds

Venkat