Coordinator rules and Historical indexes

Hi,

I am trying to evaluate cold/hot tier setup for our Druid Production system.

Our Deep storage is at 5.1 TB of data at the moment.

Currently, our production system is at 2TB of historical cluster indexes being at _default tier and I would like to setup load rules, drop rules and broadcast rules at the Coordinator level.

Do you think configuring rules at the coordinator level and restart of coordinator will require any other dependencies to be shut and started back?

Do we need explicitly clean up the historical indexes so that indexes are built again as per the new configured rules at the Coordinator level?

Chitra

Hi,

I am trying to evaluate cold/hot tier setup for our Druid Production system.

Our Deep storage is at 5.1 TB of data at the moment.

Currently, our production system is at 2TB of historical cluster indexes being at _default tier and I would like to setup load rules, drop rules and broadcast rules at the Coordinator level.

Do you think configuring rules at the coordinator level and restart of coordinator will require any other dependencies to be shut and started back?

A coordinator or any other service restart is not required at all, In case all your historicals are having same tier currently, you would need to restart them by assigning proper tier in their configs

Do we need explicitly clean up the historical indexes so that indexes are built again as per the new configured rules at the Coordinator level?

Druid coordinator can manage load/drop of segments on individual historical nodes, no manual steps needed.

Thanks for your response, Nishant!

I have few queries, I found only through google groups about how to put the actual setting property label value for hot tier and cold tier in historical runtime properties

  • druid.server.priority how does this property provides value or impact?

  • if I have below infrastructure all being in _default tier currently, can I make 2 nodes to be in hot tier and rest in _default or cold tier:

stat1

stat2

stat3

stat4

stat5

Should I delete existing historical indexes on the changed nodes from _Default to hot tier, to ensure the new coordinator rules are set?