New historical node segments loading

Hi guys,

I’ve added a new historical node in the cluster. The segments re-balancing is working (i can see it from the size of segment cache directory) but very slow. What params could i play with to increase the speed of segments re-balancing?
I see in the historical node’s log file that it loads several segments and after that a pause of many seconds, maybe even more than 1 minute.

Thanks, Vadim.

Take a look at this page for relevant configurations:

Your big ones would be druid.coordinator.period and druid.coordinator.startDelay during development so you’re not waiting so long for the coordinator to kick in.

But can i specify the number of chunks to be moved for the coordinator? Cause as i see in the logs it “moves” several chunks at a time so i can guess it does not re-balance all at once? Is the coordinator who do the job by chunk or it’s historical node who loads from mysql metainf by chunk?

You can configure the maxSegmentsToMove which defaults to 5. Use the coordinator console, and click on the edit cluster config icon in the top-left (pencil icon).

Thanks David. Updated the value.