Uneven distribution of historical segments

TROUBLESHOOTING Google Groups
#1

Hi

I tried changing druid.coordinator.balancer.strategy to cost and diskNormalized

Still few historical nodes are always at high disk usgae

#2

As discussed in Apache druid slack channel - just summarizing the details here which may be helpful for the community

Basic info :

  • Druid version 0.13

  • Two tiers ( Hot(42 nodes) /cold(23 nodes))

  • Disk imbalance
    Min -83% uses | Max 97% uses
    Shilpa had tried , druid.coordinator.balancer.strategy=diskNormalized but it dis not help.

  • setting druid.coordinator.balancer.strategy=cachingCost has helped in balancing the segments appropriately on each historicals.

  • Post this change and restarting the historical.

after some time: still rebalancing in progress

Min -82% uses | Max 89% uses

Min= one of the historical consuming MINIMUM disk % ( for segment cache)

Max=one of the historical consuming MAXIMUM disk % ( for segment cache)

Thanks and Regards,

Vaibhav