Throttle automatic segments merge process


We’re experiencing some query performance degradation while automatic merge process is running (druid.coordinator.merge.on option). Is there a way to limit number of resources used by this operation?



you can tune mergeSegmentsLimit in your coordinator dynamic configs to achieve this. more docs here -

This is not the same as what I want to achieve. As far as I understand this setting says what are maximum number of segments that may merge into a segment, in a single task. What I need is an ability to limit the number of merge tasks, and I’ve found already that I can play with druid.worker.capacity to achieve that.

FYI, druid.worker.capacity will limit the total tasks that can be ran, not specifically MergeTask, that will also limit your normal indexing tasks too.

Yes, I know. For now we’re not running any real-time tasks on this cluster due to non-timeseries event nature, so this is OK. But in the future we might have, so this looks like a temporary workaround, I agree :slight_smile:

Hi, do you know how to limit mergeTask count(may be 3 or 4), because my druid.worker.capacity=15, I don’t want to mergeTasks influence real-time tasks.

在 2017年2月23日星期四 UTC+8上午3:23:12,Nishant Bangarwa写道: