Realtime task vs Compaction task priority not respected

Hi guys,

I’m using Druid 0.13.0 and I have issues in the task priority between index_realtime and compact tasks. As you can see in the pic below, there one realtime task and 2 compact tasks currently running as well lots of compact in the Pending tasks menu. The last pending task is a realtime one.

Then when the running realtime trask finish, the compact task are run, and not the pending realtime one as per expected.

The compact tasks have a priority in the payload:

"context":{"priority":0}

``

while the realtime ones don’t:

"context":{}

``

The realtime tasks have been generated by Tranquility server 0.8.2. Tranquility config has a priority context, set to 100

{

“dataSources” : [

{

“spec” : { …

},

“properties” : {

“task.partitions” : “1”,

“task.replicants” : “1”

},

“context”: {

“priority”: 100

}

}

],

“properties” : {

“zookeeper.connect” : “MY_ZK_HOST”,

“druid.discovery.curator.path” : “/druid/discovery”,

“druid.selectors.indexing.serviceName” : “druid/overlord”,

“http.port” : “8200”,

“http.threads” : “10”,

“druidBeam.firehoseChunkSize”: “1500”

}

}

``

So why the index_realtime tasks don’t get maximum priority?

thank you!

Hi,

in Druid, the task priority is more like a lock acquisition priority rather than execution priority. So, a high priority task can preempt locks of a low priority task.

Please check http://druid.io/docs/latest/ingestion/locking-and-priority.html for details.

Thanks,

Jihoon

Thank you!