Have many PENDING TASK, Although cluster are free many slot in Middle Manager

Dear all,

  • I’m submit task with affinity:
  • in spec ingest:“maxNumConcurrentSubTasks”: 20

  • in each Middle Manager:druid.worker.capacity=32 (32 slot)

  • There are many free slot. but Task still pending

I don’t know why ???

Hi,
Are your tasks dealing with the same segments of the same data source?

If so, then it is normal :

As far as I know, Druid locks the segment when ingesting it and releases it after ingestion is done (successful or not)

So you can’t have parallel ingestion on the same segment

thanks, Guillaume
but how to avoid this locks. Can I have change ingest from batch index to kafka ingestion?