Task level memory allocation

Hello,

We have a requirement to specify different memory limit for one of the ingestion task. Using druid version 0.10.1.

Is there a link or documentation, that I can refer to achieve this ?

I did searched for this requirement in druid.io documentation, but was not able to find a link, that talks about it.

Any suggestion on this would be of great help.

Thanks,

Mani

I think it’s possible to set per-task JVM flags in the task context: https://github.com/apache/incubator-druid/issues/2320#issuecomment-173858106

Hi Jonathan,

Thanks for the reply.

Rather than mentioning at the task level configuration. Is there a configuration option at middle manager level, that can be specified for a particular job/task ?

We are not manually constructing the ingestion specs.

It would be of great help, if you can provide some pointer with regards to it.

Thanks,

Mani

Rather than mentioning at the task level configuration. Is there a configuration option at middle manager level, that can be specified for a particular job/task ?

The middle manager level only has global configuration options that apply to all tasks.

Another option could be to deploy separate middle managers with different global task memory configurations and use worker select strategies to assign tasks to specific middle managers:

http://druid.io/docs/latest/configuration/index.html#worker-select-strategy