Druid 0.9.0 - Question about a parameter of middle manager configuration

We have a configuration that is set to

druid.indexer.runner.javaOpts=-server -Xmx2g -Duser.timezone=UTC -Dfile.encoding=UTF-8 -Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager

It’s description makes me think that each of the indexing task will start with 2g of heap but what I am observing is not the same. I have 30 worker threads which should cause 60g of RAM to be used but the RAM usage is not less than 10g. Is my understanding of this option incorrect?

I found out after some more hit and trial. The max heap was 2gb but it was not hitting the limit as the segments which were being merged did not have total size that much. At least that is what I understood from my hit and trails.

Any expert can add some other reason for this?

Hi Aseem,

You should read up on the java virtual machine and its command line options to better understand this. Xmx is the maximum heap that will be allocated - if your process does not need 2g then it will not be allocated. You can specify Xms which is the minimum heap size - if you set it to the same value of Xmx then the jvm will always allocated the same amount of memory and it can be easier to reason about your system resource usage.

Regards,

–Ben