How to told MiddleManager to use whole server while Hadoop indexing, not just 1 core?

Hello,

I have r3.8xlarge instance for Druid MiddleManager, but while indexing worker using just 1-2 cores and few of memory:

reserved: 47,705g, actual: 2,722g cpu load: 114,6%

load average: 0,61, 0,91, 0,63

Tasks: 527 total, 1 running, 526 sleeping, 0 stopped, 0 zombie

%Cpu(s): 5,0 us, 0,2 sy, 0,0 ni, 94,8 id, 0,0 wa, 0,0 hi, 0,0 si, 0,0 st

How to get to use all server resources for map reducers? I have 32 cores and 230Gb so it would be nice to use maximum of server power.

Resources for peons

druid.indexer.runner.javaOpts=-server -Xmx32g -XX:+UseG1GC -XX:MaxGCPauseMillis=100 -XX:+PrintGCDetails -XX:+PrintGCTimeStamps
druid.indexer.task.baseTaskDir=var/druid/task

Peon properties

druid.indexer.fork.property.druid.monitoring.monitors=[“com.metamx.metrics.JvmMonitor”]
druid.indexer.fork.property.druid.processing.buffer.sizeBytes=536870912
druid.indexer.fork.property.druid.processing.numThreads=20
druid.indexer.fork.property.druid.segmentCache.locations=[{“path”: “var/druid/segment_cache”, “maxSize”: 0}]
druid.indexer.fork.property.druid.server.http.numThreads=50
druid.indexer.fork.property.druid.storage.archiveBaseKey=prod
druid.indexer.fork.property.druid.storage.archiveBucket=aws-prod-druid-archive
druid.indexer.fork.property.druid.storage.baseKey=prod/v1
druid.indexer.fork.property.druid.storage.bucket=dldruid
druid.indexer.fork.property.druid.storage.type=s3

druid.worker.capacity=9

``

You can use:

druid.indexer.fork.property.druid.processing.numThreads=2 (per task num threads)
druid.worker.capacity=9 (number of tasks)

Fangjin, I also posted a very similar question in: https://groups.google.com/forum/#!topic/druid-user/lQA-rpMy9p4

In Max’s config, he already has, I’ve also tried this config, does not seem MM runing peons in parallel, still only one Peon and same runtime overall

druid.indexer.fork.property.druid.processing.numThread = 20