java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "main"

Hi Druid Gurus,

When i am ingesting data, at very end of indexing task i am getting this error? Can you please tell me whats the issue? i tried bumping up XX:MaxPermSize as well in middleManager runtime.properties. I am running this on druid-0.9.1.1

2017-02-08T21:22:54,042 INFO [task-runner-0-priority-0] io.druid.indexer.IndexGeneratorJob - Adding segment data_insights_2016-07-31T00:00:00.000-07:00_2016-08-01T00:00:00.000-07:00_2017-02-08T20:52:47.486-08:00_4 to the list of published segments

2017-02-08T21:22:54,084 INFO [task-runner-0-priority-0] io.druid.indexer.IndexGeneratorJob - Adding segment data_insights_2016-07-31T00:00:00.000-07:00_2016-08-01T00:00:00.000-07:00_2017-02-08T20:52:47.486-08:00_5 to the list of published segments

2017-02-08T21:22:54,165 INFO [task-runner-0-priority-0] io.druid.indexer.IndexGeneratorJob - Adding segment data_insights_2016-07-31T00:00:00.000-07:00_2016-08-01T00:00:00.000-07:00_2017-02-08T20:52:47.486-08:00_6 to the list of published segments

2017-02-08T21:22:54,220 INFO [task-runner-0-priority-0] io.druid.indexer.IndexGeneratorJob - Adding segment data_insights_2016-07-31T00:00:00.000-07:00_2016-08-01T00:00:00.000-07:00_2017-02-08T20:52:47.486-08:00_7 to the list of published segments

2017-02-08T21:22:54,273 INFO [task-runner-0-priority-0] io.druid.indexer.IndexGeneratorJob - Adding segment data_insights_2016-07-31T00:00:00.000-07:00_2016-08-01T00:00:00.000-07:00_2017-02-08T20:52:47.486-08:00_8 to the list of published segments

2017-02-08T21:22:54,308 INFO [task-runner-0-priority-0] io.druid.indexer.IndexGeneratorJob - Adding segment data_insights_2016-07-31T00:00:00.000-07:00_2016-08-01T00:00:00.000-07:00_2017-02-08T20:52:47.486-08:00_9 to the list of published segments

2017-02-08T21:22:54,308 INFO [task-runner-0-priority-0] io.druid.indexer.JobHelper - Deleting path[hdfs://hdpy/tmp/mars/druid-indexing/data_insights/2017-02-08T205247.486-0800/03a8a5478476471c95458cab48304606]

Exception in thread “main”

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread “main”

middleManager/runtime.properties

druid.service=druid/middleManager

druid.port=8091

Number of tasks per middleManager

druid.worker.capacity=9

Task launch parameters

#druid.indexer.runner.javaOpts=-server -Xmx2g -Duser.timezone=PST -Dfile.encoding=UTF-8 -Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager

#druid.indexer.task.baseTaskDir=var/druid/task

druid.indexer.runner.javaOpts=-server -Xmx3g -Duser.timezone=UTC -Dfile.encoding=UTF-8 -XX:MaxPermSize=512m -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled -XX:+UseG1GC -XX:MaxGCPauseMillis=100 -XX:+PrintGCDetails -XX:+PrintGCTimeStamps

HTTP server threads

druid.server.http.numThreads=66

Processing threads and buffers

druid.processing.buffer.sizeBytes=536870912

druid.processing.numThreads=16

Peon properties

druid.indexer.fork.property.druid.monitoring.monitors=[“com.metamx.metrics.JvmMonitor”]

druid.indexer.fork.property.druid.processing.buffer.sizeBytes=536870912

druid.indexer.fork.property.druid.processing.numThreads=2

druid.indexer.fork.property.druid.server.http.numThreads=50

can you share the logs of the actual task ?

Sims,

Attached are the logs from indexing task.

Karteek

index.logs (690 KB)