Hadoop index task

Hi All ,

I’m seeing my hadoop index task job remain stuck forever … I don’t see any active @ hadoop …it just says accepted …my hadoop is pretty big 40 core cluster 5 node cluster …given below data size …

I even tried to change yarn-site setting and place /config/_common/ . However , there is no minor change … …Here is the last thing I see at indexer log @ overlord …

016-05-12 13:23:24 31.7 MiB json/click/y=2016/m=04/d=28/H=01/part-r-00000-958070e5-3f35-4ccf-9001-e77118b7b22b

2016-05-12 13:23:26 31.8 MiB json/click/y=2016/m=04/d=28/H=01/part-r-00001-958070e5-3f35-4ccf-9001-e77118b7b22b

2016-05-12 13:23:27 30.5 MiB json/click/y=2016/m=04/d=28/H=01/part-r-00002-958070e5-3f35-4ccf-9001-e77118b7b22b

Total Objects: 3

Total Size: 94.0 MiB

2016-05-12T22:59:12,736 WARN [task-runner-0] org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform… using builtin-java classes where applicable

10.591: [GC pause (G1 Evacuation Pause) (young), 0.0108042 secs]

[Parallel Time: 5.5 ms, GC Workers: 8]

[GC Worker Start (ms): Min: 10590.9, Avg: 10591.0, Max: 10591.1, Diff: 0.1]

[Ext Root Scanning (ms): Min: 0.4, Avg: 1.0, Max: 2.8, Diff: 2.4, Sum: 8.0]

[Update RS (ms): Min: 0.0, Avg: 0.4, Max: 0.8, Diff: 0.8, Sum: 3.3]

[Processed Buffers: Min: 0, Avg: 4.4, Max: 17, Diff: 17, Sum: 35]

[Scan RS (ms): Min: 0.0, Avg: 0.0, Max: 0.1, Diff: 0.1, Sum: 0.4]

[Code Root Scanning (ms): Min: 0.0, Avg: 0.2, Max: 1.1, Diff: 1.1, Sum: 1.9]

[Object Copy (ms): Min: 2.4, Avg: 3.5, Max: 3.9, Diff: 1.5, Sum: 28.1]

[Termination (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]

[Termination Attempts: Min: 1, Avg: 1.0, Max: 1, Diff: 0, Sum: 8]

[GC Worker Other (ms): Min: 0.0, Avg: 0.1, Max: 0.1, Diff: 0.1, Sum: 0.4]

[GC Worker Total (ms): Min: 5.2, Avg: 5.3, Max: 5.3, Diff: 0.1, Sum: 42.1]

[GC Worker End (ms): Min: 10596.2, Avg: 10596.3, Max: 10596.3, Diff: 0.1]

[Code Root Fixup: 0.2 ms]

[Code Root Purge: 0.0 ms]

[Clear CT: 0.2 ms]

[Other: 4.8 ms]

[Choose CSet: 0.0 ms]

[Ref Proc: 4.2 ms]

[Ref Enq: 0.1 ms]

[Redirty Cards: 0.2 ms]

[Humongous Register: 0.0 ms]

[Humongous Reclaim: 0.0 ms]

[Free CSet: 0.3 ms]

[Eden: 295.0M(295.0M)->0.0B(281.0M) Survivors: 7168.0K->21.0M Heap: 373.0M(504.0M)->91.5M(504.0M)]

[Times: user=0.04 sys=0.00, real=0.01 secs]

2016-05-12T22:59:12,926 INFO [task-runner-0] org.apache.hadoop.mapreduce.JobSubmitter - number of splits:35

2016-05-12T22:59:13,017 INFO [task-runner-0] org.apache.hadoop.mapreduce.JobSubmitter - Submitting tokens for job: job_1461518911968_0016

2016-05-12T22:59:13,117 INFO [task-runner-0] org.apache.hadoop.mapred.YARNRunner - Job jar is not present. Not adding any jar to the list of resources.

2016-05-12T22:59:13,594 INFO [task-runner-0] org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted application application_1461518911968_0016

\2016-05-12T22:59:13,622 INFO [task-runner-0] io.druid.indexer.DetermineHashedPartitionsJob - Job click-determine_partitions_hashed-Optional.of([2016-04-28T00:00:00.000Z/2016-04-28T01:00:00.000Z]) submitted, status available at:

2016-05-12T22:59:13,622 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job - Running job: job_1461518911968_0016

Hi All ,

I believe when we start the hadoop indexer (io.druid.cli.Main index hadoop config.json) it submit the jars …not sure If I’m missing anything here …? …

016-05-13T00:33:27,317 WARN [main] org.apache.hadoop.mapreduce.JobSubmitter - Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.

2016-05-13T00:33:27,323 WARN [main] org.apache.hadoop.mapreduce.JobSubmitter - No job jar file set. User classes may not be found. See Job or Job#setJar(String).

2016-05-13T00:33:27,659 INFO [main] org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 4

2016-05-13T00:33:27,672 WARN [main] org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform… using builtin-java classes where applicable

2016-05-13T00:33:28,070 INFO [main] org.apache.hadoop.mapreduce.JobSubmitter - number of splits:35

2016-05-13T00:33:28,146 INFO [main] org.apache.hadoop.mapreduce.JobSubmitter - Submitting tokens for job: job_1461518911968_0018

2016-05-13T00:33:28,241 INFO [main] org.apache.hadoop.mapred.YARNRunner - Job jar is not present. Not adding any jar to the list of resources.

2016-05-13T00:33:28,698 INFO [main] org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted application application_1461518911968_0018

2016-05-13T00:33:28,725 INFO [main] org.apache.hadoop.mapreduce.Job - The url to track the job:

2016-05-13T00:33:28,725 INFO [main] io.druid.indexer.DetermineHashedPartitionsJob - Job click-determine_partitions_hashed-Optional.of([2016-04-28T00:00:00.000Z/2016-04-28T01:00:00.000Z]) submitted, status available at: http://ip-.us-west-1.compute.internal:8088/proxy/application_1461518911968_0018/

2016-05-