Trying to get indexing via remote hadoop running

Hi,

We’re running hadoop on Google Compute Engine using their storage. I’ve managed to get it pushing the job to hadoop, and the map phase seems to work fine… but now getting this error which I am unable to decipher (at the bottom.)

I’ve upgraded to druid 0.8.0 but that did not help at all.

Thanks in advance,

Josh

2015-08-20T21:25:24,058 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job -  map 100% reduce 12%
2015-08-20T21:25:27,075 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job -  map 100% reduce 22%
2015-08-20T21:25:27,079 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job - Task Id : attempt_1439499944815_0009_r_000000_0, Status : FAILED
Error: com.google.common.primitives.Floats.tryParse(Ljava/lang/String;)Ljava/lang/Float;
2015-08-20T21:25:28,097 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job -  map 100% reduce 33%
2015-08-20T21:25:28,099 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job - Task Id : attempt_1439499944815_0009_r_000001_0, Status : FAILED
Error: com.google.common.primitives.Floats.tryParse(Ljava/lang/String;)Ljava/lang/Float;
2015-08-20T21:25:29,105 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job -  map 100% reduce 0%
2015-08-20T21:25:30,113 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job - Task Id : attempt_1439499944815_0009_r_000002_0, Status : FAILED
Error: com.google.common.primitives.Floats.tryParse(Ljava/lang/String;)Ljava/lang/Float;
2015-08-20T21:25:37,157 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job - Task Id : attempt_1439499944815_0009_r_000001_1, Status : FAILED
Error: com.google.common.primitives.Floats.tryParse(Ljava/lang/String;)Ljava/lang/Float;
2015-08-20T21:25:39,170 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job - Task Id : attempt_1439499944815_0009_r_000002_1, Status : FAILED
Error: com.google.common.primitives.Floats.tryParse(Ljava/lang/String;)Ljava/lang/Float;
2015-08-20T21:25:39,172 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job - Task Id : attempt_1439499944815_0009_r_000000_1, Status : FAILED
Error: com.google.common.primitives.Floats.tryParse(Ljava/lang/String;)Ljava/lang/Float;
2015-08-20T21:25:47,220 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job - Task Id : attempt_1439499944815_0009_r_000001_2, Status : FAILED
Error: com.google.common.primitives.Floats.tryParse(Ljava/lang/String;)Ljava/lang/Float;
2015-08-20T21:25:50,243 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job - Task Id : attempt_1439499944815_0009_r_000000_2, Status : FAILED
Error: com.google.common.primitives.Floats.tryParse(Ljava/lang/String;)Ljava/lang/Float;
2015-08-20T21:25:51,265 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job - Task Id : attempt_1439499944815_0009_r_000002_2, Status : FAILED
Error: com.google.common.primitives.Floats.tryParse(Ljava/lang/String;)Ljava/lang/Float;
2015-08-20T21:25:57,304 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job -  map 100% reduce 100%
2015-08-20T21:26:04,352 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job - Job job_1439499944815_0009 failed with state FAILED due to: Task failed task_1439499944815_0009_r_000001

Job failed as tasks failed. failedMaps:0 failedReduces:1

2015-08-20T21:26:04,433 INFO [task-runner-0] io.druid.indexer.JobHelper - Deleting path[/tmp/druid-indexing/testaaz/2015-08-20T212151.051Z]
2015-08-20T21:26:04,775 ERROR [task-runner-0] io.druid.indexing.overlord.ThreadPoolTaskRunner - Exception while running task[HadoopIndexTask{id=index_hadoop_testaaz_2015-08-20T21:21:51.033Z, type=index_hadoop, dataSource=testaaz}]
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
	at com.google.api.client.repackaged.com.google.common.base.Throwables.propagate(Throwables.java:160) ~[gcs-connector-latest-hadoop2.jar:?]
	at io.druid.indexing.common.task.HadoopTask.invokeForeignLoader(HadoopTask.java:132) ~[druid-indexing-service-0.8.0.jar:0.8.0]
	at io.druid.indexing.common.task.HadoopIndexTask.run(HadoopIndexTask.java:188) ~[druid-indexing-service-0.8.0.jar:0.8.0]
	at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:235) [druid-indexing-service-0.8.0.jar:0.8.0]
	at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:214) [druid-indexing-service-0.8.0.jar:0.8.0]
	at java.util.concurrent.FutureTask.run(FutureTask.java:262) [?:1.7.0_79]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [?:1.7.0_79]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [?:1.7.0_79]
	at java.lang.Thread.run(Thread.java:745) [?:1.7.0_79]
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.7.0_79]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[?:1.7.0_79]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.7.0_79]
	at java.lang.reflect.Method.invoke(Method.java:606) ~[?:1.7.0_79]
	at io.druid.indexing.common.task.HadoopTask.invokeForeignLoader(HadoopTask.java:129) ~[druid-indexing-service-0.8.0.jar:0.8.0]
	... 7 more
Caused by: com.metamx.common.ISE: Job[class io.druid.indexer.LegacyIndexGeneratorJob] failed!
	at io.druid.indexer.JobHelper.runJobs(JobHelper.java:199) ~[druid-indexing-hadoop-0.8.0.jar:0.8.0]
	at io.druid.indexer.HadoopDruidIndexerJob.run(HadoopDruidIndexerJob.java:96) ~[druid-indexing-hadoop-0.8.0.jar:0.8.0]
	at io.druid.indexing.common.task.HadoopIndexTask$HadoopIndexGeneratorInnerProcessing.runTask(HadoopIndexTask.java:241) ~[druid-indexing-service-0.8.0.jar:0.8.0]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.7.0_79]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[?:1.7.0_79]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.7.0_79]
	at java.lang.reflect.Method.invoke(Method.java:606) ~[?:1.7.0_79]
	at io.druid.indexing.common.task.HadoopTask.invokeForeignLoader(HadoopTask.java:129) ~[druid-indexing-service-0.8.0.jar:0.8.0]
	... 7 more
2015-08-20T21:26:04,785 INFO [task-runner-0] io.druid.indexing.worker.executor.ExecutorLifecycle - Task completed with status: {
  "id" : "index_hadoop_testaaz_2015-08-20T21:21:51.033Z",
  "status" : "FAILED",
  "duration" : 248293
}
2015-08-20T21:26:04,787 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void io.druid.server.coordination.AbstractDataSegmentAnnouncer.stop()] on object[io.druid.server.coordination.BatchDataSegmentAnnouncer@1fbf7a1f].

For that one you’ll have to go to the actual hadoop logs to see why the job failed.

Job failed as tasks failed. failedMaps:0 failedReduces:1

``

check the reduces