Hi Team,
We are using Custom Parsers one which is exactly similar to HadoopyStringInputRowParser.java, we have additional custom code to do extra transformation(like replacing Null values with expected default values) other than that it looks exactly similar. When we submit the task to it shows it’s using the customer parser the constructor gets called. However, the parse method is not getting invoked and it’s throwing exception at later point in time to aggregated Null values. Am i missing something?
It throws the below exception…
Root Cause:
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:91)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1936)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
Caused by: com.metamx.common.parsers.ParseException: Encountered parse error for aggregator[bid_modifier]
at io.druid.indexer.InputRowSerde.toBytes(InputRowSerde.java:102)
at io.druid.indexer.IndexGeneratorJob$IndexGeneratorMapper.innerMap(IndexGeneratorJob.java:300)
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:87)
... 8 more
Caused by: com.metamx.common.parsers.ParseException: Unable to parse metrics[bid_modifier], value[\N]
at io.druid.data.input.MapBasedRow.getFloatMetric(MapBasedRow.java:130)
at io.druid.segment.incremental.IncrementalIndex$1$3.get(IncrementalIndex.java:144)
at io.druid.query.aggregation.DoubleSumAggregator.aggregate(DoubleSumAggregator.java:62)
at io.druid.indexer.InputRowSerde.toBytes(InputRowSerde.java:97)
... 10 more
Caused by: java.lang.NumberFormatException: For input string: "\N"
at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:2043)
at sun.misc.FloatingDecimal.parseFloat(FloatingDecimal.java:122)
at java.lang.Float.parseFloat(Float.java:451)
at java.lang.Float.valueOf(Float.java:416)
at io.druid.data.input.MapBasedRow.getFloatMetric(MapBasedRow.java:127)
... 13 more
Which results in the below exception.
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at com.google.common.base.Throwables.propagate(Throwables.java:160) ~[guava-16.0.1.jar:?]
at io.druid.indexing.common.task.HadoopTask.invokeForeignLoader(HadoopTask.java:204) ~[druid-indexing-service-0.9.2-1493303904-8e35147-366.jar:0.9.2-1493303904-8e35147-366]
at io.druid.indexing.common.task.HadoopIndexTask.run(HadoopIndexTask.java:208) ~[druid-indexing-service-0.9.2-1493303904-8e35147-366.jar:0.9.2-1493303904-8e35147-366]
at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:436) [druid-indexing-service-0.9.2-1493303904-8e35147-366.jar:0.9.2-1493303904-8e35147-366]
at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:408) [druid-indexing-service-0.9.2-1493303904-8e35147-366.jar:0.9.2-1493303904-8e35147-366]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_131]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_131]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_131]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_131]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_131]
at io.druid.indexing.common.task.HadoopTask.invokeForeignLoader(HadoopTask.java:201) ~[druid-indexing-service-0.9.2-1493303904-8e35147-366.jar:0.9.2-1493303904-8e35147-366]
... 7 more
Caused by: com.metamx.common.ISE: Job[class io.druid.indexer.IndexGeneratorJob] failed!
at io.druid.indexer.JobHelper.runJobs(JobHelper.java:369) ~[druid-indexing-hadoop-0.9.2-1493303904-8e35147-366.jar:0.9.2-1493303904-8e35147-366]
at io.druid.indexer.HadoopDruidIndexerJob.run(HadoopDruidIndexerJob.java:94) ~[druid-indexing-hadoop-0.9.2-1493303904-8e35147-366.jar:0.9.2-1493303904-8e35147-366]
at io.druid.indexing.common.task.HadoopIndexTask$HadoopIndexGeneratorInnerProcessing.runTask(HadoopIndexTask.java:261) ~[druid-indexing-service-0.9.2-1493303904-8e35147-366.jar:0.9.2-1493303904-8e35147-366]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_131]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_131]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_131]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_131]
at io.druid.indexing.common.task.HadoopTask.invokeForeignLoader(HadoopTask.java:201) ~[druid-indexing-service-0.9.2-1493303904-8e35147-366.jar:0.9.2-1493303904-8e35147-366]
... 7 more