NPE in batch ingestion job

Apache Druid Q&A Google Groups
#1

Hi,

I am new to druid. I am trying to do a batch ingestion job but getting the following NPE

2017-11-28 21:12:57,507 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : io.druid.java.util.common.RE: Failure on row[{"visitor_id": "*****", "moniker_site": "**", "device_type": "***", "browser_name": "***"}]
	at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:91)
	at io.druid.indexer.DetermineHashedPartitionsJob$DetermineCardinalityMapper.run(DetermineHashedPartitionsJob.java:285)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.lang.NullPointerException
	at io.druid.data.input.MapBasedRow.getTimestampFromEpoch(MapBasedRow.java:76)
	at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:85)



I am not sure what is wrong.. Attaching my json file https://pastebin.com/0NDzVZs2



Any help is appreciated



Thanks,

Sid
#2

Your row is missing dateid?

#3

I figured out the issue. It was an issue with dateid parsing. Druid was unable to parse the column dateid.

#4

Hi Siddharth,

Can you please elaborate the fix you gave for the above data parsing issue? I am getting the same problem while ingesting data to druid.

Thanks in Advance.

Srinivas Katta

#5

this seems a miss configuration issue.

You need to make sure that the map reduce job is running with -Duser.timezone=UTC

you can set that using druid.indexer.runner.javaOpts or also you can use

druid.indexer.fork.property.hadoop.mapreduce.reduce.java.opts=-Duser.timezone=UTC -Dfile.encoding=UTF-8