Error ingesting orc data

Hi All,

we are trying to ingest orc data using druid .12. i have been running into issues with following error. Can you please let me know what i am missing? Attached is my spec file and below is the error lines from the log. let me know if you need anymore details.

java.lang.RuntimeException: java.lang.reflect.InvocationTargetException

at com.google.common.base.Throwables.propagate(Throwables.java:160) ~[guava-16.0.1.jar:?]

at io.druid.indexing.common.task.HadoopTask.invokeForeignLoader(HadoopTask.java:222) ~[druid-indexing-service-0.12.1-iap3.jar:0.12.1-iap3]

at io.druid.indexing.common.task.HadoopIndexTask.run(HadoopIndexTask.java:184) ~[druid-indexing-service-0.12.1-iap3.jar:0.12.1-iap3]

at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:450) [druid-indexing-service-0.12.1-iap3.jar:0.12.1-iap3]

at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:422) [druid-indexing-service-0.12.1-iap3.jar:0.12.1-iap3]

at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_73]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_73]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_73]

at java.lang.Thread.run(Thread.java:745) [?:1.8.0_73]

Caused by: java.lang.reflect.InvocationTargetException

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_73]

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_73]

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_73]

at java.lang.reflect.Method.invoke(Method.java:497) ~[?:1.8.0_73]

at io.druid.indexing.common.task.HadoopTask.invokeForeignLoader(HadoopTask.java:219) ~[druid-indexing-service-0.12.1-iap3.jar:0.12.1-iap3]

… 7 more

Caused by: java.lang.NoSuchFieldError: HIVE_ORC_CACHE_STRIPE_DETAILS_MEMORY_SIZE

at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$Context.(OrcInputFormat.java:582) ~[?:?]

at org.apache.hadoop.hive.ql.io.orc.OrcNewInputFormat.createContext(OrcNewInputFormat.java:145) ~[?:?]

at org.apache.hadoop.hive.ql.io.orc.OrcNewInputFormat.getSplits(OrcNewInputFormat.java:127) ~[?:?]

at org.apache.hadoop.mapreduce.lib.input.DelegatingInputFormat.getSplits(DelegatingInputFormat.java:115) ~[?:?]

at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:301) ~[?:?]

at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:318) ~[?:?]

at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:196) ~[?:?]

at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290) ~[?:?]

at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287) ~[?:?]

at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_73]

at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_73]

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869) ~[?:?]

at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287) ~[?:?]

at io.druid.indexer.DetermineHashedPartitionsJob.run(DetermineHashedPartitionsJob.java:119) ~[druid-indexing-hadoop-0.12.1-iap3.jar:0.12.1-iap3]

at io.druid.indexer.JobHelper.runJobs(JobHelper.java:371) ~[druid-indexing-hadoop-0.12.1-iap3.jar:0.12.1-iap3]

at io.druid.indexer.HadoopDruidDetermineConfigurationJob.run(HadoopDruidDetermineConfigurationJob.java:91) ~[druid-indexing-hadoop-0.12.1-iap3.jar:0.12.1-iap3]

at io.druid.indexing.common.task.HadoopIndexTask$HadoopDetermineConfigInnerProcessing.runTask(HadoopIndexTask.java:325) ~[druid-indexing-service-0.12.1-iap3.jar:0.12.1-iap3]

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_73]

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_73]

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_73]

at java.lang.reflect.Method.invoke(Method.java:497) ~[?:1.8.0_73]

at io.druid.indexing.common.task.HadoopTask.invokeForeignLoader(HadoopTask.java:219) ~[druid-indexing-service-0.12.1-iap3.jar:0.12.1-iap3]

… 7 more

Thanks,

Karteek

ingest.json (2.13 KB)

Hi Karteek:

Looks like a Hive config missing error. Do you have this “hive.orc.cache.stripe.details.mem.size” set somewhere, and is in Druid’s class path?

Thanks