Error running hadoop task after upgrade to 0.10

Hi all,

I upgraded from 0.9.1 to 0.10 recently and now I am unable to run a batch hadoop ingestion job that was working before. I am running CDH 5.7. It fails with:

2017-05-27 13:09:39,256 ERROR [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster

java.lang.UnsupportedClassVersionError: io/druid/indexer/DetermineHashedPartitionsJob$DetermineCardinalityMapper : Unsupported major.minor version 52.0

at java.lang.ClassLoader.defineClass1(Native Method)

at java.lang.ClassLoader.defineClass(ClassLoader.java:800)

at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)

at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)

at java.net.URLClassLoader.access$100(URLClassLoader.java:71)

at java.net.URLClassLoader$1.run(URLClassLoader.java:361)

at java.net.URLClassLoader$1.run(URLClassLoader.java:355)

at java.security.AccessController.doPrivileged(Native Method)

at java.net.URLClassLoader.findClass(URLClassLoader.java:354)

at java.lang.ClassLoader.loadClass(ClassLoader.java:425)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)

at java.lang.ClassLoader.loadClass(ClassLoader.java:358)

at java.lang.Class.forName0(Native Method)

at java.lang.Class.forName(Class.java:190)

at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.isChainJob(JobImpl.java:1308)

at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.makeUberDecision(JobImpl.java:1248)

at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.access$3800(JobImpl.java:140)

at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InitTransition.transition(JobImpl.java:1464)

at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InitTransition.transition(JobImpl.java:1401)

at org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)

at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)

at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)

at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)

at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:997)

at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:139)

at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:1333)

at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1101)

at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)

at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$4.run(MRAppMaster.java:1544)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:415)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)

at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1540)

at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1473)

2017-05-27 13:09:39,259 INFO [main] org.apache.hadoop.util.ExitUtil: Exiting with status 1

I’m not sure what other information would be helpful in debugging the issue.

Help would be greatly appreciated.

Thanks,

As another data point I have java 1.8 installed:

$ java -version

java version “1.8.0_101”

Java™ SE Runtime Environment (build 1.8.0_101-b13)

Java HotSpot™ 64-Bit Server VM (build 25.101-b13, mixed mode)

seems like workers in YARN were launched using java 7 you want to set it to java 8

you can fix it via

druid.indexer.fork.property.hadoop.mapreduce.reduce.env=JAVA_HOME=/home/whatever/java8/jdk64

druid.indexer.fork.property.hadoop.yarn.app.mapreduce.am.env=JAVA_HOME=/home/whatever/java8/jdk64

druid.indexer.fork.property.hadoop.mapreduce.map.env=….

I tried setting those settings correctly but it didn’t work. Even though I had java 8 CDH was still using 7. I updated the JAVA_HOME on all of the yarn hosts and now all is well.

Thanks for the help.

–Ben