Druid 0.12.1 jar conflicts with amazon 2.7.2

stack trace :

java.lang.LinkageError: loader constraint violation: when resolving overridden method “org.apache.xerces.jaxp.DocumentBuilderImpl.newDocument()Lorg/w3c/dom/Document;” the class loader (instance of org/apache/hadoop/util/ApplicationClassLoader) of the current class, org/apache/xerces/jaxp/DocumentBuilderImpl, and its superclass loader (instance of ), have different Class objects for the type org/w3c/dom/Document used in the signature

at org.apache.xerces.jaxp.DocumentBuilderFactoryImpl.newDocumentBuilder(Unknown Source)

at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2530)

at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2492)

at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2405)

at org.apache.hadoop.conf.Configuration.get(Configuration.java:1232)

at org.apache.hadoop.yarn.factory.providers.RecordFactoryProvider.getRecordFactory(RecordFactoryProvider.java:49)

at org.apache.hadoop.mapreduce.TypeConverter.(TypeConverter.java:63)

at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.call(MRAppMaster.java:471)

at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.call(MRAppMaster.java:459)

at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.callWithJobClassLoader(MRAppMaster.java:1561)

at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.createOutputCommitter(MRAppMaster.java:459)

at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceInit(MRAppMaster.java:377)

at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)

at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$4.run(MRAppMaster.java:1519)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:422)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)

at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1516)

at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1449)

I am running 0.12.1 druid against amazon hadoop dist. 2.7.2 version.

Having tried mapreduce.job.classloader=true

druid.indexer.fork.property.hadoop.mapreduce.job.classloader=true

and everything (e.g replaced guice jar from 4 to 3 and reverted. But this doesnt seem to work.

Any clues ???

and with removing the classloader opt, I have get
java.lang.NoSuchMethodError: com.fasterxml.jackson.core.JsonGenerator.setCurrentValue

my jackson jars are as follows :

./hadoop-dependencies/hadoop-client/2.7.3/jackson-core-asl-1.9.13.jar

./hadoop-dependencies/hadoop-client/2.7.3/jackson-mapper-asl-1.9.13.jar

./hadoop-dependencies/hadoop-client/2.7.3/jackson-xc-1.9.13.jar

./hadoop-dependencies/hadoop-client/2.7.3/jackson-jaxrs-1.9.13.jar

./extensions/druid-avro-extensions/jackson-annotations-2.4.6.jar

./extensions/druid-avro-extensions/jackson-core-asl-1.9.13.jar

./extensions/druid-avro-extensions/jackson-databind-2.4.6.jar

./extensions/druid-avro-extensions/jackson-mapper-asl-1.9.13.jar

./extensions/druid-avro-extensions/jackson-core-2.4.6.jar

./extensions/druid-kerberos/jackson-jaxrs-1.9.2.jar

./extensions/druid-kerberos/jackson-xc-1.9.2.jar

./extensions/druid-parquet-extensions/parquet-jackson-1.8.2.jar

./extensions/druid-parquet-extensions/jackson-core-asl-1.9.11.jar

./extensions/druid-parquet-extensions/jackson-mapper-asl-1.9.11.jar

./extensions/druid-hdfs-storage/jackson-dataformat-cbor-2.5.3.jar

./extensions/druid-hdfs-storage/jackson-annotations-2.4.6.jar

./extensions/druid-hdfs-storage/jackson-databind-2.4.6.jar

./extensions/druid-hdfs-storage/jackson-core-2.4.6.jar

./extensions/druid-hdfs-storage/jackson-xc-1.9.13.jar

./extensions/druid-hdfs-storage/jackson-jaxrs-1.9.13.jar

./lib/jackson-dataformat-cbor-2.5.3.jar

./lib/jackson-annotations-2.4.6.jar

./lib/jackson-dataformat-smile-2.4.6.jar

./lib/jackson-datatype-joda-2.4.6.jar

./lib/jackson-core-asl-1.9.13.jar

./lib/jackson-databind-2.4.6.jar

./lib/jackson-datatype-guava-2.4.6.jar

./lib/jackson-mapper-asl-1.9.13.jar

./lib/jackson-core-2.4.6.jar

./lib/jackson-jaxrs-smile-provider-2.4.6.jar

./lib/jackson-jaxrs-json-provider-2.4.6.jar

./lib/jackson-jq-0.0.7.jar

./lib/jackson-module-jaxb-annotations-2.4.6.jar

./lib/jackson-jaxrs-base-2.4.6.jar

Hi Sumit,

what is amazon hadoop distribution?

If you mean Apache Hadoop, the below should work.

  1. Don’t touch anything in {DRUID_HOME}/lib.

  2. Use hadoop-client 2.7.2. You can download necessary jars using pull-deps (http://druid.io/docs/latest/operations/pull-deps.html).

  3. Add “mapreduce.job.classloader”: “true” to the jobProperties of tuningConfig or set druid.indexer.runner.javaOpts=… -Dhadoop.mapreduce.job.classloader=true on the middleManagers.

Best,

Jihoon

Thanks Jihoon,
this is exactly what that worked, just that

  1. Use hadoop-client 2.7.2. You can download necessary jars using pull-deps (http://druid.io/docs/latest/operations/pull-deps.html).

needed to be uncommented and specified as hadoopDirectory path in conf/druid/_common/common.runtime.properties

Thanks