Could not initialize class com.metamx.common.scala.Jackson

Hi,

I used spark in tranquility to send events ,but I got some errors:

16/01/07 19:53:51 WARN scheduler.TaskSetManager: Lost task 5.0 in stage 0.0 (TID 5, node007022): java.lang.NoClassDefFoundError: Could not initialize class com.metamx.common.scala.Jackson$

at com.metamx.tranquility.druid.DruidBeams$BuilderConfig$$anon$6.(DruidBeams.scala:260)

at com.metamx.tranquility.druid.DruidBeams$BuilderConfig.buildAll(DruidBeams.scala:259)

at com.metamx.tranquility.druid.DruidBeams$Builder.buildBeam(DruidBeams.scala:182)

at com.mafengwo.page.SimpleEventBeamFactory.makeBeam$lzycompute(process.scala:111)

at com.mafengwo.page.SimpleEventBeamFactory.makeBeam(process.scala:75)

at com.metamx.tranquility.spark.BeamRDD$$anonfun$propagate$1.apply(BeamRDD.scala:37)

at com.metamx.tranquility.spark.BeamRDD$$anonfun$propagate$1.apply(BeamRDD.scala:36)

at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:898)

at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:898)

at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1848)

at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1848)

at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)

at org.apache.spark.scheduler.Task.run(Task.scala:88)

at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

16/01/07 19:53:51 INFO scheduler.TaskSetManager: Lost task 37.0 in stage 0.0 (TID 37) on executor node007022: java.lang.NoClassDefFoundError (Could not initialize class com.metamx.common.scala.Jackson$) [duplicate 1]

16/01/07 19:53:51 WARN scheduler.TaskSetManager: Lost task 53.0 in stage 0.0 (TID 53, node007022): java.lang.NoSuchFieldError: WRITE_DURATIONS_AS_TIMESTAMPS

at com.fasterxml.jackson.datatype.joda.ser.DurationSerializer.(DurationSerializer.java:28)

at com.fasterxml.jackson.datatype.joda.ser.DurationSerializer.(DurationSerializer.java:25)

at com.fasterxml.jackson.datatype.joda.JodaModule.(JodaModule.java:45)

at com.metamx.common.scala.Jackson$$anonfun$newObjectMapper$1.apply(Jackson.scala:69)

at com.metamx.common.scala.Jackson$$anonfun$newObjectMapper$1.apply(Jackson.scala:68)

at com.metamx.common.scala.Predef$EffectOps.withEffect(Predef.scala:44)

at com.metamx.common.scala.Jackson$class.newObjectMapper(Jackson.scala:67)

at com.metamx.common.scala.Jackson$.newObjectMapper(Jackson.scala:10)

at com.metamx.common.scala.Jackson$class.newObjectMapper(Jackson.scala:64)

at com.metamx.common.scala.Jackson$.newObjectMapper(Jackson.scala:10)

at com.metamx.common.scala.Jackson$class.init(Jackson.scala:14)

at com.metamx.common.scala.Jackson$.(Jackson.scala:10)

at com.metamx.common.scala.Jackson$.(Jackson.scala)

at com.metamx.tranquility.druid.DruidBeams$BuilderConfig$$anon$6.(DruidBeams.scala:260)

at com.metamx.tranquility.druid.DruidBeams$BuilderConfig.buildAll(DruidBeams.scala:259)

at com.metamx.tranquility.druid.DruidBeams$Builder.buildBeam(DruidBeams.scala:182)

at com.mafengwo.page.SimpleEventBeamFactory.makeBeam$lzycompute(process.scala:111)

at com.mafengwo.page.SimpleEventBeamFactory.makeBeam(process.scala:75)

at com.metamx.tranquility.spark.BeamRDD$$anonfun$propagate$1.apply(BeamRDD.scala:37)

at com.metamx.tranquility.spark.BeamRDD$$anonfun$propagate$1.apply(BeamRDD.scala:36)

at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:898)

at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:898)

at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1848)

at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1848)

at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)

at org.apache.spark.scheduler.Task.run(Task.scala:88)

at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

my druid version is 0.8.2 and tranquility version is 0.6.4.what’s you advice here?Thanks!

This seems like a jackson version conflict between what tranquillity uses and what spark uses. What version of spark are you using? Do you know what jackson jars it is depending on?

You might find some of the pointers in this thread useful: https://groups.google.com/d/topic/druid-user/R7RZ30iq8Vw/discussion