Error with tranquility extensions (Possible Bug)

While adding druid extensions to tranquility (specifically the druid-stats extensions) the service fails to start and provides a the following stack trace when trying to load the druid-stats extension. This ONLY happens when my ingestion spec includes the variance however and the service will run fine when it is not there.

java.lang.NoClassDefFoundError: io/druid/java/util/common/IAE

at java.lang.Class.getDeclaredFields0(Native Method)

at java.lang.Class.privateGetDeclaredFields(Class.java:2583)

at java.lang.Class.getDeclaredFields(Class.java:1916)

at com.fasterxml.jackson.databind.introspect.AnnotatedClass._findFields(AnnotatedClass.java:689)

at com.fasterxml.jackson.databind.introspect.AnnotatedClass.resolveFields(AnnotatedClass.java:470)

at com.fasterxml.jackson.databind.introspect.AnnotatedClass.fields(AnnotatedClass.java:282)

at com.fasterxml.jackson.databind.introspect.POJOPropertiesCollector._addFields(POJOPropertiesCollector.java:384)

at com.fasterxml.jackson.databind.introspect.POJOPropertiesCollector.collect(POJOPropertiesCollector.java:237)

at com.fasterxml.jackson.databind.introspect.BasicClassIntrospector.collectProperties(BasicClassIntrospector.java:142)

at com.fasterxml.jackson.databind.introspect.BasicClassIntrospector.forDeserialization(BasicClassIntrospector.java:81)

at com.fasterxml.jackson.databind.introspect.BasicClassIntrospector.forDeserialization(BasicClassIntrospector.java:11)

at com.fasterxml.jackson.databind.DeserializationConfig.introspect(DeserializationConfig.java:550)

at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer(DeserializerCache.java:327)

at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:264)

at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244)

at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142)

at com.fasterxml.jackson.databind.DeserializationContext.findContextualValueDeserializer(DeserializationContext.java:380)

at com.fasterxml.jackson.databind.jsontype.impl.TypeDeserializerBase._findDeserializer(TypeDeserializerBase.java:181)

at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:99)

at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:84)

at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:132)

at com.fasterxml.jackson.databind.deser.std.ObjectArrayDeserializer.deserialize(ObjectArrayDeserializer.java:151)

at com.fasterxml.jackson.databind.deser.std.ObjectArrayDeserializer.deserialize(ObjectArrayDeserializer.java:17)

at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:538)

at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:344)

at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1064)

at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:264)

at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:124)

at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:538)

at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:344)

at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1064)

at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:264)

at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:124)

at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:2769)

at com.fasterxml.jackson.databind.ObjectMapper.convertValue(ObjectMapper.java:2700)

at com.metamx.tranquility.druid.DruidBeams$.makeFireDepartment(DruidBeams.scala:433)

at com.metamx.tranquility.druid.DruidBeams$.fromConfigInternal(DruidBeams.scala:299)

at com.metamx.tranquility.druid.DruidBeams$.fromConfig(DruidBeams.scala:204)

at com.metamx.tranquility.server.http.ServerMain$$anonfun$2.apply(ServerMain.scala:118)

at com.metamx.tranquility.server.http.ServerMain$$anonfun$2.apply(ServerMain.scala:98)

at com.metamx.common.scala.collection.package$MapLikeOps$$anonfun$strictMapValues$1.apply(package.scala:143)

at com.metamx.common.scala.collection.package$MapLikeOps$$anonfun$strictMapValues$1.apply(package.scala:143)

at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245)

at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245)

at scala.collection.immutable.Map$Map2.foreach(Map.scala:137)

at scala.collection.TraversableLike$class.map(TraversableLike.scala:245)

at scala.collection.AbstractTraversable.map(Traversable.scala:104)

at com.metamx.common.scala.collection.package$MapLikeOps.strictMapValues(package.scala:143)

at com.metamx.tranquility.server.http.ServerMain$.createServlet(ServerMain.scala:98)

at com.metamx.tranquility.server.http.ServerMain$.main(ServerMain.scala:77)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at com.twitter.app.App$$anonfun$nonExitingMain$3.apply(App.scala:168)

at com.twitter.app.App$$anonfun$nonExitingMain$3.apply(App.scala:167)

at scala.Option.foreach(Option.scala:257)

at com.twitter.app.App$class.nonExitingMain(App.scala:167)

at com.metamx.tranquility.server.http.ServerMain$.nonExitingMain(ServerMain.scala:49)

at com.twitter.app.App$class.main(App.scala:133)

at com.metamx.tranquility.server.http.ServerMain$.main(ServerMain.scala:49)

at com.metamx.tranquility.distribution.DistributionMain$.main(DistributionMain.scala:34)

at com.metamx.tranquility.distribution.DistributionMain.main(DistributionMain.scala)

Caused by: java.lang.ClassNotFoundException: io.druid.java.util.common.IAE

at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

… 63 more

Furthermore, I understand that this error is associated with the java-util library so I tried to manually include that which resulted in the following stack.

2018-02-12 22:56:39,861 [main] INFO i.d.initialization.Initialization - Loading extension [java-util-0.10.1.jar] for class [io.druid.initialization.DruidModule]

java.lang.ExceptionInInitializerError

at com.metamx.tranquility.druid.DruidBeams$.makeFireDepartment(DruidBeams.scala:433)

at com.metamx.tranquility.druid.DruidBeams$.fromConfigInternal(DruidBeams.scala:299)

at com.metamx.tranquility.druid.DruidBeams$.fromConfig(DruidBeams.scala:204)

at com.metamx.tranquility.server.http.ServerMain$$anonfun$2.apply(ServerMain.scala:118)

at com.metamx.tranquility.server.http.ServerMain$$anonfun$2.apply(ServerMain.scala:98)

at com.metamx.common.scala.collection.package$MapLikeOps$$anonfun$strictMapValues$1.apply(package.scala:143)

at com.metamx.common.scala.collection.package$MapLikeOps$$anonfun$strictMapValues$1.apply(package.scala:143)

at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245)

at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245)

at scala.collection.immutable.Map$Map2.foreach(Map.scala:137)

at scala.collection.TraversableLike$class.map(TraversableLike.scala:245)

at scala.collection.AbstractTraversable.map(Traversable.scala:104)

at com.metamx.common.scala.collection.package$MapLikeOps.strictMapValues(package.scala:143)

at com.metamx.tranquility.server.http.ServerMain$.createServlet(ServerMain.scala:98)

at com.metamx.tranquility.server.http.ServerMain$.main(ServerMain.scala:77)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at com.twitter.app.App$$anonfun$nonExitingMain$3.apply(App.scala:168)

at com.twitter.app.App$$anonfun$nonExitingMain$3.apply(App.scala:167)

at scala.Option.foreach(Option.scala:257)

at com.twitter.app.App$class.nonExitingMain(App.scala:167)

at com.metamx.tranquility.server.http.ServerMain$.nonExitingMain(ServerMain.scala:49)

at com.twitter.app.App$class.main(App.scala:133)

at com.metamx.tranquility.server.http.ServerMain$.main(ServerMain.scala:49)

at com.metamx.tranquility.distribution.DistributionMain$.main(DistributionMain.scala:34)

at com.metamx.tranquility.distribution.DistributionMain.main(DistributionMain.scala)

Caused by: java.lang.IllegalArgumentException: Parameter ‘directory’ is not a directory

at org.apache.commons.io.FileUtils.validateListFilesParameters(FileUtils.java:545)

at org.apache.commons.io.FileUtils.listFiles(FileUtils.java:521)

at org.apache.commons.io.FileUtils.listFiles(FileUtils.java:691)

at io.druid.initialization.Initialization.getClassLoaderForExtension(Initialization.java:269)

at io.druid.initialization.Initialization.getFromExtensions(Initialization.java:156)

at com.metamx.tranquility.druid.DruidGuicer.(DruidGuicer.scala:124)

at com.metamx.tranquility.druid.DruidGuicer$.(DruidGuicer.scala:138)

at com.metamx.tranquility.druid.DruidGuicer$.(DruidGuicer.scala)

… 28 more

Exception thrown in main on startup

I feel like I am doing something wrong but i have not been able to find anything helpful on the forums or docs after looking for hours.

Tranquility v0.8.2

Druid v0.10.1

Any help is much appreciated.

Hi Josh,

I bet this is because Tranquility is built against Druid 0.9.2 but the extension you are loading into it is built against Druid 0.10.1. There have been some efforts to update Tranquility to the latest Druid (https://github.com/druid-io/tranquility/pull/233) but they have not borne fruit yet. I think it’s mostly because other than loading extensions, the older Tranquility works fine against the latest Druid, so nobody has been motivated enough to finish it yet. Maybe you will be the one :slight_smile:

Thanks for the update and reference! Knowing this I took the extension from Druid 0.9.2 and loaded them into Tranquility and it seems to work fine. Do you see any issues that could come from doing this? From what I see the variance and Standard Deviation Support has not been modified in about 2 years.

Thanks,

Josh

Hi Josh,

It should work fine. In reality the only thing that has to match up between Tranquility and Druid versions of an extension is the JSON format for wire transfer, and that generally doesn’t change much.