Mismatch between tranquility and druid in specifying approximate histogram aggregtion type

If I specify an ingestion time aggregation as

{

“type”: “approximateHistogram”,

“name”: “loadtime”,

“fieldName”: “loadtime”

}

Tranquility errors out. If I change “approximateHistogram” to “histogram” tranquility doesn’t complain, but it causes an error in Druid

These are the versions of tranquility and druid

compile group: ‘io.druid’, name: ‘druid-api’, version: ‘0.9.2’

compile group: ‘io.druid’, name: ‘tranquility-core_2.11’, version: ‘0.8.2’

Any ideas ?

Thanks,

Radhakrishnan

The aggregator types are “approxHistogramFold” and “approxHistogram”, not “approximateHistogram”. For ingestion you should use “approxHistogramFold”.

If that doesn’t help, you also might need to load the extension into Tranquility, see: https://github.com/druid-io/tranquility/blob/master/docs/configuration.md#loading-druid-extensions

This

{

“type”: “approxHistogram”,

“name”: “loadtime”,

“fieldName”: “loadtime”

},

{

“type”: “approxHistogram”,

“name”: “connecttime”,

“fieldName”: “connecttime”

}

Leads to

Exception in thread “main” java.lang.IllegalArgumentException: Could not resolve type id ‘approxHistogram’ into a subtype of [simple type, class io.druid.query.aggregation.AggregatorFactory]: known type ids = [AggregatorFactory, cardinality, count, doubleMax, doubleMin, doubleSum, filtered, histogram, hyperUnique, javascript, longMax, longMin, longSum]

at [Source: N/A; line: -1, column: -1] (through reference chain: io.druid.segment.realtime.FireDepartment[“dataSchema”]->io.druid.segment.indexing.DataSchema[“metricsSpec”]->Object[1])

at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:3425)

at com.fasterxml.jackson.databind.ObjectMapper.convertValue(ObjectMapper.java:3351)

at com.metamx.tranquility.druid.DruidBeams$.makeFireDepartment(DruidBeams.scala:433)

at com.metamx.tranquility.druid.DruidBeams$.fromConfigInternal(DruidBeams.scala:299)

at com.metamx.tranquility.druid.DruidBeams$.fromConfig(DruidBeams.scala:204)

at com.metamx.tranquility.druid.DruidBeams$.fromConfig(DruidBeams.scala:123)

at com.metamx.tranquility.druid.DruidBeams.fromConfig(DruidBeams.scala)

at com.appdynamics.eum.resourcetiming.SamplingBasedDataIngester.main(SamplingBasedDataIngester.java:27)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)

Caused by: com.fasterxml.jackson.databind.JsonMappingException: Could not resolve type id ‘approxHistogram’ into a subtype of [simple type, class io.druid.query.aggregation.AggregatorFactory]: known type ids = [AggregatorFactory, cardinality, count, doubleMax, doubleMin, doubleSum, filtered, histogram, hyperUnique, javascript, longMax, longMin, longSum]

at [Source: N/A; line: -1, column: -1] (through reference chain: io.druid.segment.realtime.FireDepartment[“dataSchema”]->io.druid.segment.indexing.DataSchema[“metricsSpec”]->Object[1])

at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:148)

at com.fasterxml.jackson.databind.DeserializationContext.unknownTypeException(DeserializationContext.java:964)

at com.fasterxml.jackson.databind.jsontype.impl.TypeDeserializerBase._handleUnknownTypeId(TypeDeserializerBase.java:271)

at com.fasterxml.jackson.databind.jsontype.impl.TypeDeserializerBase._findDeserializer(TypeDeserializerBase.java:159)

at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:108)

at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:93)

at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:131)

at com.fasterxml.jackson.databind.deser.std.ObjectArrayDeserializer.deserialize(ObjectArrayDeserializer.java:158)

at com.fasterxml.jackson.databind.deser.std.ObjectArrayDeserializer.deserialize(ObjectArrayDeserializer.java:17)

at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:520)

at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:461)

at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:377)

at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1100)

at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:294)

at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:131)

at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:520)

at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:461)

at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:377)

at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1100)

at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:294)

at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:131)

at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:3420)

… 12 more

Specifying the following JVM properties

-Ddruid.extensions.directory=/Users/jrkrishnan/druid-0.9.2/extensions -Ddruid.extensions.loadList=’[“druid-kafka-eight”,“druid-s3-extensions”,“druid-histogram”,“druid-datasketches”,“druid-lookups-cached-global”,“mysql-metadata-storage”]’

Leads to

Exception in thread “main” java.lang.ExceptionInInitializerError

at com.metamx.tranquility.druid.DruidBeams$.makeFireDepartment(DruidBeams.scala:433)

at com.metamx.tranquility.druid.DruidBeams$.fromConfigInternal(DruidBeams.scala:299)

at com.metamx.tranquility.druid.DruidBeams$.fromConfig(DruidBeams.scala:204)

at com.metamx.tranquility.druid.DruidBeams$.fromConfig(DruidBeams.scala:123)

at com.metamx.tranquility.druid.DruidBeams.fromConfig(DruidBeams.scala)

at com.appdynamics.eum.resourcetiming.SamplingBasedDataIngester.main(SamplingBasedDataIngester.java:27)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)

Caused by: com.google.inject.ProvisionException: Unable to provision, see the following errors:

  1. Problem parsing object at prefix[druid.extensions]: Can not deserialize instance of java.util.ArrayList out of VALUE_STRING token

at [Source: N/A; line: -1, column: -1] (through reference chain: io.druid.guice.ExtensionsConfig[“loadList”]).

at io.druid.guice.JsonConfigProvider.bind(JsonConfigProvider.java:131)

at io.druid.guice.JsonConfigProvider.bind(JsonConfigProvider.java:131)

while locating com.google.common.base.Supplier<io.druid.guice.ExtensionsConfig>

at io.druid.guice.JsonConfigProvider.bind(JsonConfigProvider.java:132)

while locating io.druid.guice.ExtensionsConfig

1 error

at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1028)

at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1054)

at com.metamx.tranquility.druid.DruidGuicer.(DruidGuicer.scala:101)

at com.metamx.tranquility.druid.DruidGuicer$.(DruidGuicer.scala:138)

at com.metamx.tranquility.druid.DruidGuicer$.(DruidGuicer.scala)

… 11 more

Caused by: java.lang.IllegalArgumentException: Can not deserialize instance of java.util.ArrayList out of VALUE_STRING token

at [Source: N/A; line: -1, column: -1] (through reference chain: io.druid.guice.ExtensionsConfig[“loadList”])

at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:3425)

at com.fasterxml.jackson.databind.ObjectMapper.convertValue(ObjectMapper.java:3351)

at io.druid.guice.JsonConfigurator.configurate(JsonConfigurator.java:100)

at io.druid.guice.JsonConfigProvider.get(JsonConfigProvider.java:200)

at io.druid.guice.JsonConfigProvider.get(JsonConfigProvider.java:79)

at com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81)

at com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:53)

at com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:61)

at com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:45)

at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)

at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092)

at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)

at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:194)

at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41)

at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1019)

at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092)

at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1015)

at io.druid.guice.SupplierProvider.get(SupplierProvider.java:52)

at com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81)

at com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:53)

at com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:61)

at com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:45)

at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1019)

at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1085)

at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1015)

… 15 more

Caused by: com.fasterxml.jackson.databind.JsonMappingException: Can not deserialize instance of java.util.ArrayList out of VALUE_STRING token

at [Source: N/A; line: -1, column: -1] (through reference chain: io.druid.guice.ExtensionsConfig[“loadList”])

at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:148)

at com.fasterxml.jackson.databind.DeserializationContext.mappingException(DeserializationContext.java:854)

at com.fasterxml.jackson.databind.DeserializationContext.mappingException(DeserializationContext.java:850)

at com.fasterxml.jackson.databind.deser.std.StringCollectionDeserializer.handleNonArray(StringCollectionDeserializer.java:240)

at com.fasterxml.jackson.databind.deser.std.StringCollectionDeserializer.deserialize(StringCollectionDeserializer.java:171)

at com.fasterxml.jackson.databind.deser.std.StringCollectionDeserializer.deserialize(StringCollectionDeserializer.java:161)

at com.fasterxml.jackson.databind.deser.std.StringCollectionDeserializer.deserialize(StringCollectionDeserializer.java:19)

at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:520)

at com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:101)

at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:256)

at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:125)

at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:3420)

… 39 more

Hi, I meet the same error, do you have any solution?

在 2017年2月7日星期二 UTC+8下午5:24:07,Radhakrishnan Jankiraman写道:

I got same error with the below config:
{

      "type": "approxHistogram",

      "name": "histogram_col",			 

      "fieldName": "imps",

      "resolution": 10000,

      "lowerLimit": 1

}

{“error”:“Could not resolve type id ‘approxHistogram’ into a subtype of [simple type, class io.druid.query.aggregation.AggregatorFactory]\n at [Source: HttpInputOverHTTP@4e2203c1[c=3182,q=1,[0]=EOF,s=STREAM]; line: 1, column: 2140] (through reference chain: Object[5])”

Did you guys found any solution? Im also facing same issue.