Druid Datasketches not loading


I am trying to include data sketches extension for adding metric to perform distinct count

have added the following in common run time properties of all nodes except historical and restarted the nodes

druid.extensions.loadList=[“druid-datasketches”, “druid-lookups-cached-global”, “mysql-metadata-storage”]

but it seems to be missing when I query and throws the following error

Could not resolve type id ‘HLLSketchBuild’ into a subtype of [simple type, class io.druid.query.aggregation.AggregatorFactory]

Hi Shilpa,

You need to add DS extension list to all historical nodes. As historical nodes are responsible for data and they compute count distinct for DS.

If you are using post aggregations DS, you need to add the extension list to broker node also.

Hope this helps.

Regards, Chari.

Hi Chari,

Thanks, I am trying out on quickstart config where all nodes uses single common conf file. I have added the extension and restarted the nodes.

Also it fails at broker during query

[qtp449105069-61] io.druid.server.QueryResource - Exception occurred on request [unparsable query]

com.fasterxml.jackson.databind.JsonMappingException: Could not resolve type id ‘HLLSketchBuild’ into a subtype of [simple type, class io.druid.query.aggregation.AggregatorFactory]

at [Source: HttpInputOverHTTP@b3a907c[c=847,q=1,[0]=EOF,s=STREAM]; line: 2, column: 3] (through reference chain: java.util.ArrayList[1])

at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:148) ~[jackson-databind-2.4.6.jar:2.4.6]

at com.fasterxml.jackson.databind.DeserializationContext.unknownTypeException(DeserializationContext.java:862) ~[jackson-databind-2.4.6.jar:2.4.6]

at com.fasterxml.jackson.databind.jsontype.impl.TypeDeserializerBase._findDeserializer(TypeDeserializerBase.java:167) ~[jackson-databind-2.4.6.jar:2.4.6]

at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:99) ~[jackson-databind-2.4.6.jar:2.4.6]

at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:84) ~[jackson-databind-2.4.6.jar:2.4.6]

Hi Shilpa,

Could you verify your druid service logs if the “druid-datasketches” extension is loaded successfully or not ?

Eg : you can see entries in the druid service logs as below if the extension is loaded properly :

2019-09-24T12:55:47,306 INFO [main] org.apache.druid.initialization.Initialization - added URL[file:/home/ec2-user/apache-druid-0.15.0-incubating/extensions/druid-datasketches/druid-datasketches-0.15.0-incubating.jar] for extension[druid-datasketches]
2019-09-24T12:55:48,196 INFO [main] org.apache.druid.initialization.Initialization - Loading extension [druid-datasketches] for class [interface org.apache.druid.initialization.DruidModule]
2019-09-24T12:55:59,597 INFO [main] org.apache.druid.cli.CliBroker - * druid.extensions.loadList: [“druid-hdfs-storage”, “druid-kafka-indexing-service”, “druid-datasketches”]

If not, please verify if you have updated the correct config file :

For quickstart it should be :

Thanks and Regards,


Hi Vaibhav

I do see broker loads the extension