UOE: CompressedHistogramAggregatorFactory does not implement makeAggregateCombiner()

TROUBLESHOOTING Google Groups
#1

I am running a hadoop ingestion task which succeds in druid 0.12.1 but fails in druid 0.15.1 with the same ingestion spec and data with the following error:

2020-04-13T19:30:15,674 INFO [task-runner-0-priority-0] org.apache.hadoop.mapreduce.Job - Task Id : attempt_1586805081376_0002_r_000002_1, Status : FAILED
Error: org.apache.druid.java.util.common.UOE: [org.apache.druid.query.aggregation.histogram.CompressedHistogramAggregatorFactory] does not implement makeAggregateCombiner()
        at org.apache.druid.query.aggregation.AggregatorFactory.makeAggregateCombiner(AggregatorFactory.java:79)
        at org.apache.druid.query.aggregation.AggregatorFactory.makeNullableAggregateCombiner(AggregatorFactory.java:93)
        at org.apache.druid.segment.RowCombiningTimeAndDimsIterator.lambda$new$1(RowCombiningTimeAndDimsIterator.java:120)
        at java.util.Arrays.setAll(Arrays.java:4698)
        at org.apache.druid.segment.RowCombiningTimeAndDimsIterator.<init>(RowCombiningTimeAndDimsIterator.java:120)
        at org.apache.druid.segment.IndexMergerV9.lambda$merge$2(IndexMergerV9.java:910)
        at org.apache.druid.segment.IndexMergerV9.makeMergedTimeAndDimsIterator(IndexMergerV9.java:1031)
        at org.apache.druid.segment.IndexMergerV9.makeIndexFiles(IndexMergerV9.java:180)
        at org.apache.druid.segment.IndexMergerV9.merge(IndexMergerV9.java:915)
        at org.apache.druid.segment.IndexMergerV9.mergeQueryableIndex(IndexMergerV9.java:833)
        at org.apache.druid.indexer.IndexGeneratorJob$IndexGeneratorReducer.mergeQueryableIndex(IndexGeneratorJob.java:610)
        at org.apache.druid.indexer.IndexGeneratorJob$IndexGeneratorReducer.reduce(IndexGeneratorJob.java:782)
        at org.apache.druid.indexer.IndexGeneratorJob$IndexGeneratorReducer.reduce(IndexGeneratorJob.java:569)
        at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:171)
        at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:627)
        at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)

``

``

#2

I think this was a regression introduced in 0.13.0 release. Offending PR - https://github.com/apache/druid/pull/5335
The fix is to have CompressedHistogramAggregatorFactory implement the makeAggregatorCombiner() method.

There are couple of extensions I know of that had to be fixed. Corresponding PRs:

https://github.com/apache/druid/issues/6877

https://github.com/apache/druid/issues/9069

#3

I did not have any operation defined for makeAggregatorCombiner() before, can you describe what the method expects to return and what placeholder code can I put in if there was no operation before?

#4

I can’t find a reference to CompressedHistogramAggregatorFactory in the Druid code base. Is it a custom inbuilt extension? Or probably it was deleted in a release that I haven’t looked at.

Either way, take a look at https://github.com/apache/druid/blob/master/processing/src/main/java/org/apache/druid/query/aggregation/HistogramAggregatorFactory.java#L89

as a reference.

HTH.