NoSuchMethodError in Hadoop


I'm facing the following stack trace in the mapper while executing HadoopDruidIndexer task on EMR. I'm suspecting its pulling in an old version of Guava, as EMR comes bundled with v11. Any thoughts on how to fix?

2016-01-19 22:07:06,528 ERROR [main] org.apache.hadoop.mapred.YarnChild: Error running child : java.lang.NoSuchMethodError:;)Lcom/google/common/hash/Hasher;
        at io.druid.query.aggregation.cardinality.CardinalityAggregator.hashRow(
        at io.druid.query.aggregation.cardinality.CardinalityAggregator.aggregate(
        at io.druid.indexer.InputRowSerde.toBytes(
        at io.druid.indexer.IndexGeneratorJob$IndexGeneratorMapper.innerMap(
        at org.apache.hadoop.mapred.MapTask.runNewMapper(
        at org.apache.hadoop.mapred.YarnChild$
        at Method)
        at org.apache.hadoop.mapred.YarnChild.main(

Hey Dan,

We generally try to be careful about which Guava methods we use on code paths that are typically called during Hadoop jobs. Normally people don’t use the “cardinality” aggregator at indexing time (it’s intended for query-time counting of string columns). If you want to create a hyperloglog column at indexing time, you could use the “hyperUnique” aggregator.

Thanks Gian, I’ll give it a try.
I was under the immersion I was building up a HLL column. I think the documentation for cardinality/hyperUnique aggregators is a bit misleading.

Cardinality aggregator works on string dimensions and creates a HLL aggregator at query time, not ingest time.

Agree that docs should be better.

If you wanted to make a contribution to improve the docs, that would be super helpful to us.