Filtered aggregator in batch and realtime ingestion

Hey all,
Were filtered aggregators only for query time aggregations not for batch time aggregations?

I have a wide set with different types of events that we need to keep count on, imp, click… etc the events are distinguished by an event_type_id dimension.

I tried to use a filtered aggregator in the ingestion spec of realtime to increment specific count metrics when the dimension matched a specific value.

I get an error when i try to do that… (i don’t have the error anymore to post)

Is there a way to use filtered aggregation as a part of realtime or batch ingestion specs?

Filtered aggregators were designed for query-time use. We should make it clearer in our documentation which aggregators can be used at ingestion time and which ones not.

We have an issue opened for this: https://github.com/druid-io/druid/issues/2061

Is there a reason it’s being restricted to query-time usage? I’ve noticed discrepency between counts done during ingestion and the filtered aggregation counts.

As far as I know it just hasn’t been implemented yet.

What discrepancies are you seeing?

Aaron, please read about rollup in Druid: http://druid.io/docs/latest/

and

http://druid.io/docs/0.8.2/ingestion/faq.html