SelectQuery and granularity

I’m confused about the granularity property for SelectQuery. Since it’s returning the row level data, does it have any effect at all? Only in certain circumstances?

I tried an experiment (using 0.8.2) where I did some real time ingestion of events where the ingestion spec had both segmentGranularity and queryGranularity of 1 minute.

I sent three events in, something like:

{ timestamp: 00:00, dim: foo, metric1: 10 }

{ timestamp: 00:30, dim: foo, metric2: 20 }

{ timestamp: 01:15, dim: foo, metric3: 30 }

So the first two events are in the first minute-long segment (00:00-00:59), and got rolled up due to queryGranularity. The last event went into the second segment (01:00-01:59). All as expected.

I tried a Select query over this interval with granularity: hour. The response reflected how the rows were stored in the segments:

{ timestamp: 00:00, dim: foo, metric1: 10, metric2: 20 } # from 00:00-00:59

{ timestamp: 01:15, dim: foo, metric3: 30 } # from 01:00-01:59

That was not entirely surprising, based on my understanding of how a Select query works. Though for my use case, it would have been handy had the two rows been rolled up - for context of the kind of scenario I’m playing with, see this post. If Select did roll them up to a single “row”, I could emit my events at their real timestamp rather than having to “backdate” them all to the original timestamp, which will necessitate keeping indexers around longer to accommodate the backdated events.

It leaves me wondering, what is the purpose of granularity for this kind of query? Does it ever have any effect? Only if granularity is finer than the original queryGranularity or something?

Hey Ryan,

You can think of “granularity” in Druid (for any query) as meaning “do a subquery for each granular bucket”. For “timeseries” this is clearly useful. For “topN” it is useful in some cases (you want the top values broken down by hour). For “select” this seems of limited usefulness, at least to me. I’ve only ever found select with granularity “all” to really be useful.