Querying mixed queryGranularity segments

Hi all,

Because of roll-up of “old” segments from hourly to daily granularity. We have within the same datasource segments with 2 different granularities. Day from t-j to t-i and hour from t-i to t, j < i

When issuing a timeserie query with hour granularity on an interval that has both hourly and daily segments (we set queryGranularity to be equal to segmentGranularity), Druid also returns segments with segmentGranularity/queryGranularity of DAY.

This seem a bit odd to me since we are showing different things on the same plot. More over it can be misleading to the end-user (using Pivot for instance), who sees a sudden drop of the timeserie data at the rollup limit.

Is there a way to prevent this “problem” or a plan to change how Druid is handling a case like this ?

Thanks for your input.

Here is a potentially relevant Pivot issue: https://github.com/implydata/pivot/issues/226

I am not aware of a meaningful way to combine data bucketed to two different granularities in the same chart. All the solutions bellow focus on how to make it so that it does not happen.

Here are some possible solutions with Pivot:

  1. One solution is to restrict Pivot (or whatever graphing tool you use) so that the minimum granularity that can be bucketed on is ‘P1D’. This will prevent your users form being confused because they will never see the drop, but that might not be acceptable as you might want to see the hourly data.

  2. You could also not rollup your older segments to a rougher granularity, keep it all hourly. This will be more expensive (less rollup) but your users will appreciate being able to see hourly data going back. I.e. if they are doing a filter over the granularity boundary and splitting by hour the ideal experience would be for there not to be a granularity boundary

  3. Saving the best solution for last: you can create two Pivot data cubes one called “My Data Hourly” that is restricted (via subsetFilter) to [t-i, t] and another data cube on the same datasource with the same dimensions but restricted to P1D bucketing called “My Data Historical”

Hope this helps.