After some tests, it appears like the broker merge all segments results anytime and i’m a bit confuse at this point, let me explain.
We have a use case where the segment granularity is “day”, and we launch a query on a week of data but we need the result for each day.
So technically the broker receive the query, forward to each segment (7) in the week of data, then merge the results and sort per day.
We know that because when we ask for the result to be weekly, it is faster than per day result.
So here are my questions:
1- Why query a week of data (when segment granularity is “day”) is faster when we ask for the result to be weekly than ask for the result to be per day?
- The broker doesn’t sort the result of merged segments results when the interval match the period ? In this case, why the broker merge the result of segments when we ask for day result ?
- The broker sort the result in the two cases but it is much faster when the period is one week than a period of one day ?
2- If the broker merge and sort every time, can we imagine, in the future, to build this kind of intelligence in druid (if the query period match the segment granularity then don’t merge and sort) or is it way too complex?
Many thanks guys,