groupBy v2 query fails if interval includes real-time segment

Hi!

Is it possible that groupBy v2 fails on segments that are still being processed by real-time nodes?

We’ve ran many tests on our environment, and old groupBy seems to be work OK if the interval is ahead of time, but v2 fails if segments are present on real-time node.

Thanks,

It definitely is supposed to work and I’ve seen it work in other environments.

What is failing exactly?

When there is a groupBy query with the interval containing segmens in real-time nodes, there are two cases:

  • If strategy is v1, works ok.

  • if strategy is v2, the return is Unkown exception. There isnt even any detailed information or error on historical segments.

Tomorrow I will provide some log information and example queries!

Thanks Gian

One thing to double-check is make sure you have numMergeBuffers set on your middleManagers. If they don’t have merge buffers then they can’t do v2 groupBys. If that’s not it, then logs are probably going to be helpful.

Yes that was it! My bad not to check that, of course segments present on middle managers would need merge buffers as well.

Thanks again for all the help Gian, much appreciated!!

Yes that was it! My bad not to check that, of course segments present on middle managers would need merge buffers as well.

Thanks again for all the help Gian, much appreciated!!