Is there a resource limit parameter for groupBy v2?

It will automatically limit resources based upon available memory. It will spill to disk based upon druid.query.groupBy.maxOnDiskStorage, which is 0 by default. If a query exceeds memory it will fail immediately. If you set druid.query.groupBy.maxOnDiskStorage to non-0, then a query will fail only if it runs out of disk space.

Here’s more reading and a bit of context.