Unable to Cache GroupBy query

Hi,

I am using groupBy query to query druid using v2 strategy, but i don’t think it is getting cached. It is taking 66 seconds every time for getting the response.

Please find below my configuration for broker node -

Query cache

druid.broker.cache.useCache=true

druid.broker.cache.unCacheable=[“select”]

I also have useCache and populateCache enabled for historical nodes.

When I check the logs, I found something like -

Event [{“feed”:“metrics”,“timestamp”:“2017-02-08T23:58:00.837Z”,“service”:“druid/broker”,“host”:“localhost:8080”,“metric”:“segment/scan/pending”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2017-02-08T23:58:00.837Z”,“service”:“druid/broker”,“host”:“localhost:8080”,“metric”:“query/cache/delta/numEntries”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2017-02-08T23:58:00.837Z”,“service”:“druid/broker”,“host”:“localhost:8080”,“metric”:“query/cache/delta/sizeBytes”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2017-02-08T23:58:00.837Z”,“service”:“druid/broker”,“host”:“localhost:8080”,“metric”:“query/cache/delta/hits”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2017-02-08T23:58:00.837Z”,“service”:“druid/broker”,“host”:“localhost:8080”,“metric”:“query/cache/delta/misses”,“value”:38332}]

Event [{“feed”:“metrics”,“timestamp”:“2017-02-08T23:58:00.837Z”,“service”:“druid/broker”,“host”:“localhost:8080”,“metric”:“query/cache/delta/evictions”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2017-02-08T23:58:00.837Z”,“service”:“druid/broker”,“host”:“localhost:8080”,“metric”:“query/cache/delta/hitRate”,“value”:0.0}]

Event [{“feed”:“metrics”,“timestamp”:“2017-02-08T23:58:00.837Z”,“service”:“druid/broker”,“host”:“localhost:8080”,“metric”:“query/cache/delta/averageBytes”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2017-02-08T23:58:00.837Z”,“service”:“druid/broker”,“host”:“localhost:8080”,“metric”:“query/cache/delta/timeouts”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2017-02-08T23:58:00.838Z”,“service”:“druid/broker”,“host”:“localhost:8080”,“metric”:“query/cache/delta/errors”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2017-02-08T23:58:00.838Z”,“service”:“druid/broker”,“host”:“localhost:8080”,“metric”:“query/cache/total/numEntries”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2017-02-08T23:58:00.838Z”,“service”:“druid/broker”,“host”:“localhost:8080”,“metric”:“query/cache/total/sizeBytes”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2017-02-08T23:58:00.838Z”,“service”:“druid/broker”,“host”:“localhost:8080”,“metric”:“query/cache/total/hits”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2017-02-08T23:58:00.838Z”,“service”:“druid/broker”,“host”:“localhost:8080”,“metric”:“query/cache/total/misses”,“value”:3452957}]

Event [{“feed”:“metrics”,“timestamp”:“2017-02-08T23:58:00.838Z”,“service”:“druid/broker”,“host”:“localhost:8080”,“metric”:“query/cache/total/evictions”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2017-02-08T23:58:00.838Z”,“service”:“druid/broker”,“host”:“localhost:8080”,“metric”:“query/cache/total/hitRate”,“value”:0.0}]

Event [{“feed”:“metrics”,“timestamp”:“2017-02-08T23:58:00.838Z”,“service”:“druid/broker”,“host”:“localhost:8080”,“metric”:“query/cache/total/averageBytes”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2017-02-08T23:58:00.838Z”,“service”:“druid/broker”,“host”:“localhost:8080”,“metric”:“query/cache/total/timeouts”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2017-02-08T23:58:00.838Z”,“service”:“druid/broker”,“host”:“localhost:8080”,“metric”:“query/cache/total/errors”,“value”:0}]

Here I only get value for misses. Could someone please let me know, what is it that I am missing?

Thanks,

Vraj

Hey Vraj,

If your cache is not shared, then without populateCache set on the broker, it won’t actually have anything in its cache and every lookup will miss. Also, https://github.com/druid-io/druid/issues/3820 means that you should avoid enabling groupBy v2 caching on the broker. Instead, try caching on historical nodes.

Are the historical nodes always missing too? Do they have all of their cache properties set? It should look something like this:

Query cache

druid.historical.cache.useCache=true

druid.historical.cache.populateCache=true

druid.historical.cache.unCacheable=[“select”]

druid.cache.type=local

druid.cache.sizeInBytes=2000000000

In particular, if you’re missing druid.cache.sizeInBytes, or useCache, or populateCache, then caching won’t work right.