why historical do not cache anything

hi, druid users

I enabled cache for query of broker node and historical node. but from the metrics log, I find there is only cache entry in broker node,but none in historical node.

here is my config of historical node of cache

druid.cache.type=local

druid.cache.sizeInBytes=10737418240

druid.historical.cache.useCache=true

druid.historical.cache.populateCache=true

and below is the metrics log of broker node:

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:28:52.603Z”,“service”:“broker”,“host”:“10.251.194.12:8080”,“metric”:“cache/delta/numEntries”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:28:52.603Z”,“service”:“broker”,“host”:“10.251.194.12:8080”,“metric”:“cache/delta/sizeBytes”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:28:52.603Z”,“service”:“broker”,“host”:“10.251.194.12:8080”,“metric”:“cache/delta/hits”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:28:52.603Z”,“service”:“broker”,“host”:“10.251.194.12:8080”,“metric”:“cache/delta/misses”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:28:52.604Z”,“service”:“broker”,“host”:“10.251.194.12:8080”,“metric”:“cache/delta/evictions”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:28:52.604Z”,“service”:“broker”,“host”:“10.251.194.12:8080”,“metric”:“cache/delta/hitRate”,“value”:0.0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:28:52.604Z”,“service”:“broker”,“host”:“10.251.194.12:8080”,“metric”:“cache/delta/averageBytes”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:28:52.604Z”,“service”:“broker”,“host”:“10.251.194.12:8080”,“metric”:“cache/delta/timeouts”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:28:52.604Z”,“service”:“broker”,“host”:“10.251.194.12:8080”,“metric”:“cache/delta/errors”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:28:52.604Z”,“service”:“broker”,“host”:“10.251.194.12:8080”,“metric”:“cache/total/numEntries”,“value”:13455}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:28:52.604Z”,“service”:“broker”,“host”:“10.251.194.12:8080”,“metric”:“cache/total/sizeBytes”,“value”:87319215}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:28:52.604Z”,“service”:“broker”,“host”:“10.251.194.12:8080”,“metric”:“cache/total/hits”,“value”:7379061}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:28:52.605Z”,“service”:“broker”,“host”:“10.251.194.12:8080”,“metric”:“cache/total/misses”,“value”:142287}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:28:52.605Z”,“service”:“broker”,“host”:“10.251.194.12:8080”,“metric”:“cache/total/evictions”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:28:52.605Z”,“service”:“broker”,“host”:“10.251.194.12:8080”,“metric”:“cache/total/hitRate”,“value”:0.9810822474907424}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:28:52.605Z”,“service”:“broker”,“host”:“10.251.194.12:8080”,“metric”:“cache/total/averageBytes”,“value”:6489}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:28:52.605Z”,“service”:“broker”,“host”:“10.251.194.12:8080”,“metric”:“cache/total/timeouts”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:28:52.605Z”,“service”:“broker”,“host”:“10.251.194.12:8080”,“metric”:“cache/total/errors”,“value”:0}]

and below is the metrics log of historical node:

vent [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:30:09.119Z”,“service”:“historical”,“host”:“10.251.192.248:8080”,“metric”:“cache/delta/numEntries”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:30:09.119Z”,“service”:“historical”,“host”:“10.251.192.248:8080”,“metric”:“cache/delta/sizeBytes”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:30:09.119Z”,“service”:“historical”,“host”:“10.251.192.248:8080”,“metric”:“cache/delta/hits”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:30:09.119Z”,“service”:“historical”,“host”:“10.251.192.248:8080”,“metric”:“cache/delta/misses”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:30:09.119Z”,“service”:“historical”,“host”:“10.251.192.248:8080”,“metric”:“cache/delta/evictions”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:30:09.120Z”,“service”:“historical”,“host”:“10.251.192.248:8080”,“metric”:“cache/delta/hitRate”,“value”:0.0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:30:09.120Z”,“service”:“historical”,“host”:“10.251.192.248:8080”,“metric”:“cache/delta/averageBytes”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:30:09.120Z”,“service”:“historical”,“host”:“10.251.192.248:8080”,“metric”:“cache/delta/timeouts”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:30:09.120Z”,“service”:“historical”,“host”:“10.251.192.248:8080”,“metric”:“cache/delta/errors”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:30:09.120Z”,“service”:“historical”,“host”:“10.251.192.248:8080”,“metric”:“cache/total/numEntries”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:30:09.120Z”,“service”:“historical”,“host”:“10.251.192.248:8080”,“metric”:“cache/total/sizeBytes”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:30:09.121Z”,“service”:“historical”,“host”:“10.251.192.248:8080”,“metric”:“cache/total/hits”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:30:09.121Z”,“service”:“historical”,“host”:“10.251.192.248:8080”,“metric”:“cache/total/misses”,“value”:36058}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:30:09.121Z”,“service”:“historical”,“host”:“10.251.192.248:8080”,“metric”:“cache/total/evictions”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:30:09.121Z”,“service”:“historical”,“host”:“10.251.192.248:8080”,“metric”:“cache/total/hitRate”,“value”:0.0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:30:09.121Z”,“service”:“historical”,“host”:“10.251.192.248:8080”,“metric”:“cache/total/averageBytes”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:30:09.121Z”,“service”:“historical”,“host”:“10.251.192.248:8080”,“metric”:“cache/total/timeouts”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:30:09.121Z”,“service”:“historical”,“host”:“10.251.192.248:8080”,“metric”:“cache/total/errors”,“value”:0}]

my question is why historical node does not have node cache entry?

thx.

Event [{“feed”:“metrics”,“timestamp”:“2015-09-15T12:37:14.511Z”,“service”:“historical”,“host”:“10.251.194.244:8080”,“metric”:“request/time”,“value”:40,“user2”:"***********",“user3”:"{“bySegment”:true,“finalize”:false,“intermediate”:true,“populateCache”:false,“priority”:0,“queryId”:“d7c4f853-43fb-46fb-9252-e1ae67bd4c74”,“timeout”:300000}",“user4”:“topN”,“user5”:[“2015-09-09T12:00:00.000Z/2015-09-09T13:00:00.000Z”],“user6”:“true”,“user7”:“10.251.194.12”,“user8”:“d7c4f853-43fb-46fb-9252-e1ae67bd4c74”,“user9”:“PT60M”}]

and from the log, the populateCache alway false. is there some config I miss config?

在 2015年9月15日星期二 UTC+8下午8:34:01,何文斌写道:

and this is another issue I nee help, I set the max cache size is 10GB, but druid begin to remove cache entry while my cache size in about 80M(80082947).

I don’t know why. and also I find another issue, the cache will all gone if you do not query for a while. hope some advices from you guys .

thx.

timestamp":“2015-09-16T03:08:00.078Z”,“service”:“broker”,“host”:“10.251.193.6:8080”,“metric”:“cache/delta/sizeBytes”,“value”:-18549}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-16T03:08:00.078Z”,“service”:“broker”,“host”:“10.251.193.6:8080”,“metric”:“cache/delta/hits”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-16T03:08:00.078Z”,“service”:“broker”,“host”:“10.251.193.6:8080”,“metric”:“cache/delta/misses”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-16T03:08:00.078Z”,“service”:“broker”,“host”:“10.251.193.6:8080”,“metric”:“cache/delta/evictions”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-16T03:08:00.078Z”,“service”:“broker”,“host”:“10.251.193.6:8080”,“metric”:“cache/delta/hitRate”,“value”:0.0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-16T03:08:00.079Z”,“service”:“broker”,“host”:“10.251.193.6:8080”,“metric”:“cache/delta/averageBytes”,“value”:475}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-16T03:08:00.079Z”,“service”:“broker”,“host”:“10.251.193.6:8080”,“metric”:“cache/delta/timeouts”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-16T03:08:00.079Z”,“service”:“broker”,“host”:“10.251.193.6:8080”,“metric”:“cache/delta/errors”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-16T03:08:00.079Z”,“service”:“broker”,“host”:“10.251.193.6:8080”,“metric”:“cache/total/numEntries”,“value”:46632}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-16T03:08:00.079Z”,“service”:“broker”,“host”:“10.251.193.6:8080”,“metric”:“cache/total/sizeBytes”,“value”:80082947}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-16T03:08:00.079Z”,“service”:“broker”,“host”:“10.251.193.6:8080”,“metric”:“cache/total/hits”,“value”:21667396}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-16T03:08:00.079Z”,“service”:“broker”,“host”:“10.251.193.6:8080”,“metric”:“cache/total/misses”,“value”:189587}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-16T03:08:00.079Z”,“service”:“broker”,“host”:“10.251.193.6:8080”,“metric”:“cache/total/evictions”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-16T03:08:00.079Z”,“service”:“broker”,“host”:“10.251.193.6:8080”,“metric”:“cache/total/hitRate”,“value”:0.9913260215282228}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-16T03:08:00.080Z”,“service”:“broker”,“host”:“10.251.193.6:8080”,“metric”:“cache/total/averageBytes”,“value”:1717}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-16T03:08:00.080Z”,“service”:“broker”,“host”:“10.251.193.6:8080”,“metric”:“cache/total/timeouts”,“value”:0}]

Event [{“feed”:“metrics”,“timestamp”:“2015-09-16T03:08:00.080Z”,“service”:“broker”,“host”:“10.251.193.6:8080”,“metric”:“cache/total/errors”,“value”:0}]

在 2015年9月15日星期二 UTC+8下午8:34:01,何文斌写道:

Your config says “historical” but your emitter is reporting “broker”. Is this a historical or broker node?

It looks like the cache was turned on for historical nodes, not broker nodes.

I wonder if the broker also has caching enabled and queries are hitting the broker cache and not the historical one.

thx Fangjin & charles

so if I enable cache both historical and broker, then historical node would not cache so much because most part query hit the cache in broker node, is it right?

thx.

在 2015年9月17日星期四 UTC+8上午8:12:04,Fangjin Yang写道:

Yes.