We are enabling cache on our Historical processes. We have fairly decent machines for our Historical with 16 cores and 80GB RAM and 1TB SSD. Going through Historical processes configuration[1] & cache configuration[2], I have few questions:
-
Can the configuration mentioned in [2] be set along with the process level cache configuration such as druid.historical.cache.*? Or are they mutually exclusive?
-
What is the difference between druid.cache.sizeInBytes & druid.historical.cache.maxEntrySize ? Can both of them be configured?
-
I set druid.historical.cache.maxEntrySize on my Historical to 10 GB but when restarting, the server threw the following error
-
Caused by: com.fasterxml.jackson.databind.exc.InvalidFormatException: Cannot deserialize value of type
intfrom String “10737418240”: Overflow: numeric value (10737418240) out of range of Integer (-2147483648 - 2147483647)at [Source: UNKNOWN; line: -1, column: -1] (through reference chain: org.apache.druid.client.cache.CacheConfig[“maxEntrySize”])
-
Regarding the above error, can we not configure cache size to be more than 2GB? We have thetaSketch metrics which will need a lot of memory to be cached, and that is why we started with 10GB for druid.historical.cache.maxEntrySize. Am I wrong with the assumption?
-
When having thetaSketch metrics in our data source, is there any guideline or recommendation on how cache should be configured?
[1] https://druid.apache.org/docs/latest/configuration/index.html#historical-caching
[2] https://druid.apache.org/docs/latest/configuration/index.html#cache-configuration
Thanks.