Multiple segment cache locations

Hi Team,

In our historical nodes, we have multiple mounts can we have multiple segment cache locations and add the overall maxSize as shown below



Processing threads and buffers



HTTP server threads



Segment storage

druid.segmentCache.locations=[{“path”: “/mnt01/data/indexCache”, “maxSize”: 1800000000000},{“path”: “/mnt02/data/indexCache”, “maxSize”: 1800000000000},{“path”: “/mnt03/data/indexCache”, “maxSize”: 1800000000000},{“path”: “/mnt04/data/indexCache”, “maxSize”: 1800000000000}]

druid.monitoring.monitors=[“io.druid.server.metrics.HistoricalMetricsMonitor”, “com.metamx.metrics.JvmMonitor”]



Hi Sathish, what is your question?

A related question for me is - how are the multiple segment cache locations used by druid - is the second location used after the first one fills up? Or is there a way to map segments of a datasource to a particular cache location? The documentation is a bit thin on this.

Hi Satish/Neelesh,

Yes, It is possible to specify multiple segment cache locations in Druid as specified in configs above.

When a new segment is to loaded in segment cache, The historical node tries to select the locations in the order of remaining capacity i.e the one having the most available space is selected for loading the segment.

No, currently it is not possible specify an affinity based on datasource for segment cache to be used.

Hi Nishant,

Thanks for the clarification, it is working as expected


Thanks for the details, Nishant! We were exploring if it was possible to have encrypted segment cache for specific datasources, hence the question.