Running Historical Nodes In-Memory

I’m hoping to start running some historical nodes in-memory (or at least a percentage in-memory) instead of memory mapping them all but I’m having trouble sorting through the documentation/forum to find what I’m looking for. I’ve heard that lowering the druid.server.maxSize is the key to this, but I’m messing with that config and finding that druid is just using that number to size the on disk storage. Any help/pointing me in the right direction would be super appreciated.


Michael Capeloto

Druid always memory-maps its segments, so the idea is that if your JVM heap + druid.server.maxSize + off-heap scratch space + anything else running on the box is less than your total ram, you’ll effectively be running in memory since segments will never get paged out by the OS.

Ah thanks for the explanation! I’ve seen that formula but I guess my brain was whiffing on the actual dynamic.