server.maxSize confusion

Hi,

In the Performance-FAQ it says about maxSize

Setting this parameter to a value greater than the total memory capacity on a node and may cause disk paging to occur.
Is this the “druid.server.maxSize” for historical?

And again in all examples server.maxSize is set to exactly the same size as the druid.segmentCache.locations maxSize which is local disk storage size.
Also in the production cluster example it’s set to almost 300GB, while the node has 244GB of ram
So I’m confused, is the Performance FAQ wrong or is everyone else on this mailinglist wrong?
Is this the size to set how much data is stored on local disk or how much data is stored in off-heap ram?

Thanks
/Elias A

see inline

Hi,

In the Performance-FAQ it says about maxSize

Setting this parameter to a value greater than the total memory capacity on a node and may cause disk paging to occur.
Is this the “druid.server.maxSize” for historical?

Yes this is the max size of total segment served by one historical node

And again in all examples server.maxSize is set to exactly the same size as the druid.segmentCache.locations maxSize which is local disk storage size.
Also in the production cluster example it’s set to almost 300GB, while the node has 244GB of ram
So I’m confused, is the Performance FAQ wrong or is everyone else on this mailinglist wrong?

Nothing wrong, the segments file are memory mapped hence if you have enough swap space (ram + disk) the OS will manage paging of the files

Is this the size to set how much data is stored on local disk or how much data is stored in off-heap ram?

it is how much data that will be memory mapped via the OS hence some pages will be on RAM and some on Disk if the memory is full.