Query Time Lookup scalability

How scalable are query time lookups in terms of size? Are there particular breakpoints in number of rows where performance drops off, or is it generally linear? Would it make sense to split the lookups into smaller buckets with psuedo-namespacing past a certain size?

(We’re still on the deprecated statically configured lookups under 0.9.1.1 for now, if it makes a difference… we do plan to move to the dynamically configured style fairly soon.)

If you’re using the namespace extension with onheap lookups, they’re ultimately backed by a ConcurrentHashMap and would have the same scalability as that (which tends to be good).

When I did initial performance testing performance was not statistically impacted up to thousands of values in the lookup.

When it gets up to 100’s of thousands or millions there can be an impact.

So if you’re less than 1k you’re absolutely fine. If you’re in the 10s of thousands range you’re probably fine, in the 100’s of thousands you’ll need to be careful and might see a performance impact, and in the millions you’ll see a performance hit.