I have two related questions.
The first is a quick question on the wording on lookup sizes in the Druid documentation. It is about the following paragraph on http://druid.io/docs/latest/development/extensions-core/lookups-cached-global.html:
*Globally cached lookups are appropriate for lookups which are not possible to pass at query time due to their size,
or are not desired to be passed at query time because the data is to reside in and be handled by the Druid servers,
and are small enough to reasonably populate on a node. This usually means tens to tens of thousands of entries per lookup.*Does this mean that there is no way to do ‘joins’ with large tables of dynamically changing data at query time with Druid?
The second question is more general, and I need to explain a bit before I can pose it.
My use case is that I want to do a low-latency topN query, where the items need to be filtered by availability (among others) at query time. The availability of an item is a piece of dynamically changing data that resides somewhere else (there is some freedom in choosing ‘somewhere else’, be it an RDBMS or a key value store). The total number of items is around 2 million, while the number of events is in the order of 20 million per day. Is there some way to accomplish this efficiently using Druid?
Kind regards, and thanks in advance (for reading this),