The Story so Far and Real Time Streaming From Druid

So thanks to the various comments on my posts - they were helpful.

What I have now is:

kafka messaging

druid ingestion from kafka

queries whose results provide evidence druid is receiving the data from kafka

So all good.

My counterpart is building some js web front ends to visualize the analytics we are doing on our druid data.

What we need are methods to access say the most recent interval x of data from druid, from java script ( we are using react and node.js).

Then we need to stream updates as they occur and roll the interval x along as updates occur.

I figure this is something done a million times… does any one have any suggestions that might point us in the right direction?


It seems time boundary query would provide you the needed maxTime (after rollup) of data available.

Also, some people use datasource metadata query for getting the raw timestamp (before rollup) of the data being ingested.