Recently we want to use Druid to help setting up our data pipeline, thus I am using is locally to evaluate the solution. So I tried to setup an realtime analytics scenario with Tranquility and Caravel. The process is like this:
Events ----> Tranquility ----> Druid —> Caravel/Metabase
What I find is that it’s slow locally when Druid indexing the json data, which I understand normal in local environment. One problem I find is that Caravel try get the druid datasource by visiting this uri
/druid/coordinator/v1/metadata/datasources, and it can’t get the datasource until the indexing task is finished.
I am a little confused of this, is the new
datasource added after the indexing finished? or is there an config about a time range for creating the datasource when indexing realtime events?
I also tried Metabase, which I found it would be faster to get the datasource information of druid. It would be interesting to see why this is diferent.
So my actual question is , given there’s an event posted to druid, when would druid create the
datasource and let it show up through
Thanks a lot.