Tranquility Client side behavior for realtime ingestion

Hi :

We are trying to use Druid for implementing a real time data pipeline.

While using the tranquility api … we have certain questions about dropping messages :-

  1. In this since the sender is sending messages asynchronously to druid , What are the guarantees for the call :-

*FutureEventListener’s OnSuccess method . *

Does this mean the tranquility client successfully dispatched the message to Druid and then druids realtime server process will take care of actual ingestion ?

  1. sender.flush() and sender.stop() could there be a window of time in between these 2 where on client side message can be lost ?

More so what does that sender.flush() actually achieve.

  1. For a steady state operation( no fault taulerence scenarios) , could there be a scenario where the realtime node can drop a message successfully sent from tranquility client .

Thanks a bunch !!

Hi :

Actually looked at the code a little closer .

#1 Guarantee is give by the twitter api so not an issue now .

#2 We still need help with this .

Will appreciate some help to get this off the ground .

sender.flush() will make sure all the events we have sent via calling sender.send() are processed and callbacks for either onSuccess or onFailure are called.
Since there will be no more events to be processed in above example between flush and stop. Its safe to call a stop after flush.