Is tranquility/Druid good for hifg thoughput realtime analytics

Hi ,

I have been experimenting with druid with many days , try to insert data via tranquility , its seems to fail at smallest of load I get with dropped message or some transient errors

Is Tranquilitly /Druid is good choice for realtime systems ?or its good for historic data push via hadoop

Do you think apache flink (for inmemory aggregation) + cassandra is better and proven for realtime analytics

Regards ,


its seems tranquility drops everything as soon as reaches semi decent tps , why its dropping …cant it handle higher put in queue?

Really disappointing

0:47:32.674 [main] DEBUG c.m.t.tranquilizer.Tranquilizer - Swapping out buffer with 1 messages, 1 batches now pending.

20:47:32.674 [main] DEBUG c.m.t.tranquilizer.Tranquilizer - Sending buffer with 1 messages.

20:47:32.674 [main] DEBUG c.m.t.tranquilizer.Tranquilizer - Sent[0], dropped[1], failed[0] out of 1 messages from batch #213303. 0 batches still pending.

20:47:32.674 [main] WARN m.d.s.d.t.DataIngestionService - Dropped message: %s

com.metamx.tranquility.tranquilizer.MessageDroppedException: Message dropped

Messages are dropped by tranquility if they are older than windowPeriod. you can try to tune windowPeriod to increase the threshold there.