I have developed a firehose in python which will keep streaming me json data (using http get requests to an external site).
This will stream about 500 messages/data per second. I would like to ingest this data to druid and report if using caravel.
I am not sure if i should use Stream push or stream pull. Please suggest how to decide between the two.
If i use stream push , can i use tranquility. I heard the tranquility might not scale for production system. If so, what are the other options apart from tranquility.
If i use stream pull, are the examples which i can use. I am not able to find detailed example.