Use Spark with Tranquility

We are pre-processing data with Spark and wanted to use Tranquility to ingest data into Druid. I can’t find any APIs to use Spark with Druid. What is the best way forward?

Hey Saksham,

You can use tranquility’s direct API along with Spark Streaming’s foreachRDD to do this. The section “Design Patterns for using foreachRDD” on the Spark docs have some samples: http://spark.apache.org/docs/latest/streaming-programming-guide.html

Something like the last sample there should work well with tranquility. Tranquility beams can be used by multiple threads at a time, so you don’t need a ConnectionPool- just a singleton would suffice.