Kinesis indexing service for druid

Hello druid-users,

Is there a kinesis indexing service for druid similar to the kafka indexing service. I noticed mention of druid-kinesis-indexing-service in imply documents: https://docs.imply.io/cloud/manage-data/ingestion-kinesis or is it proprietary? Is there an open source equivalent?

Hi Arup,

It is not open source, although you can try out the extension by downloading the Imply distribution at: https://imply.io/get-started. We have been considering opening it, though, so you may see it appear as a contribution to Druid in the future.

Gian,

Thank you for your response. I think kinesis indexer would make a great open source contribution, and appreciate the effort put in building it. I will definitely play with it.

Another question, does imply license allow using the kinesis indexer in production alongside the oss distro of druid? Or do we need to obtain a different license to run it in production.

Thanks,

Arup

Meanwhile there is a pull request in the Druid repo for adding a Kinesis Indexing Service, currently scheduled for Druid 0.14

As far as I can see, this does not seem to be related to the Imply product, rather an independent contribution. Perhaps this also influences Imply’s decision as to whether to opensource their implementation or not.

Josh (the author of that PR) did that work while interning at Imply :slight_smile: