Kafka İngestion

Hi everyone,

Is there any way to ingest historical data by kafka.
I’m using druid 0.9 with kafka firehose

Best regards,


Hey Yunus,

Right now, there is not a way to do that directly from Kafka. We are working on one for a future release though :slight_smile:

Currently the best way to load historical data from Kafka is to first write it to S3 or HDFS (using something like Secor or Camus or Gobblin) and then load it from there into Druid using Druid’s batch loading mechanisms.

is there an issue for this on github to follow?

22 Nisan 2016 Cuma 20:50:04 UTC+3 tarihinde Gian Merlino yazdı:

Hi Yunus,

you can follow https://github.com/druid-io/druid/issues/1642 for the ingestion improvements.