Kafka Indexing service not loading old data


I am trying the Kafka Indexing Service with druid 0.10.0 (actually imply.io distribution). When loading realtime data, it works fine. However, if I try to load old data, it does nothing. There are no errors logged, it’s like it’s not even receiving the events.

How can I debug this? Is there any particular setting I should check?


That’s strange, it should be able to load older data. Maybe confirm that it’s not just stuck, by checking whether if you produce some older data to Kafka and then later, also some newer data, does the newer data all get loaded. Also make sure you haven’t turned on lateMessageRejectionPeriod. Also double-check to see if there’s anything suspicious in the index_kafka_xxx task logs.