Failed to make progress reading messages at (Kafka-Topic)

Apache Druid Q&A Google Groups
#1

I am experiencing an error below when ingesting a kafka topic from a druid supervisor:-

2019-07-04T07:31:38,730 ERROR [task-runner-0-priority-0] org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner - Encountered exception while running task.
org.apache.kafka.common.KafkaException: Failed to make progress reading messages at writesms24-0=29661. Received a non-empty fetch response from the server, but no complete records were found.

2019-07-04T07:31:38,739 INFO [task-runner-0-priority-0] org.apache.druid.indexing.overlord.TaskRunnerUtils - Task [index_kafka_writesms26_538144a4a67713e_cfcibngo] status changed to [FAILED].
2019-07-04T07:31:38,741 INFO [task-runner-0-priority-0] org.apache.druid.indexing.worker.executor.ExecutorLifecycle - Task completed with status: {
“id” : “index_kafka_writesms26_538144a4a67713e_cfcibngo”,
“status” : “FAILED”,
“duration” : 1638,
“errorMsg” : “org.apache.kafka.common.KafkaException: Failed to make progress reading messages at writesms24-0=296…”
}

I have herein attached the necessary files that could be of interest to hack this


[sms-kafka-supervisor.json|attachment](upload://j0zgN0RUHFEtqh31gwqHmJBiq2h.json) (1.24 KB)
#2

Can you check if your Kafka brokers are version 0.10.x or higher?

The Kafka indexing service uses the Java consumer that was introduced in Kafka 0.10.x. As there were protocol changes made in this version, Kafka 0.10.x consumers might not be compatible with older brokers.

#3

Thank you Muthu Lalapet, upgrading my druid from v 0.14 to 0.15 has resolved the issue. Cheers