Kafka index service OffsetOutOfRangeException error

Hi,

I got the OffsetOutOfRangeException from time to time and result in the kafka index task failure.

2017-10-26T20:24:16,165 WARN [task-runner-0-priority-0] io.druid.indexing.kafka.KafkaIndexTask - OffsetOutOfRangeException with message [Offsets out of range with no configured reset policy for partitions: {xxxx-1=3577891}]

The following is the configuration of kafka index configuration and i set resetOffsetAutomatically to true per the druid document.

 "tuningConfig": {
    "type": "kafka",
    "maxRowsPerSegment": 5000000,
    "resetOffsetAutomatically": true
  },
  "ioConfig": {
    "topic": "xxxx",
    "consumerProperties": {
      "bootstrap.servers": "xxxxxx"
    },
    "taskCount": 3,
    "replicas": 1,
    "taskDuration": "PT1H",
    "useEarliestOffset": "false"
  }

Is there any configuration in druid i can do to prevent the task from being failed due to this exception? The current kafka log retention time is 168 hours so i am not sure why i got this error in druid.

Thanks in advance.

Is there any more logging you can provide? The message “OffsetOutOfRangeException with message [%s]” is logged before KIS tries to correct the offset pointer. (io.druid.indexing.kafka.KafkaIndexTask line 432)

Hi PeiWang,

The error message is saying that Druid is trying to read offset 3577891 from topic xxxx partition 1, but Kafka isn’t serving it up.

First I would suggest double checking that you are using a version in which resetOffsetAutomatically is supported. It was a relatively recent addition.

Other than that, is it possible that your ingestion was stuck long enough that the mentioned offset (3577891) got expired? Or is it possible that someone deleted or renamed the topic? Or changed the topic name in the Druid Kafka indexing spec? Any of those, I think, could potentially cause this.