Error with kafka indexing service

2018-04-11T18:03:01,873 ERROR [task-runner-0-priority-0] io.druid.indexing.overlord.ThreadPoolTaskRunner - Exception while running task[KafkaIndexTask{id=index_kafka_processo-kafka_8d2a4d662c5e8ed_gddacjcg, type=index_kafka, dataSource=processo-kafka}]
io.druid.java.util.common.ISE: WTF?! Got offset[6] after offset[5] in partition[0].
	at io.druid.indexing.kafka.KafkaIndexTask.run(KafkaIndexTask.java:671) ~[?:?]
	at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:444) [druid-indexing-service-0.12.0.jar:0.12.0]
	at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:416) [druid-indexing-service-0.12.0.jar:0.12.0]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_161]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_161]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_161]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_161]
2018-04-11T18:03:01,898 INFO [task-runner-0-priority-0] io.druid.indexing.overlord.TaskRunnerUtils - Task [index_kafka_processo-kafka_8d2a4d662c5e8ed_gddacjcg] status changed to [FAILED].
2018-04-11T18:03:01,901 INFO [task-runner-0-priority-0] io.druid.indexing.worker.executor.ExecutorLifecycle - Task completed with status: {
  "id" : "index_kafka_processo-kafka_8d2a4d662c5e8ed_gddacjcg",
  "status" : "FAILED",
  "duration" : 150872
}

I´m new in Druid, and I want to ask if someone can help me to solve this error

quarta-feira, 11 de Abril de 2018 às 19:11:23 UTC+1, Aricson do rosario escreveu:

Looking at the code, setting “skipOffsetGaps” in the ioConfig of your supervisor spec would prevent that exception from being thrown (a warning would be logged instead).

I’m not familiar enough with the Kafka indexing to say if that’s the correct thing to do though (i.e., are such offset gaps normal?), maybe someone else can comment on that.

Thanks,

Jon

I will try your sugestion.

Thanks Jon

After setting “skipOffsetGaps” in the ioConfig, kafka indexing service seems to working, at least with realtime data.

Thanks Jon.