I’m using druid-0.9.1 with features of Kafka-indexing-service. That’s really awesome!
My imply cluster has one node with zk + broker + pivot + overlord + coordinator, one node with overlord + coordinator for HA, and eight nodes with middlemanager + historical
My Kafka cluster has nine servers, and data retention is set by 2 HOUR.
In Spec of Kafka-indexing-service, I add an option “auto.offset.reset” : “latest” in “consumerProperties“, when I restart supervisor for my topic, I can find “auto.offset.reset=latest” in overlord.log, but this option is still set “none” in worker Spec, so worker will throws an OffsetOutofRangeException when reading an expired offset in partitions . Many Kafka indexing tasks failed and numbers of segments of each hour reduce a lot.
How can I fix this problem, can anyone help me solve this problem?
“bootstrap.servers”: “184.108.40.206:9092, 220.127.116.11:9092,18.104.22.168:9092,22.214.171.124:9092,126.96.36.199:9092,188.8.131.52:9092,184.108.40.206:9092,220.127.116.11:9092,18.104.22.168:9092,”,
“auto.offset.reset” : “latest”