Kafka Indexing Service - Unable to delete tasks?

Hello, we successfully setup a kafka-druid cluster with the Kafka-Indexing-Service.
Everything was working until we added one dimension. Once added, we decided to delete all old data (because it was test-dummy-data anyway). Then we cleaned up all kafka messages, cleaned the druid datasources (deleted: s3 folder, druid-mysql, druid cache), restarted druid services, restarted the indexing supervisor.

Problem (while the producer is shutdown here and does not send messages to kafka):

  • we start the druid services and the supervisor after a cleanup and we immedeately get tasks which we do not know where they come from?!? Those task seem to be in some kind of loop, you kill them and there is a new task. If you do not kill them they are eventually processed as “success” but also there is no datasource setup and endless new tasks created.

Basically the question is if there are still some other caches druid uses for tasks which we forgot to delete?

Anyone can help here? Thanks

These tasks are created by the kafka supervisor, you will need to shutdown the supervisor by sending a request to-
POST /druid/indexer/v1/supervisor//shutdown

docs - http://druid.io/docs/latest/development/extensions-core/kafka-ingestion.html

Of course we also stopped the supervisor before deleting the tasks.
Immeadeately once we start the supervisor again (after deleting everything) we got the old tasks again right away.

Solved, solution was:

POST /druid/indexer/v1/supervisor/<supervisorId>/reset