Hello druid-team,
We are using kafka for realtime data streaming ingestion to our data sources. But one thing we observed is that sometimes kafka index tasks are going into idle state. Neither new kafka index tasks are getting created nor existing task is getting terminated.
And when we check respective kafka supervisor status, it is in UNHEALTHY SUPERVISOR state, having both actively running task and publishing task. The listed actively running task’s remainingSeconds field value is 0, but not able to close by itself. And the listed publishing task which got created two days back, whose remainingSeconds field value is also 0, is still getting listed in publishing tasks.
Thinking that old publishing task is not allowing the existing active task to get completed, i tried to forcefully kill this publishing task. But I was getting Task does not exist error.
When checked in Overlord logs, frequent exceptions are being thrown for the respective supervisor telling that some data is missing.
But after restarting the supervisor, tasks are getting into normal state.
This is happening once in a while. Not able to find the cause also.
Please someone can tell that why we are running into this problem once in a while.
Regards,
Roopini
