Hi Druid Users,
I have the following situation:
Druid nodes were working properly for some realtime space until my Kafka cluster crashed. Then one of the realtimes escalated in terms of cpu utilization. I managed to revive Kafka cluster, but what happened was following:
- the affected realtime ceased to stabilize (cpu utilization has stayed on very high level)
- a substantial number of tasks were produced in /tmp/persistent/task/ of overlord
- those tasks only accumulated until I was forced to remove the oldest ones to have a server operative
- I restarted the realtime, after removing all pertinent merged_* tasks
But I ended up in the same situation of ever increasing, not really finalized, merged_* tasks. Pushing segment is only partial as a consequence
What is the recommended procedure to restart realtime, clean Druid in order to startover without abovementioned burden?
How to prop Druid to stop merging tasks for a given stream?
What is a complete procedure to clean after terminating a given stream? What should be done?
Thank you very much in advance for all hints and answers,