Temp folder size was increasing due to that Peons Processing taking more time. How to clear temp folder automatically?

Relates to Apache Druid

Temporary files created during ingestion will be removed once the segment that it produces has been published. If you have longer running tasks that are consuming larger volumes, you may need to increase the available space on temporary folder.
If you haven’t already read this, I think you’ll find it useful:

I have verified that When Middle manager launches the Peons , creating a folder with naming pattern like druid-realtime-persistXXXXX and some files. Files were deleted but folders were not deleted.

Nearly above 2 lakhs folders are exist with empty data.

Please suggest is there any missing part from our configurations or else is this any version upgrade required to close the observation.

Historical configured with high memory settings and temp folder also has enough space.

We started to run a script which will delete temp folders based configurable older than x days.

Thank you.

Hi @Punnaiah_Guptha . Just wanted to let you know that I was able to reproduce this in 0.22.1. We’ve created a bug report here: Temporary working folders are left behind on Middle Managers after tasks complete · Issue #12332 · apache/druid · GitHub

1 Like