When irrelevant segments are removed from deep storage?

Hello,

I’m seeing that irrelevant segments, like:

  • ones created by real-time pipeline that were replaced by batch indexing process

  • ones created by batch indexer that were replaced by re-indexing process

are still present in the deep storage (s3 in our case).

Is there any process that removes them?

Sorry for flooding this forum with questions, but I think Druid is really cool, and I’m eager to learn to use it properly.

Thanks!

Michael

There is no automated process that removes them. You can invoke the
"kill task" at any time from the overlord though. You can read more
about it at

http://druid.io/docs/latest/misc/tasks.html

--Eric

Hello Eric,

Where do I find the list of tasks and respective unused segments?

Thanks!

Michael

Nevermind, I’ve figured out already.

you might be interested in the PR Himanshu created for periodic deletion of segments- https://github.com/druid-io/druid/pull/1990