Realtime node not deleting segment even after handing off to Historical node

I am ingesting data in Real-time node via Kafka Firehose. After handing off the segment to historical node ( verified via logs and coordinator console), realtime node doesn’t delete some segments from the local tmp folder. Some segments are deleted after a lot of time ( after 9-10 hours).

  1. What might be the cause for the delay?

  2. Is there any way to force the deletion of such segments ?

  3. How are tasks like handing-off and deletion of segments scheduled?

I think you can change realtime.spec:windowPeriod.

在 2015年9月16日星期三 UTC+8上午4:47:45,Saksham Garg写道:

I have set ‘windowPeriod’ to 20m.
I don’t think window period is the problem.

Hi Saksham, are you sure the handoff has completed? As in, you see the segment being served by your historicals? Are there any exceptions or otherwise strange behaviour in your logs around the time segments should be removed?