Tranquility with kafka

Hello All,

I am s3 as Deep storage. I am able to store segments on s3 success fully if I try to ingest data using batch processing. But can’t see any data on s3 if I use tranquility with Kafka.

Looks I am missing some parameters. Please see the attached kafka.json.


kafka.json (2.43 KB)

I have also attached the reamtime task log.

realtime_task.log (68.4 KB)

Realtime index tasks publishes the segment to Deep storage after segmentGranularity has passed.

from your task logs it seems you have set the segmentGranularity as DAY so they will be handed over to the historical after a DAY.
Do you see handoff not happening even after that ?

Hello Nishant,

Thanks for pointing this. I can see data now on s3.