Is Stream data type is "Real Time" , it will be store into Deep Storage?

Hi Team,

I am using tranquility service to fetch data from Kinesis then push to Druid via Tranquility Server. It seems working , however, if restart or change to a new node, the stream data is stored to Deep storage ( the data type is index , it is working). For example, after restart server I could see batch date is not change in Druid , however, them stream data using real time type is seems reset back to 0 and no historical data. How can I
solve this issue?

Thanks,

Tom

Hi Tom,

If I understand your question correctly, you’re asking that the realtime task type also stores data in deep storage. Then, yes, it stores data in deep storage.

however, them stream data using real time type is seems reset back to 0 and no historical data.

I’m not sure what you mean by this question. Would you please give more details?

Jihoon

Hi Jihoon,

Thanks for replied to me, I am ingest data in both (batch and streaming), the streaming via Post stream data via Tranquility Server.

I could see the Stream data and Batch data went all the way to with success indexed. However , there are two things I am seeing.

  1. I don’t see Stream data is stored in Deep Storage (S3) only the Batch data

  2. When I restarted any services or any server issues , the stream data is gone , only Batch recovered back nicely.

Thanks,

Tom