Right now, am in POC stage and have a single node running all the 5 services (historical, broker, coordinator, overlord, middle-manager) and this node is running on UTC Timezone.
I have a requirement wherein the data ingested should be in IST timezone for retrieving reports in an easier manner.
For example: To get all the sales data for a particular day, if this value is converted to an IST before the ingestion (By adding 5.5 hours to the timestamp), it will be easier for the users to just select a calendar date and get the desired results. If this 5.5 hour is not added, it will be required for the user to select date along with 18:30 of day 1 and next date along with 18:30 of day 2.
Whenever I try to ingest the IST converted data in batch mode, it’ll result in me trying to ingest future data (Since IST is 5.5 hours ahead of UTC) and the ingestion shows success message but the query will not result in any results (as the result is ahead in time). If I don’t the offset, the data ingestion is successful, but the user experience of selecting 18.30 is not going to be good.
Also, if I get into the streaming mode, the data with a future timestamp will be dropped (If I do the offset of 5.5 hours). What needs to be done in such situations?
Thanks In Advance,