We are calculating Daily data from the Hourly data source.
But instead of aggregating the data for all 24 hour into a single row, it is distributing the data across multiple segments created for Daily data source.
So when i am firing a select query on daily data source i am receiving 3 rows instead of 1, as data is distributed across 3 segments.
Is there any way of getting the desired result without exceeding the segment size limit.
Please help, Thanks in Advance.
I guess it might be due to difference in two roll-up modes :
best-effort (default) - bit faster rollup but there is chance for records to be in different segments which are supposed to be in same segment
perfect - bit slower but records will be in same segment which are supposed to be in same segment.
Did you try enabling perfect rollup?
How are you ingesting it? Stream or batch? If batch, does the ingestion run once a day? Also, are you saying for the same set of dimensions, you have multiple rows each with different metric values?
You may want to lookup segment compaction jobs.