Varying query and segment granularity within a data source

Hi! My understanding is that it’s possible to vary query and segment granularity within a data source. For example, you could have real-time Kafka ingestion running with queryGranularity NONE and segmentGranularity HOUR, but also periodically use native batch ingestion to reingest older data (with ingestSegment firehose, say) at queryGranularity MINUTE and segmentGranularity DAY. Does this setup work? I don’t see anything in the docs or code that says you can’t do this, but nothing that explicitly says you can.


Hi Dave, I believe you can. Basically setting queryGranularity=NONE is same as setting it to milliseconds, and it makes sense to re-ingest with coarser granularities.

That makes sense. And all timestamps are rounded down at ingestion to the segment’s queryGranularity, so queries that aren’t aligned with the segment’s queryGranularity will see either all the data in the queryGranularity “bucket” or none, depending on whether or not the query covers the opening millisecond, right?