Downsampling with fraction during the ingestion


We have a huge amount of data flowing in every day and cannot store all the data in raw format, where we wanted to apply downsampling to reduce the number of data points. In druid, we can ingest data by specifying formula for that column, so that it will aggregate and store the data in such format. Similar way Is downsampling with some fraction is possible, during the ingestion or after the ingestion ?


Here, It’s a time-series data with 100ms granularity, we wanted to reduce to 1min. can pick any datapoint with the 1 min batch.


You can use roll-ups and aggregate your data at ingestion time. There’s no formula needed. Druid will perform roll-up based on time interval. In this case per minute.