Append Data without replacing or doubling the values

I have 3 different data but with the same timestamp. uploading them by using appendtoexisting:false replaces the existing one with the new one. Or by using appendtoexisting:true sums the exact same values. I just want it to be updated not added.

Welcome @Berkay_Ertem! Do you have rollup enabled?

Hi Berkay_Ertem,

This is expected behavior in Druid. The segments are designed to be immutable once written and cached in the historicals. You can either add a new segment using append OR overwrite the old segment. Both of which you have tried out.

If you want traditional UPSERT type functionality, you will have to do this outside of Druid in your ETL pipeline and then reingest using overwrite. The source of truth in this case can either be the old data in Druid or taken from the original source.

2 Likes

See this doc on UPDATE options with Druid Data management · Apache Druid