Updaring events in druid

I have an use case where I need to update an event already ingested in druid to set new values to a specific dimensions. is it possible? and what is the best way to do it?

Thank you for the help.

Hi Jimena,

This page might be helpful: http://druid.io/docs/latest/ingestion/update-existing-data.html

Thank you for your response Gian,
I already reviewed the druid documentation and I’m little worried about if druid really can help me, I have the next situations:

I need to manage a lot of events and I want to implement a real time ingestion strategy, when the events are ingested in druid they have an initial data(state), but after the events are ingested in druid, the data(some dimensions values) associated with the events can change, even twice, three or more times.

In the initial design I have discovered that I would have to create several datasources for the reports I want to generate, and in each datasource I need to replicate some data of the events(because druid does not support joins), the problem is when the events data change, I would need to rebuild(reindex) several segments in different data sources, even, I would have to know in some way, which segments in each datasouce, contain exactly the data I want to change and I’m not sure if that is possible, for example, first, how to know in which segments are the data that match with specific search criteria?, how to rebuild the raw data contained in those specific segments?(because it will be a real time ingestion so I dont keep a copy of the data, and that would be too expensive for my case).

After that, I would have to create a way to change the specific data and reindexing and it would become a very frequent task that would overload the cluster and I think it would affect performance.

I appreciate if you can give me some recommendations.

The lookups feature may be useful here: http://druid.io/docs/latest/querying/lookups.html

  • Jon