Updating dimension values

Hi there!

I had a question about updating values in Druid. I just started using Druid about a week ago and after experimenting with some sample data sets I’ve been trying to look in to using lookups, but I’m not sure if they can do the type of thing that I’m looking for.

In my data, we have some dimensions that are inferences about a user’s attributes, but these inferences could change as we gather more information about the user. For example, we may be assuming that a user is single (stored as “1”), but then gain information that changes our assumption to be married (“2”). There are many rows in the table stored under the same userID to keep track of all the different inferences about that user. Essentially what we want to be able to do (in SQL) is: UPDATE my_table SET attribute=‘2’ WHERE userID=‘12345’ AND attribute=‘1’;

Is there a way to do this using lookups? If not, would there be a way to delete rows for this user and then add back rows with the updated attributes? Or would we have to completely reindex our data to do this?

Thanks!

Ryan

Any ideas or feedback would be much appreciated!

http://druid.io/docs/latest/ingestion/update-existing-data.html I think this is what you need.

Hi Ananth, thanks for the response.

I had looked at that page, but where I was unclear was on whether or not I could use lookups. Are you suggesting that this is a situation where lookups are not sufficient so I need to look into reindexing?

Thanks,

Ryan