Generating a user table with pre-processed atributes

Hello there!

We’ve been successfully using Druid for some months and now we want to extend its capabilities.

We currently have an event database which is very similar to the one that comes by default on the druid installation:

timestamp publisher advertiser gender country click price

2011-01-01T01:01:35Z bieberfever.com google.com Male USA 0 0.65

2011-01-01T01:03:63Z bieberfever.com google.com Male USA 0 0.62

2011-01-01T01:04:51Z bieberfever.com google.com Male USA 1 0.45

2011-01-01T01:00:00Z ultratrimfast.com google.com Female UK 0 0.87

2011-01-01T02:00:00Z ultratrimfast.com google.com Female UK 0 0.99

2011-01-01T02:00:00Z ultratrimfast.com google.com Female UK 1 1.53

With the difference that every row has a unique user identifier.

What we want to do is to aggregate data for every unique user, and present that information inside another table, we like to call it “User Table”.

The main goal is to have 1 row = 1 user, with many columns that describes how is that user. Also some of those columns need to be multi-value colums.

Why do we need that? Because there are some queries that we are not able to execute with the given dataset format. We need to re-format it in a user table.

How can we achieve that? We need to use Druid outstanding performance on their indexes, but the problem is that the format is timestamp oriented. We would like to update users, pretty much something like “update” statement on SQL databases.

Any feedback or comment would be helpful, thanks a lot for your time!