Mixing partitionSpecs when batch ingesting and delta ingesting

I have been using druid-0.8.1-rc2

I have been batching files from Camus, and then using the delta ingestion to batch in new files.

It runs much faster. Nice!

So I have a question, what happens if you originally batch ingested files using one partitionSpec, say with “type” : “dimension” and then you later delta ingested with “type”:“hashed”.

What are the implications of changing the partitionSpec for part of the segment?


Hmm, I think Himanshu may have to take this one.

Hey Johnny,

Delta ingestion will read rows from segments just like any other rows, and will make a whole new set of output segments. Think of it as a way of mixing in some Druid data with some raw data to create new Druid data. So in your case, after running a delta ingestion you’ll just have a “hashed” partitioned segment set.