Update existing data (Adding a new dimension)


Could someone please help me the steps for updating existing data in Druid. I have a requirement to add a **new dimension to the existing data in Druid (the new dimension to be added will have value which will be obtained by transforming one of the existing dimension value in the data). **

I see that we have an option reading segments from Druid and re-indexing it using https://druid.apache.org/docs/0.14.2-incubating/ingestion/firehose.html#ingestsegmentfirehose. Will this option allow me to achieve the goal of adding additional column by transforming exisitng dimension value? Could someone please share a sample ingest segment firehose spec?


Vinay Patil

Hi Vinay,
Sample ingest segment firehose

I guess you can try to create a new datasource by specifying the existing datasource in the ioconfig. Something like this-

“ioConfig” : {
“type” : “index”,
“firehose” : {
“type”: “combining”,
“delegates”: [
“type” : “ingestSegment”,
“dataSource” : “original_datasource”
“appendToExisting” : false

And perhaps you can add post transforms to add the metrics you wanted to add? I haven’t given this a shot but I’d be interested to know if it works.