Moving deep storage segments from one HDFS cluster to another

Hi,
I read multiple threads on moving Druid envt (including deep storage and metadata store) but in my case, everything remains same but deep storage. I have a new cluster which is more powerful and i want to move my deep storage segments to new hadoop cluster. Can i do:

  1. Stop ALL the services (I’m fine with downtime)

  2. Distcp all the segments (keep folder structure consistent with old cluster)

  3. Restart all the services

Will this work? In my case, I provide hadoop xml config files in path while starting the services and once I move to new cluster, i will provide updated xml configs. Do i need to update loadSpec in metadata store? Anyone tried this?

Thanks,

-Suresh

Hi Suresh,

There is a cli tool to update metadata store. http://druid.io/docs/latest/operations/insert-segment-to-db.html.

moving/copying old data requires:

  1. Moving the actual segments.
  2. Copying/updating the metadata in the segments table, you can use http://druid.io/docs/latest/operations/insert-segment-to-db.html to update/copy metadata in segments table.

Thanks,
Akash