Migrating data from one cluster to another

I’m wondering if it is possible to migrate segments to a different environment’s druid cluster by simply snapshotting the postgres database?

This can be easily done I am more worried about how agnostic the metadata is in regard to what coordinator it came from? Are there any foreseeable issues?

Alternatives are welcome :slight_smile:

Okay for any of those who find themselves in a similar situation the easiest way to migrate the segment data is not the tool, but to just dump the segments table and restore to the new cluster’s database:

1 #!/bin/bash

2

3

4 export DATE=date +"%m-%d-%y"

5

6 pg_dump --host druid-db.mydomain.com --port 5432 --username druid --format plain --verbose --file “./druid_segments-table-${DATE}.dump” --table public.druid_segments --data-only druid

7

``

Notice that in the dump I use --data-only flag so it does not try to recreate the table, etc.

Then restore:

psql -h new-druid-cluster.mydb.com -p 5432 -U druid -W < druid_segments-table-09-15-17.dump

``

After this I restart the coordinator, and wait 3-5 minutes you should see it loading on your coordinator and into your historicals. Be sure your disk space is enough to hold your dataset form the initial cluster.

Is this migration dependent on the version of Druid? Is it possible to move from 0.9.x to 0.11.0 using this approach?

Two weeks ago I migrated 1TB of druid segments with the insert-segment-to-db tool http://druid.io/docs/latest/operations/insert-segment-to-db.html

I just copied the segments from one cluster deep storage to the new, and told the new cluster to scan that folder.