I took a brief look at
to see if it would fit your use case and I don’t think it will because the mover assumes you’re not going across loadSpec types. (aka, you’re staying within S3, HDFS, cassandra, azure, etc… not moving from one type to another)
As such it would require development effort to get a task which does proper locking, copying locally then remotely, updating segment metadata, and verification.
I’m curious if there’s a way to get a hadoop task to do this as part of distcp or similar. I’ll try and ping one of the other devs about it.