For CentOS-based systems:<\/p>\n\n\n\n

sudo yum update -y\nsudo yum install -y curl unzip<\/pre>\n\n\n\n
Option 1. Installing via script<\/h3>\n\n\n\n
You can install rclone quickly and easily using their handy install script that works with most Unix operating systems:<\/p>\n\n\n\n
sudo curl https:\/\/rclone.org\/install.sh | sudo bash<\/pre>\n\n\n\n
Note that blindly executing scripts from the internet is not ideal and you need to be able to trust the script provider. If you\u2019re not sure about what you\u2019re running here, we recommend double-checking the install.sh<\/em> file before executing it.<\/p>\n\n\n\n
Option 2. Installing manually<\/h3>\n\n\n\n
Alternatively, you can install rclone manually by downloading the precompiled binary.<\/p>\n\n\n\n
First, download the binary package and unzip it to a convenient location. Then change into the resulting new directory.<\/p>\n\n\n\n
curl -O https:\/\/downloads.rclone.org\/rclone-current-linux-amd64.zip\nunzip rclone-current-linux-amd64.zip\ncd rclone-*-linux-amd64<\/pre>\n\n\n\n
Next, copy the binary file to the relevant location and set the permissions accordingly.<\/p>\n\n\n\n
sudo cp rclone \/usr\/bin\/\nsudo chown root:root \/usr\/bin\/rclone\nsudo chmod 755 \/usr\/bin\/rclone<\/pre>\n\n\n\n
Optionally, also can also install the manpage.<\/p>\n\n\n\n
sudo mkdir -p \/usr\/local\/share\/man\/man1\nsudo cp rclone.1 \/usr\/local\/share\/man\/man1\/\nsudo mandb<\/pre>\n\n\n\n
Once you\u2019ve got rclone installed, continue below with configuring the migration source and target.<\/p>\n\n\n\n
Configuring object storage connections<\/h2>\n\n\n\n
Rclone offers a simple and easy-to-read configuration file which we\u2019ll use to add our old and new Object Storage remotes, which rclone calls remote storage devices.<\/p>\n\n\n\n
Next, check the rclone.conf file location with the following command:<\/p>\n\n\n\n
rclone config file<\/pre>\n\n\n\n
Configuration file is stored at:\n\/root\/.config\/rclone\/rclone.conf<\/pre>\n\n\n\n
Then open the new file in a text editor. Note that the location of your configuration file may differ. Check the output of the previous command.<\/p>\n\n\n\n
nano \/root\/.config\/rclone\/rclone.conf<\/pre>\n\n\n\n
Enter the following example configuration and replace the details highlighted in red<\/span>. For example, <REGION> would be fi-hel2<\/tt> if your Object Storage is deployed in FI-HEL2.<\/p>\n\n\n\n
[OLD-OBJECTSTORAGE]\ntype = s3\nprovider = Other\nenv_auth = false\naccess_key_id = <ACCESS_KEY><\/span>\nsecret_access_key = <SECRET_KEY><\/span>\nregion = <REGION><\/span>\nendpoint = <ENDPOINT><\/span>\nlocation_constraint = <REGION><\/span>\nacl = private\n\n[NEW-OBJECTSTORAGE]\ntype = s3\nprovider = Other\nenv_auth = false\naccess_key_id = <ACCESS_KEY><\/span>\nsecret_access_key = <SECRET_KEY><\/span>\nregion = <REGION><\/span>\nendpoint = <ENDPOINT><\/span>\nlocation_constraint = <REGION><\/span>\nacl = private<\/pre>\n\n\n\n
You will need to set the following for both old and new Object Storage in the config file:<\/p>\n\n\n\n
<ACCESS_KEY>:<\/tt> This should be your \"access_key\" for this Object Storage instance\n<SECRET_KEY>:<\/tt> This should be your \"secret_key\" for this Object Storage instance\n<REGION>:<\/tt> This is your region in lowercase<\/strong>, so for FI-HEL2 you'd use fi-hel2\n<ENDPOINT>:<\/tt> Your Object Storage endpoint URL, available from the \"S3 API Access\" link<\/pre>\n\n\n\n
The Access and Secret keys would have been presented to you when you created the Object Storages.
<\/p>\n\n\n\n
If you did not save the keys at the time when you created the Object Storage, you can regenerate them by clicking the Settings<\/em> button at your Object Storage details.
<\/p>\n\n\n\n
Then scrolling down to the bottom of the Settings page where you\u2019ll find the Regenerate<\/em> button. After new Access and Secret keys have been generated, download the key file and store it somewhere safe.<\/p>\n\n\n\n
<\/figure>\n\n\n\n
Note that the format of the region and endpoint will differ depending on which provider you are migrating from. If you are unsure, check your provider\u2019s documentation for S3cmd configuration instructions on where to find these details.<\/p>\n\n\n\n
Confirm the 2 new remotes are available to rclone by running the configuration check.<\/p>\n\n\n\n
rclone config<\/pre>\n\n\n\n
If you see the following, you\u2019re all set:<\/p>\n\n\n\n
Name                 Type\n====                 ====\nNEW-OBJECTSTORAGE    s3\nOLD-OBJECTSTORAGE    s3<\/pre>\n\n\n\n
The last step then is to initiate the migration. Continue in the next section on how to run rclone.<\/p>\n\n\n\n
Migrating object storage data<\/h2>\n\n\n\n
To migrate your files between Object Storage instances we need a simple, one-line command:<\/p>\n\n\n\n
rclone copy --progress OLD-OBJECTSTORAGE:\/ NEW-OBJECTSTORAGE:\/<\/pre>\n\n\n\n
This will result in all data from the old Object Storage instance to be copied to the new one. You will see the progress thanks to --progress<\/tt> and once complete with no errors, you can confirm that all of your data has moved across successfully, leaving you free to delete your old object storage instance.<\/p>\n\n\n\n
Note that depending on the amount of data and the network between the locations, the migration might take some time.<\/p>\n\n\n\n
Running rclone in screen<\/h3>\n\n\n\n
Luckily, rclone is able to continue the operation in case the copy process is interrupted. But you may wish to set up a screen to run the process which allows you to close any SSH connections to the copying server without cancelling the process.<\/p>\n\n\n\n
First, check that screen is installed or install it if missing.<\/p>\n\n\n\n
# Debian and Ubuntu\nsudo apt install screen -y<\/pre>\n\n\n\n
# CentOS\nsudo yum install screen -y<\/pre>\n\n\n\n
Next, open a new screen with the following command.<\/p>\n\n\n\n
screen -U<\/pre>\n\n\n\n
Then run the migration in the new terminal screen.<\/p>\n\n\n\n
rclone copy --progress OLD-OBJECTSTORAGE:\/ NEW-OBJECTSTORAGE:\/<\/pre>\n\n\n\n
This way, you can leave the terminal by detaching the screen with ctrl+D and the copy operation won\u2019t be interrupted. Additionally, you can always resume a detached screen by using the following command.<\/p>\n\n\n\n
screen -Urd<\/pre>\n\n\n\n
That\u2019s it! You now have everything in place to migrate data between any S3-compliant object storages.<\/p>\n","protected":false},"featured_media":19364,"comment_status":"open","ping_status":"closed","template":"","community-category":[127,128],"class_list":["post-24919","tutorial","type-tutorial","status-publish","has-post-thumbnail","hentry","community-category-managed-services","community-category-object-storage"],"acf":[],"_links":{"self":[{"href":"https:\/\/studiogo.tech\/upcloudold\/wp-json\/wp\/v2\/tutorial\/24919","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/studiogo.tech\/upcloudold\/wp-json\/wp\/v2\/tutorial"}],"about":[{"href":"https:\/\/studiogo.tech\/upcloudold\/wp-json\/wp\/v2\/types\/tutorial"}],"replies":[{"embeddable":true,"href":"https:\/\/studiogo.tech\/upcloudold\/wp-json\/wp\/v2\/comments?post=24919"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/studiogo.tech\/upcloudold\/wp-json\/wp\/v2\/media\/19364"}],"wp:attachment":[{"href":"https:\/\/studiogo.tech\/upcloudold\/wp-json\/wp\/v2\/media?parent=24919"}],"wp:term":[{"taxonomy":"community-category","embeddable":true,"href":"https:\/\/studiogo.tech\/upcloudold\/wp-json\/wp\/v2\/community-category?post=24919"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}