If you are looking to migrate to UpCloud or want to use your own media, get started quickly using Storage Import. The Storage Import API allows you to easily import installation media or even entire virtual machine images.<\/p>\n\n\n\n

Migrate servers from on-premise or any other cloud quickly and easily by simply creating new storage out of your server image and deploying it to a cloud server.<\/p>\n\n\n\n

Test hosting on UpCloud!<\/a><\/div>\n<\/div>\n\n\n\n

How Storage Import Works<\/h2>\n\n\n\n
Storage Import creates a new installation media or server storage out of any system or server image available online by fetching the requested file and uploading it onto a new storage device. It can be useful for migrating existing server images from on-premise hosting or from other cloud providers.<\/p>\n\n\n\n
Storage Import is also handy for using custom installation media if your favourite Linux distribution is not yet available as a public template or ready-made install disk.<\/p>\n\n\n\n
Supported file types:<\/p>\n\n\n\n
\n
RAW storage images<\/li>\n\n\n\n
IMG storage images<\/li>\n\n\n\n
ISO archive files<\/li>\n\n\n\n
GZIP compressed files<\/li>\n\n\n\n
XZ compressed files<\/li>\n<\/ul>\n\n\n\n
The target storage needs to be free and not in operation, either;<\/p>\n\n\n\n
\n
detached from any cloud server<\/li>\n\n\n\n
if attached, the cloud server needs to be shutdown<\/li>\n\n\n\n
not in a maintenance state<\/li>\n<\/ul>\n\n\n\n
Note that the system image or installation media you wish to upload needs to be made available for download for Storage Import to be able to retrieve it. The process supports the use of HTTPS and Basic Auth for secure upload.<\/p>\n\n\n\n
Setting password protection using Basic Auth<\/h2>\n\n\n\n
We recommend using Basic Auth password protection when preparing files and images for upload. This needs to be done on the webserver that is hosting your files and the steps will differ depending on the operating system and web server in use. Here we\u2019ve outlined some of the options you have for setting password protection on files and folders on your web server.<\/p>\n\n\n\n
Start by installing Apache2 utils. The utils package includes a password generator tool which we\u2019ll be using to encode the credentials.<\/p>\n\n\n\n
sudo apt install apache2-utils<\/pre>\n\n\n\n
Afterwards, continue below with the instructions applicable to your web server.<\/p>\n\n\n\n
Apache2<\/h2>\n\n\n\n
Generate a password file using the `htpasswd<\/tt> command as shown below. Replace the username<\/span> with whatever you want to use. On running the command, you will be asked to set the password, and enter it twice when prompted.<\/p>\n\n\n\n`
sudo htpasswd -c \/etc\/apache2\/.htpasswd username<\/span><\/pre>\n\n\n\nNew password:\nRe-type new password:\nAdding password for user username<\/pre>\n\n\n\nNext, edit your site configuration. The file name will differ depending on your set up but it\u2019s generally stored in \/etc\/apache2\/sites-available\/<\/tt> and ends in .conf<\/tt> file type notation.<\/p>\n\n\n\n sudo nano \/etc\/apatche2\/sites-available\/example.conf<\/span><\/pre>\n\n\n\nOnce you\u2019ve opened the file for editing, add the following authentication parameters as shown below if not already present. You may need to add the <Directory \u201c\/path\/to\/html\u201d><\/em> segment if your configuration does not contain one. You can set the path to your webroot to enable authentication site-wide.<\/p>\n\n\n\n VirtualHost :80>\n ...\n <Directory \"\/var\/www\/html\">\n AuthName \"Restricted\"\n AuthType Basic\n AuthBasicProvider file\n AuthUserFile \"\/etc\/apache2\/.htpasswd\"\n Require valid-user\n ...\n <\/Directory><\/pre>\n\n\n\nAlternatively, if you want to only enable authentication for a specific folder, set the directory with the path to the folder containing your private files. Otherwise include the same parameters.<\/p>\n\n\n\n VirtualHost :80>\n ...\n <Directory \"\/var\/www\/html\/files<\/span>\"><\/pre>\n\n\n\nWhen done, save the file and exit the editor.<\/p>\n\n\n\n Then reload the webserver.<\/p>\n\n\n\n sudo systemctl reload apatche2<\/pre>\n\n\n\nDone! You should now be prompted for credentials when browsing to the password-protected URL.<\/p>\n\n\n\n Nginx<\/h3>\n\n\n\nWe\u2019ll use the htpasswd<\/em> installed above also with nginx web servers. Run the following command to generate a password file. Replace the username<\/span> with whatever you wish. When\u00a0running the command, you will be asked to set the password and enter it twice when prompted.<\/p>\n\n\n\n sudo htpasswd -c \/etc\/nginx\/.htpasswd username<\/span><\/pre>\n\n\n\nNew password:\nRe-type new password:\nAdding password for user username<\/pre>\n\n\n\nOnce you\u2019ve generated the password file, you\u2019ll need to set it in your web server configuration file. The file name will differ depending on your website but it\u2019s generally stored in \/etc\/nginx\/sites-available\/<\/tt> and ends in .conf<\/tt> file type notation.<\/p>\n\n\n\n sudo nano \/etc\/nginx\/sites-available\/example.conf<\/span><\/pre>\n\n\n\nWith the website configuration open for edit, set enables authentication by adding the two parameters as shown below. Adding these two lines directly to the server section will enable password protection for the whole site.<\/p>\n\n\n\n server {\n ...\n auth_basic \"Restricted access\";\n auth_basic_user_file \/etc\/nginx\/.htpasswd;\n ...\n<\/pre>\n\n\n\nAlternatively, you can include the authentication parameters within a location segment to set the password for a specific directory.<\/p>\n\n\n\n server {\n ...\n location \/files<\/span> {\n auth_basic \"Restricted access\";\n auth_basic_user_file \/etc\/nginx\/.htpasswd;\n }\n ...\n<\/pre>\n\n\n\nOnce done, save the file and exit the editor.<\/p>\n\n\n\n You can test that the syntax is correct before applying the changes by using the next command.<\/p>\n\n\n\n sudo nginx -t<\/pre>\n\n\n\nnginx: the configuration file \/etc\/nginx\/nginx.conf syntax is ok\nnginx: configuration file \/etc\/nginx\/nginx.conf test is successful<\/pre>\n\n\n\nIf you get a confirmation such as the one shown above, you are ready to enable the new configuration by reloading the nginx service.<\/p>\n\n\n\n sudo systemctl reload nginx<\/pre>\n\n\n\nThat\u2019s it! Your files are now set to require authentication to access and be safe from prying eyes.<\/p>\n\n\n\n Accessing password-protected files<\/h3>\n\n\n\nWhen fetching password-protected files, add your credentials at the beginning of the URL separated from the rest by an @ sign.<\/p>\n\n\n\n https:\/\/username:password@<\/span>example.com\/files\/system.img<\/pre>\n\n\n\nStorage Import API should then be able to access your target file and begin uploading it.<\/p>\n\n\n\n Create new storage for import<\/h2>\n\n\n\nThe UpCloud API allows users to run programmable requests to manage and operate their cloud resources. It\u2019s a quick way to get started by using the Storage Import API with just a couple of commands instead of going through the steps in the control panel.<\/p>\n\n\n\n If you are not yet familiar with the UpCloud API, learn more in our getting started tutorial<\/a>.<\/p>\n\n\n\n Create new storage for import<\/h3>\n\n\n\nFirst, you will need a storage device of the same size or larger than the file you want to import. If you do not already have free storage, create a new one with the following request.<\/p>\n\n\n\nPOST \/1.3\/storage\/<\/pre>\n\n\n\n{\n \"storage\": {\n \"size\": \"10\",\n \"tier\": \"maxiops\",\n \"title\": \"storage-import\",\n \"zone\": \"fi-hel1\"\n }\n}<\/pre>\n\n\n\nWhen done, you\u2019ll see a response similar to the example below. Note down your storage UUID as you will need it for the other API operations.<\/p>\n\n\n\n{\n \"storage\": {\n \"access\": \"private\",\n \"backup_rule\": {},\n \"backups\": {\n \"backup\": []\n },\n \"license\": 0,\n \"servers\": {\n \"server\": []\n },\n \"size\": 10,\n \"state\": \"online\",\n \"tier\": \"maxiops\",\n \"title\": \"import-test-ubuntu-20-04\",\n \"type\": \"normal\",\n \"uuid\": \"0108427c-aff8-4ce9-9897-849fb89feae0<\/span>\",\n \"zone\": \"fi-hel1\"\n }\n}<\/pre>\n\n\n\nOnce you have a suitable storage device available, continue with one of the two import methods below.<\/p>\n\n\n\nCreate Storage Import<\/h2>\n\n\n\nThe Storage Import API offers two methods for importing data: HTTP import<\/strong> or direct upload<\/strong>. Depending on your file size and where it\u2019s stored, you may wish to use one over the other.<\/p>\n\n\n\nIt\u2019s also possible to use Storage Import via your UpCloud Control Panel<\/a>.<\/p>\n\n\n\n HTTP Import<\/strong> is used to upload files made available on a web server. This is often more convenient for uploading larger files such as storage images between cloud servers. It can also be used to upload installation media directly from the OS provider\u2019s download link.<\/p>\n\n\n\n Direct upload<\/strong> allows uploading straight from your local computer without the need for setting up a web server. This requires the use of a tool such as curl<\/tt> to run the upload and which needs to remain open until the import has finished.<\/p>\n\n\n\n HTTP Import<\/h3>\n\n\n\nWhen you have a storage device ready for import, begin the upload by sending the following request. Set the UUID of your target storage device in the request and the URL of the file you want to import in the body text as shown in the example below.<\/p>\n\n\n\nPOST \/1.3\/storage\/{uuid}<\/span>\/import<\/pre>\n\n\n\n{\n \"storage_import\": {\n \"source\": \"http_import\",\n \"source_location\": \"https:\/\/user:password@example.com\/files\/system.img<\/span>\"\n }\n}<\/pre>\n\n\n\nOn successful request, you\u2019ll see a response similar to the following example. The initial state of the upload will show preparing or prepared.<\/p>\n\n\n\n{\n \"storage_import\": {\n \"client_content_length\": 0,\n \"client_content_type\": \"\",\n \"completed\": \"\",\n \"created\": \"2020-06-23T19:13:15Z\",\n \"error_code\": \"\",\n \"error_message\": \"\",\n \"md5sum\": \"\",\n \"read_bytes\": 0,\n \"sha256sum\": \"\",\n \"source\": \"http_import\",\n \"source_location\": \"https:\/\/[REDACTED]@example.com\/files\/system.img\",\n \"state\": \"prepared\"<\/span>,\n \"uuid\": \"0747a8c1-3a34-4908-925e-9ef9a369af51\",\n \"written_bytes\": 0\n }\n}<\/pre>\n\n\n\nThe import operation will then commence granted the URL you submitted is accessible.<\/p>\n\n\n\nDirect upload<\/h3>\n\n\n\nTo begin direct upload, use the following request to create the import task. Set the UUID of your target storage device in the request and the source as direct_upload<\/tt> like in the example underneath.<\/p>\n\n\n\nPOST \/1.3\/storage\/{uuid}<\/span>\/import<\/pre>\n\n\n\n{\n \"storage_import\": {\n \"source\": \"direct_upload\"\n }\n}<\/pre>\n\n\n\nYou will then see a response such as an example output below. The response includes the direct_upload_url<\/em> that can be used to start the upload itself.<\/p>\n\n\n\n{\n \"storage_import\": {\n \"client_content_length\": 0,\n \"client_content_type\": \"\",\n \"completed\": \"\",\n \"created\": \"2020-06-26T16:06:59Z\",\n \"error_code\": \"\",\n \"error_message\": \"\",\n \"md5sum\": \"\",\n \"read_bytes\": 0,\n \"sha256sum\": \"\",\n \"source\": \"direct_upload\",\n \"direct_upload_url\": \"https:\/\/fi-hel1.img.upcloud.com\/uploader\/session\/0747a8c1-3a34-4908-925e-9ef9a369af51<\/span>\",\n \"state\": \"prepared\",\n \"uuid\": \"0747a8c1-3a34-4908-925e-9ef9a369af51\",\n \"written_bytes\": 0\n }\n}<\/pre>\n\n\n\nYou can then import files straight from your local computer by using the direct upload method. Note that the link will expire after 10 minutes if the direct upload is not started.<\/p>\n\n\n\ncurl --data-binary @\/path\/to\/files\/system.img --fail -XPUT https:\/\/fi-hel1.img.upcloud.com\/uploader\/session\/0747a8c1-3a34-4908-925e-9ef9a369af51<\/pre>\n\n\n\nWhen using something other than raw images (for example, a gzipped image), you need to also supply the Content-Type header.<\/p>\n\n\n\n curl -H 'Content-Type: application\/gzip' --data-binary @\/path\/to\/files\/system.img.gz --fail -XPUT https:\/\/fi-hel1.img.upcloud.com\/uploader\/session\/0747a8c1-3a34-4908-925e-9ef9a369af51<\/pre>\n\n\n\nOnce the upload has finished, you\u2019ll see an output similar to the example underneath.<\/p>\n\n\n\n {\n \"written_bytes\":9521070080,\n \"md5sum\":\"f03d31c11136e24c10c705b7b3efc39f\",\n \"sha256sum\":\"caf3fd69c77c439f162e2ba6040e9c320c4ff0d69aad1340a514319a9264df9f\"\n}<\/pre>\n\n\n\nThat\u2019s it! The data you imported is then available on the target storage devices for you to use as you wish.<\/p>\n\n\n\n Storage Import status<\/h2>\n\n\n\nThe import process will take time depending on the size of the file and the speed of the network connectivity. Especially system storage images are often large and require more time to upload.<\/p>\n\n\n\n You can check the state of the process using the following request. Set the UUID of your import storage device in the request URL.<\/p>\n\n\n\n GET \/1.3\/storage\/{uuid}<\/span>\/import<\/pre>\n\n\n\nThe response will show the current state of the importing process and the progress made in writing to the storage so far.<\/p>\n\n\n\n {\n \"storage_import\": {\n \"client_content_length\": 10737418240,\n \"client_content_type\": \"\",\n \"completed\": \"\",\n \"created\": \"2020-06-23T19:15:57Z\",\n \"error_code\": \"\",\n \"error_message\": \"\",\n \"md5sum\": \"\",\n \"read_bytes\": 1661214003,\n \"sha256sum\": \"\",\n \"source\": \"http_import\",\n \"source_location\": \"https:\/\/[REDACTED]@example.com\/files\/system.img\",\n \"state\": \"importing\"<\/span>,\n \"uuid\": \"0734ad68-e715-4687-89c5-774d708b9515\",\n \"written_bytes\": 1661214003\n }\n}<\/pre>\n\n\n\nFeel free to check the import status again later to see how the operation is progressing.<\/p>\n\n\n\n Cancelling Storage Import<\/h2>\n\n\n\nWhen the upload is started via the Storage Import API to a stand-alone storage device, the process will continue in the background without interruption to other operations.<\/p>\n\n\n\n However, if you wish to stop the import, you can cancel the operation using the following API request. Set the UUID of your import storage device in the request URL.<\/p>\n\n\n\n POST \/1.3\/storage\/{uuid}<\/span>\/import\/cancel<\/pre>\n\n\n\nYou will then get a response something along the lines of the example underneath. The state of the import will first show cancelling.<\/p>\n\n\n\n {\n \"storage_import\": {\n \"client_content_length\": 10737418240,\n \"client_content_type\": \"\",\n \"completed\": \"2020-06-23T19:19:09Z\",\n \"created\": \"2020-06-23T19:15:57Z\",\n \"error_code\": \"CLIENT_FAILURE\",\n \"error_message\": \"import task was cancelled\",\n \"md5sum\": \"6b2bfc737ef922f47ed4751d474170c6\",\n \"read_bytes\": 2494455091,\n \"sha256sum\": \"19567e0a4c83ae010c35fa094e5832e09ac20655912141138df38354ef51d030\",\n \"source\": \"http_import\",\n \"source_location\": \"https:\/\/[REDACTED]@example.com\/files\/system.img\",\n \"state\": \"cancelling\"<\/span>,\n \"uuid\": \"0734ad68-e715-4687-89c5-774d708b9515\",\n \"written_bytes\": 2494455091\n }\n}<\/pre>\n\n\n\nAfter cancelling, the storage device will return to the normal online state and again allow new storage import or other requests.<\/p>\n\n\n\n Making use of new storage<\/h2>\n\n\n\nOnce your upload has finished, you can put the new storage device to use the way you wish.<\/p>\n\n\n\n Storage image with a boot partition<\/h3>\n\n\n\nIf your storage image contains a boot partition, you can set it as the main device for a standalone cloud server.<\/p>\n\n\n\n Depending on your system storage and the previous networking setup, you may not have network access out of the box. You can use the web console at your UpCloud Control Panel or any VNC viewer to reach your cloud server<\/a> to restore connectivity.<\/p>\n\n\n\n The default network interfaces on our cloud servers include the following:<\/p>\n\n\n\n \nPublic IPv4 for Internet access<\/li>\n\n\n\n Private utility network<\/li>\n\n\n\nPublic IPv6 network connection<\/li>\n<\/ul>\n\n\n\nIf you do not need all of the default network interfaces, you can detach unwanted networks to simplify your setup. Then configure the network connection<\/a> you want to use.<\/p>\n\n\n\n File storage<\/h3>\n\n\n\nStorage images that contain a file system can be mounted to any cloud server to access the files. Uploaded storage will work much the same as any other storage device allowing you to mount it at the directory point<\/a> you wish.<\/p>\n\n\n\n Installation media<\/h3>\n\n\n\nIf you uploaded installation media<\/a>, change the boot order to select CDROM as the first device and get started with the installation process at the next startup.<\/p>\n","protected":false},"featured_media":16836,"comment_status":"open","ping_status":"closed","template":"","community-category":[109,110,122],"class_list":["post-24898","tutorial","type-tutorial","status-publish","has-post-thumbnail","hentry","community-category-upcloud-services","community-category-storage","community-category-api"],"acf":[],"_links":{"self":[{"href":"https:\/\/studiogo.tech\/upcloudold\/wp-json\/wp\/v2\/tutorial\/24898","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/studiogo.tech\/upcloudold\/wp-json\/wp\/v2\/tutorial"}],"about":[{"href":"https:\/\/studiogo.tech\/upcloudold\/wp-json\/wp\/v2\/types\/tutorial"}],"replies":[{"embeddable":true,"href":"https:\/\/studiogo.tech\/upcloudold\/wp-json\/wp\/v2\/comments?post=24898"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/studiogo.tech\/upcloudold\/wp-json\/wp\/v2\/media\/16836"}],"wp:attachment":[{"href":"https:\/\/studiogo.tech\/upcloudold\/wp-json\/wp\/v2\/media?parent=24898"}],"wp:term":[{"taxonomy":"community-category","embeddable":true,"href":"https:\/\/studiogo.tech\/upcloudold\/wp-json\/wp\/v2\/community-category?post=24898"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}

Making use of new storage<\/h2>\n\n\n\n
Once your upload has finished, you can put the new storage device to use the way you wish.<\/p>\n\n\n\n

File storage<\/h3>\n\n\n\n
Storage images that contain a file system can be mounted to any cloud server to access the files. Uploaded storage will work much the same as any other storage device allowing you to mount it at the directory point<\/a> you wish.<\/p>\n\n\n\n