Azure AzCopy- A very helpful tool

Written by Surbhi Agarwal | Nov 27, 2018 7:06:00 AM

Azure is a leading cloud provider and it has many offerings that are very useful for ISVs and enterprises. Amongst the pool of useful tools, AzCopy is one of the most powerful tools.

AzCopy is a command-line utility which is designed for copying data to or from Microsoft Azure Blob, File, and Table storage which use simple commands designed for optimal performance. You can copy data between a file system and a storage account or that can be between storage accounts. That totally depends on your work requirement.

I am going to show how to transfer huge data from one Linux server to another Linux server with the help of AzCopy. It also provides us the information of all the files which are transferred, skipped, faced errors or successfully moved. This helps us to keep track of the data which we are transferring from one server to other. These servers can be Windows or Linux based on user requirements.

I am going to use Ubuntu 16.04 (Servers) and File storage to transfer data from one server to another server.

Step 1: Install AzCopy on both the Ubuntu server

Commands:

echo "deb [arch=amd64] https://packages.microsoft.com/repos/microsoft-ubuntu-xenial-prod/ xenial main" > azure.list
sudo cp ./azure.list /etc/apt/sources.list.d/
sudo apt-key adv --keyserver packages.microsoft.com --recv-keys EB3E94ADBE1229CF
sudo apt-get update
sudo apt-get install azcopy

Note: apt-get update is a necessary command because if you don’t run this command then it may give out some error such as - AzCopy will not work properly.

Step 2: Go to storage account then Files

Step 3: Go to file then to file share

Step 4: Now click on the file share. Then give the name and size of the block you want to store data from one server (Source) to this file share and then again to another server (destination) where you want that data to be copied. Click on ‘create’ to create file share.

Step 5: Run the following set of commands on the server one (Source). So, data or files will get copied from server one to file share. In destination, you need to give the URL of your file share as given below in the picture. In dest-key you have to give the private key of your storage account. The source will be your location of the file which you need to transfer to file share.

When you will run this command, you will see the following output.

Commands:
azcopy \
--source /opt/sampledata \
--destination https://xxxxxx.xxxx.core.windows.net/hello \

--dest-key Cdj7eg4waG3tfyu6RSJhoFFd5pCSrLQj1urt/kpvCPE4PJ3o7yAOhh11Ma8dR7xxIlzeGR6vF0kjyIiCw
== \

--recursive

Step 6: Run the following set of commands on another server (destination). So, data or files will get copied from file share to destination server. In the source, you need to give a URL of your file share as given below in picture, in source-key you have to give the private key of your storage account. In the source, you need to give the URL of your file share. And destination will be the path you want the file to be copied to.

Commands:
azcopy \
--source https://xxxxx.xxxxx.core.windows.net/hello \

--destination /opt/sampledata \
--source-key
Cdj7eg4waG3tfyuLoFFd5pCSrLQj1urt/kpvCPE4PJ3o7yAOhh11Ma8dR7xxIlzeGR6vF0kjyIiCw== \
--recursive

Step 7: We have successfully copied the data from one server to another. Therefore, AzCopy is a very useful tool. This is safe but sometimes we have to keep in mind that it does not copy files with the same name. We have to be careful about that. As you can see in the screenshot, first sample data was not there and later it was present in another server (destination).

View full post