As a photographer I needed an inexpensive way to protect my image files. AzCopy now has a sync option that works well. Here is how to set it up. This is the “nuts and bolts” approach to get you up and running one time. Later, I’ll show you how I schedule the synchronization.
There are 4 steps. I assume that you already have an Azure tenant and know how to create an Azure Storage Account for your Blobs. Once you have that all set up you need to do the following:
In order for AzCopy to access Azure Blob Storage, the account performing the action must be in the Storage Blob Data Contributor (Preview) role. You can do this at different levels, for this post I am using the Resource Group level so that I can access several Storage Accounts.
Once you download and unzip the executable, put it somewhere easy to find. In my case I add it to a “tools” folder. Then to make it easy in PowerShell, I create an alias to it.
New-Alias -Name azcopy -Value "D:\Data\Azure Tools\azcopy.exe"
Then you can test that it worked with the
azcopy --help # or to get sync help azcopy sync --help
The next step will create a local OAuth login file for your PowerShell session. I tried using only
azcopy -login but, for me, it only works if I specify my
azcopy login --tenant-id=7c7c7b9e-f745-47e0-96a4-caa015491ec1 To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code FOO12345 to authenticate.
Once you login with the same account you granted access to in the Configure Security section. You should see the prompt success in PowerShell.
I can check my access to the destination with the
azcopy list "https://myprodstorage.blob.core.windows.net/backups" List is using OAuth token for authentication. pictures/2011/2011-10-07/Agilty Practice-10.CR2 pictures/2011/2011-10-07/Agilty Practice-10.xmp pictures/2011/2011-10-07/Agilty Practice-11.CR2 pictures/2011/2011-10-07/Agilty Practice-11.xmp
All that is left is to decide what to sync. In my case I keep my images and sidecar files on an external drive. Each year I automatically create a new folder and have subfolders by shoot date in
YYYY-MM-DD format. If I want to sync 2013 I use the following command.
#--log-level takes Log Levels: DEBUG, INFO, WARNING, ERROR, PANIC, and FATAL azcopy sync "P:\Data\Pictures\2013\" "https://myprodstorage.blob.core.windows.net/backups/pictures/2013" --recursive=true --log-level=INFO Using OAuth token for authentication. Job 39425003-8a96-7a46-5293-80dcbfb44f40 has started 39425003-8a96-7a46-5293-80dcbfb44f40.log file created in C:\Users\Matthew/.azcopy 493 Done, 0 Failed, 3361 Pending, 0 Skipped, 3854 Total, 2-sec Throughput (Mb/s): 589.0365
AzCopy creates a sync job that you can interrogate with PowerShell (or just watch with rapt attention like I do). As you can see in this initial sync, AzCopy found 3854 files to sync. Once the job completes:
Job 39425003-8a96-7a46-5293-80dcbfb44f40 summary Elapsed Time (Minutes): 21.0068 Total Number Of Transfers: 3854 Number of Transfers Completed: 3854 Number of Transfers Failed: 0 Number of Transfers Skipped: 0 TotalBytesTransferred: 60050731232 Final Job Status: Completed
If I make changes to the files and then run the command again.
azcopy sync "P:\Data\Pictures\2013\" "https://myprodstorage.blob.core.windows.net/backups/pictures/2013" --recursive=true --log-level=INFO Using OAuth token for authentication. Job f9193419-be1f-884b-5b92-ba77ba0c0b06 has started f9193419-be1f-884b-5b92-ba77ba0c0b06.log file created in C:\Users\Matthew/.azcopy 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s): 0 Job f9193419-be1f-884b-5b92-ba77ba0c0b06 summary Elapsed Time (Minutes): 0.0333 Total Number Of Transfers: 1 Number of Transfers Completed: 1 Number of Transfers Failed: 0 Number of Transfers Skipped: 0 TotalBytesTransferred: 9 Final Job Status: Completed
I am cheap. It pains me to see the prices photo sharing sites charge for storage. Most won’t store my RAW and sidecar files either. If I cannot protect the original RAW file then even free storage is not worth the effort. Now, I can automate the sync job and use the inexpensive storage provided by Azure to ensure my precious photos are safe. Plus AzCopy is a community project with AMAZING support. They worked through the authentication issues I had and got me up and running.
Ready to start your next project with us? That’s great! Give us a call or send us an email and we will get back to you as soon as possible!