If you store data in multiple cloud storage solutions, you may have had to transfer data between them. Learn how to copy data from Azure to Amazon S3.
If your business stores data in multiple cloud storage solutions, chances are you've already encountered many instances when you had to transfer data from one cloud solution to another. In this post, you'll learn an easy way to do that using JSCAPE MFT Server. As an example, we'll show you how to set up JSCAPE MFT Server so that you can automatically copy data from Azure to AWS S3.
Note: Although JSCAPE MFT Server will be orchestrating the copying process from Azure to S3, none of the files being copied will be stored on your JSCAPE MFT Server host. All files involved will be copied directly from Azure to S3.
Watch the video
Would you prefer to watch a video version of this tutorial instead? You can play the video below. Otherwise, just skip it if you wish to continue reading.
One advantage of using JSCAPE MFT Server is that, because you already use it for a wide range of file transfer workflows, adding cloud-to-cloud data transfers into the mix further simplifies management of your organization's file transfer processes.
Recommended read: How to Transfer Files Between Cloud Storage Services
Alright, let's get this tutorial on the road. I'm going to assume you already have an Azure File Service account and an Amazon S3 account.
If you already have those, login to your JSCAPE MFT Server administrative web user interface and go to the Trading Partners module. Click Add to add a new trading partner.
We need two trading partners here. An Azure File Service trading partner and an Amazon S3 trading partner. We already showed you how to set up an Amazon S3 trading partner in the post How To Mirror an FTP/S folder to S3, so I suggest you read that first if you haven't done so yet.
To add an Azure File Service trading partner, just expand the protocol drop-down list, select Microsoft Azure File Service, and then click OK.
Give this trading partner a name, say, 'tp-azure'.
Next, specify your Microsoft Azure Storage account name and its corresponding Access key.
Click the Test server button to test your connection. If the test succeeded, click OK.
Once you get back to the main screen, you should see your newly created Microsoft Azure File Service trading partner among your list of trading partners. Of course, you should also see your Amazon S3 trading partner.
Now that you have your Azure and S3 trading partners ready, the next step would be to create the trigger that would copy data from Azure to S3. Go to the Triggers module and click Add.
For those who are using JSCAPE MFT Server version 12.1 and higher, you'll be presented with a Trigger Template dialog where you can choose a template that best describes the workflow you're about to automate. Let's just leave that blank and click OK
Give this trigger a name, e.g. 'copy from azure to s3'.
After that, specify an event that would initiate this mirroring process. We just want this trigger to execute at a particular schedule, so let's choose the Current Time event type.
In the next screen, you can use the Expression Builder to build an expression that would define the schedule when this particular trigger should fire. I want my trigger to execute at 5:59 AM everyday, so I built the expression as shown on the screen.
Recommended read:Introducing the New Trigger Conditions Expression Builder
When you get to the next screen, click the Add button to add a new trigger action. This is the part of the trigger creation process where you would specify how the copying process should be carried out.
Select the Trading Partner Synchronization action from the drop-down list and then click OK.
Let me now walk you through the key settings for this Trading Partner Synchronization action.
The first ones you'll encounter are PartnerA and PartnerB. PartnerA is basically the source. It's the trading partner from which the target trading partner will be copying from. So, in our case, that would be the Azure trading partner.
PartnerB, on the other hand, is the target. It's the trading partner to which the source trading partner will be copying data to. In our case, that would be the Amazon S3 trading partner.
Next up are PathA and PathB. PathA is the relative directory path in PartnerA which PathB will be copying from, and PathB is the relative path on PartnerB that PathA will be copying to.
The next setting we need to specify is the Copy Condition. This is the condition JSCAPE MFT Server will use to determine whether to commence copying files each time the predefined schedule of this trigger is up.
If you select:
- different time, JSCAPE MFT Server will commence copying if it sees that file timestamps on A are different from the ones on B;
- different size, JSCAPE MFT Server will commence copying if it sees that file sizes on A are different from the ones on B;
- different content, JSCAPE MFT Server will commence copying if it sees that the content in A is different from the content in B
Let's just choose different time for now.
Another setting you need to specify is the Synchronization Mode. There are four options:
- mirror - New and modified files from A are copied to B; redundant files in B will be deleted;
- synchronize - New and modified files from both paths are copied to each other
- backup - All files from A are copied to B
- contribute - New and modified files from A are copied to B
Based on the descriptions given and your particular use case, you'll likely have to choose between backup and contribute. I'm going to choose contribute, so my trigger is only going to transfer files from A that are new new and/or modified.
Lastly, you need to specify the Result Directory. This is where the results of the mirroring process will be written to.
Click OK and then drag an arrow from the Start output of the Workflow node to the Trading Partner Synchronization Action node.
Recommended read: Introducing the Redesigned Trigger Action Workflow
Click OK to finalize the trigger creation process.
That's it. Now you know how to configure JSCAPE MFT Server so that you can copy data from Azure to S3.
Try this yourself
Would you like to try this yourself? Download the FREE, fully-functional Starter Edition of JSCAPE MFT Server now.
Download JSCAPE MFT Server TrialRelated content:
How to Download Newly Added Files from an AWS S3 Folder
Set Up Automated File Transfers with Windows SMB Share
Moving Data Across a Hybrid Cloud Doesn't Have To Cost An Arm and A Leg