Learn how to monitor an Amazon S3 folder for new files and then transfer each new file to Azure
When you have a multi-cloud infrastructure, there will be several instances when you'll want to transfer data from one cloud platform to another. Perhaps you'd want to aggregate data, perform backups, carry out migrations, and so on. Whatever the case may be, you'll normally want to perform these data transfers automatically. One way to do that is by using JSCAPE MFT Server.
JSCAPE's managed file transfer server integrates with several cloud storage solutions like Amazon S3, Azure Files, Google Storage, Dropbox, and many others. By combining this with its capability to carry out automated file transfers, JSCAPE MFT Server can make cloud integrations quick and easy. Let me show you how it's done.
Watch the video
Would you prefer to watch a video version of this tutorial instead? You can play the video below. Otherwise, just skip it if you wish to continue reading.
In this example, we're going to set up JSCAPE MFT Server so it can monitor an Amazon S3 folder for new files and then transfer each newly added file to Azure.
There are a couple of ways to do that using JSCAPE MFT Server, but one way is by using a directory monitor that detects newly added files on the Amazon S3 folder and a trigger with the following elements:
1. It listens for a Directory Monitor File Added event originating from the monitored S3 folder;
2. A trigger action that downloads each new file from S3; and
3. A succeeding trigger action that would take each of those download files as they come in and upload them to Azure.
We already gave a tutorial that covers #1 and #2, so you can use that as reference. Read the post or watch the video in the post 'How to Download Newly Added Files from an AWS S3 Folder'.
If you watched or read that tutorial, you'll notice that we used an Amazon S3 trading partner there. Here, in step 3, we'll also need an Azure trading partner. If you don't know how to create one, I suggest you read the post or watch the video on the post 'How To Connect And Upload Files From Your Server To An Azure File Share'. Just disregard the parts that talk about a directory monitor and a trigger that uploads files. All you need at this point is to learn how to create an Azure trading partner.
Once you have an Azure trading partner, all that's left for you to do is to add a trigger action to the trigger described in steps 1 and 2, that would upload each freshly downloaded files to that Azure trading partner.
So, in my case, I'm just going to edit the trigger 'download new s3 files', which, if you read the post or watched the video in the post 'How to Download Newly Added Files from an AWS S3 Folder', you'll notice that that's the trigger that downloads files from my S3 folder.
In your case, you might want to use a more appropriate trigger name, say, 'Download files from S3 and upload to Azure' or something like that. Since I can no longer change the name of my trigger, I'll just give it a more appropriate description here.
After that, you just need to proceed to the list of trigger actions and add a new trigger action. Click the Add button, expand the Action drop-down list, select the Trading Partner File Upload action, and click OK.
Once you're inside the trigger action parameters dialog, select the Azure trading partner from the partner drop-down list. In my case, that would be 'tp-azure'. Next, specify the local path of each file you want to upload. Since it's just the same file that was downloaded from S3, just use the exact same path and variable you used in the Trading Partner File Download action that downloaded files from S3. So, in my case, that would be:
C:\downloadfroms3\%Name%
And if you want to store that file in a specific directory in Azure, then you can specify that in the Remote Directory field.
It's not that crucial, but another thing you might want to do is go to the Advanced tab and add a note that would describe what this trigger action does. In fact, I recommend you do this in all your trigger actions moving forward. It's as useful as adding comments in program code. Once you have several triggers and trigger actions on your MFT Server instance, it can be very difficult to troubleshoot without any descriptions and notes that tell you what a particular part does.
So, this is how your list of trigger actions is going to look like once you start adding notes.
Click OK until you're back at the main screen.
That's it. Now you know how to configure JSCAPE MFT Server to monitor an Amazon S3 for new files and transfer those files to Azure.
Try this yourself
Would you like to try this yourself? Download the FREE, fully-functional Starter Edition of JSCAPE MFT Server now.