How to Choose Encryption Key Lengths for File Transfers | JSCAPE

Note: This post was originally posted on July 6, 2012 but was updated on August 28, 2018

Overview

Whenever you generate a server key on JSCAPE MFT Server, you're asked to specify an encryption key length. Since longer keys translate to stronger security, choosing the longer key length might seem like a no-brainer. But why did JSCAPE have to include an option for a shorter key then? Will there be instances where you'll have to choose the shorter key? The answer is yes. And in this post, we'll explain when that can be the case.

Figure 1

Before we get to that part, let me first show you the difference when you connect via encrypted file transfer protocols (like FTPS or SFTP) using longer encryption keys versus connecting using shorter keys. This difference was quite evident in an experiment I did that you can easily replicate using the free, Starter Editions of JSCAPE MFT Server and JSCAPE MFT Monitor.

Longer encryption keys are more computationally expensive

I conducted a series of tests using the following applications:

• JSCAPE MFT Server - This is where I set up my encrypted file transfer services
• JSCAPE MFT Monitor - This is the tool I used to carry out load testing on the services running on MFT Server.

I used this set up to compare the performance of certain secure file transfers using three (3) different key lengths: an RSA 1024-bit key length, an RSA 2048-bit key length, and an RSA 4096-bit key length. I ran one set of tests on an FTPS service and another on an SFTP service. Both FTPS and SFTP employ server keys to encrypt files during file transfers and both exhibited similar behaviors.

Here's a snapshot of some of the tests I did on the SFTP service.

Figure 2 - JSCAPE MFT Monitor connect time test results for 1024, 2048, and 4096-bit keys

Let me explain what I did here.

On MFT Server

First, I set up 3 SFTP services on MFT Server; the first on port 22, the second on port 222, and the third on port 2222. Each of these services used RSA keys of different lengths:

• SFTP on port 22 used a 1024-bit RSA key;
• SFTP on port 222 used a 2048-bit RSA key; and
• SFTP on port 2222 used a 4096-bit RSA key

On MFT Monitor

Next, I went to my MFT Monitor instance and set up 3 different monitors for load testing. These monitors connected to the SFTP services on my MFT Server instance:

• A monitor named 'sftp-key-1024' that connected to the SFTP service on port 22;
• A monitor named 'sftp-key-2048' that connected to the SFTP service on port 222; and
• A monitor named 'sftp-key-4096' that connected to the SFTP service on port 2222.

I then ran 3 sets of tests. In the first test, I ran each monitor with a user count of only 1 user.

Figure 3 - load tests for 1 user

On the second test, I increased the user count of each monitor to 50.

Figure 4 - load tests for 50 users

And then on the third test, I increased the user count further to 100 users.

Figure 5 - load tests for 100 users

For this post, we're particularly interested in one statistic: the "average connect time" (Connect Time in the screenshots). The connect time is the time it takes for a connection to be established. Why that? You see, part of the process of establishing a connection for secure file transfer services like SFTP and FTPS is the negotiation of the session key, which is initially encrypted by the public key you create during server key generation. For an FTPS session, which uses SSL or TLS, this part is known as the SSL (technically TLS these days) handshake.

The SSL-handshake (as well as the equivalent process in SFTP) is a resource-hungry operation. Furthermore, its appetite grows with key size. For example, in my SFTP tests, you'll notice that the "average connect time" increases with key size. So, for example, if you look at Figure 3, the connection time to the 1024-bit key SFTP service took 109ms, the connection to the 2048-bit key SFTP service took 125 ms, and the connection to the 4096-bit key SFTP service took 156ms.

The same trend applies to the 50-user tests and the 100-user tests. Connection times become noticeably longer when you increase the number of concurrent users because the server will be using a lot of CPU to handle the concurrent connections.

Based on the results, we can say that when you increase the key length (1024 to 2048 to 4096) of your public keys, your connection times will be generally longer. So, what's making this happen?

Let's take a look at the events involved at the start of FTPS connections (SFTP connections also start in almost the same way).

Note: I did this in a lab environment. In a real world scenario, network conditions can get even worse due to latency and packet loss.

How the SSL Handshake impacts connection time

SSL sessions always start with what is known as the SSL handshake. Here are the steps of the handshake in a nutshell. Please note that this is only a simplified version, containing only the portions that are most relevant to this post.

As you can see from the later part of the SSL handshake, the main purpose of the public/private key pair is to encrypt/decrypt what is known as the session key. Once an SSL handshake is complete, succeeding encryption/decryption operations will be carried out by the session key. The session key is a symmetric key, while the public key as well as its private key pair (which is stored in your managed file transfer server) are asymmetric keys.

Decryption is more expensive than encryption

Generally speaking, decryption is slower than encryption. And when you increase the key length, decryption speed slows down even more.

This has large implications on servers because, during the SSL handshake, decryption takes place on the server side. And really, when you talk about servers, you virtually always have a one-to-many relationship. That is, one managed file transfer server will normally cater to dozens, hundreds, or perhaps even thousands of clients.

Get all those clients to initiate SSL connections to your managed file transfer server at the same time, and you might just have a server crash in your hands.

In fact, there's a particular DoS (Denial of Service) attack tool that exploits this particular SSL vulnerability. The tool works by repeatedly requesting the SSL-enabled server to enter into "renegotiation", which forces the server to keep on entering into an SSL handshake.

How to choose the right key length

First of all, if you have a really powerful server with lots of memory and CPU, then certainly going for the longer encryption key all the time should be a no-brainer. However, if you've got limited resources, you might have to be selective when doing file transfers that use long encryption key lengths (or encryption at all).

Try to look into the kind of information you send through your file transfer system. Does it contain data that you need to keep confidential? A good way to start would be to review laws and regulations like SOX, GLBA, HIPAA, PCI-DSS, EU Data Protection Directive, and state data breach notification laws. For all you know, in your case, strong encryption might already be mandated by law.

If you see a regulation that affects you, check if it requires a minimum key length.

Some of the types of information covered by the laws/regulations mentioned earlier include: personal information, financial reporting data, medical records, and credit card information. These types of data will most probably have to be protected with strong encryption.

But there are also other types of information which may not be included there but still require optimal protection. Things like secret formulas, source codes, salaries of key managers, strategies, list of customers, list of suppliers, customers with special discounts, student grades, evaluation results, and so on, might be the kind of information you'd like to keep secret.

Once you've determined the kind of information that goes through your file transfer system, do some risk analysis to identify possible risk sources and possible levels of motivation. For example, do you have data that might motivate a hacker, industrial spy, terrorist, or malicious employee to launch any kind of attack on your system?

If the answer is yes, then you may need to go for longer key lengths.

Summary

There are no hard and fast rules here. Each organization is unique, so you can use what you've learned here as a guide and then decide for yourself when to use a long encryption key or a short one. I hope we were able to provide you with ample information so you could make an educated decision.

Want to try this out yourself?

Download the free, fully-functional starter editions of JSCAPE MFT Server and JSCAPE MFT Monitor now.

Download MFT Server

Download MFT Monitor