How to encrypt/decrypt long input messages with RSA? [Openssl, C]

Question

I wrote a simple test program that encrypts/decrypts a message.

I have a keylength:

int keylength = 1024; // it can also be 2048, 4096

and max input length:

int maxlen = (keylength/8)-11;

and I know that my input size should be < than maxlen, something like this:

if(insize >= maxlen)
        printf("cannot encrypt/decrypt!\n");

My question is simple - is it possible (if so, how can I do this) to encrypt/decrypt with RSA messages LONGER than maxlen?

Main code is also, very simple but works only when insize < maxlen:

   if((encBytes=RSA_public_encrypt(strlen(buff1)+1, buff1, buff2, keypair, RSA_PKCS1_PADDING)) == -1)
    {
        printf("error\n");
    }

    if((decBytes=RSA_private_decrypt(encBytes, buff2, buff3, keypair, RSA_PKCS1_PADDING)) == -1)
    {
        printf("error\n");
    }

Neurovascular

Normally, you encrypt your long message using a symmetric cipher (AES seems to be a popular choice) and encrypt the AES key only using RSA. – Zirconium

As @JoachimIsaksson pointed out, you have a fundamental misunderstanding of applied cryptography. RSA decryption is extremely computationally expensive compared to symmetric key decryption. – Arundell

@Arundell ... which speaks for the security of assymetric encryption, considering brute force attacks. – Grof

Answer 1

Encrypting long messages requires combined scheme - RSA algorithm encrypts session key (i.e. AES key), and data itself is encrypted with that key. I would recommend to not invent another bicycle and use well established scheme, i.e. PKCS#7/CMS or OpenPGP, depending on your needs.

Uproarious

What do you mean under 'not work'? Try to not pad exactly to modulus size, I guess that RSA_Encrypt adds padding by itself. – Uproarious

By 'it doesnt work' I mean that it shows errors on encrypt/decrypt - doesnt encrypt/decrypt whole message (even without padding). Please see my edit. – Neurovascular

Answer 2

You would be able to encrypt long messages with RSA the same way as it is done with block ciphers. That is, encrypt the messages in blocks and bind the blocks with an appropriate chaining mode. However, this is not the usual way to do it and you won't find support for it (RSA chaining) in the libraries you use.

Since RSA is quite slow, the usual way to encrypt large messages is using hybrid encryption. In hybrid encryption you use a fast symmetric encryption algorithm (like AES) for encrypting the data with a random key. The random key is then encrypted with RSA and send along with the symmetric key encrypted data.

EDIT:

As fore your implementation, you have insize = 1300 and keylength = 1024 which gives maxlen = 117. To encrypt the full message you those needs 12 encrypts, that each produce 128 bytes, giving an encrypted size of 1536 bytes. In your code you only allocates buffers of 1416 bytes. Also, you don't seem to allow for 128 bytes output as you only increment with 117 in:

RSA_public_encrypt(maxlen, buff1+i, buff2+i, keypair, RSA_PKCS1_PADDING)

and

i += maxlen;

Endoparasite

Yes, I know all of that. The point is, I want to do encrypting/decrypting ONLY by using RSA, as a block cipher, as you suggested. My main problem is that the code I posted in my first post doesnt work - I always get RSA encrypt/decrypt error, thats the real issue of this topic. Are you able to help? – Neurovascular

Answer 3

You can use RSA as block cipher in that case. That is break the message to several blocks smaller than maxlen. Otherwise impossible.

Brain

@Neurovascular It's not practical to use RSA as a block cipher, because it's terribly slow. A typical CPU can RSA 100 kilobytes of data per second compared to AES, which is around 1,000–10,000 times faster. – Adal

@Neurovascular that's just an implementation mistake, nothing to do with using RSA as block cipher. Vlad, at least point to the use of hybrid encryption, just saying that it is not possible (although strictly true, if only RSA may be used) is not so helpful for Stackoverflow. – Montane

Answer 4

If you want to run RSA in a "block cipher" kind of mode, you would need to run it in a loop.

Like most of the other commenters, I'd like to point out that this is a bad use of RSA - You should just encrypt a AES key with RSA then use AES for the longer message.

However, I'm not one to let practicality get in the way of learning, so here's how you'd do it. This code isn't tested, since I don't know what libraries you are using. It's also a little overly-verbose, for readability.

int inLength = strlen(buff1)+1;
int numBlocks = (inLength / maxlen) + 1;

for (int i = 0; i < numBlocks; i++) {
    int bytesDone = i * maxlen;
    int remainingLen = inLength - bytesDone;
    int thisLen; // The length of this block

    if (remainingLen > maxlen) {
       thisLen = maxlen;
    } else {
        thisLen = remainingLen;
    }

    if((encBytes=RSA_public_encrypt(thisLen, buff1 + bytesDone, buff2 + bytesDone, keypair, RSA_PKCS1_PADDING)) == -1)
    {
        printf("error\n");
    }

    // Okay, IDK what the first parameter to this should be. It depends on the library. You can figure it out, hopefully.
    if((decBytes=RSA_private_decrypt(idk, buff2 + bytesDone, buff3 + bytesDone, keypair, RSA_PKCS1_PADDING)) == -1)
    {
        printf("error\n");
    }
}

Plankton

Answer 5

maxlen actually depends on a key length and padding mode. Think newer padding scheme ´OAEP´ e.g. in Java Encryption Engine takes additional 42 bytes instead of 11. Known libraries are not designed for using RSA in a block cipher mode.

For that purpose, beyond fragmentation as answered above, security aspects require further modification of the padding scheme, e,g, https://crypto.stackexchange.com/a/97974/98888

Grof