Android 8.0: IllegalBlocksizeException when using RSA/ECB/OAEPWithSHA-512AndMGF1Padding

Question

I usually find answers for most of our issues here but this time I need to ask :-).

We have encountered a problem with RSA encryption / decryption in one of our Apps running on Android 8.0 (API level 26).

We've been using RSA with "RSA/ECB/OAEPWithSHA-256AndMGF1Padding", which works fine on all versions up to Android 7.1. The same code running on Android 8.0 throws an IllegalBlocksizeException when calling Cipher.doFinal().

Here is the code to reproduce the issue:

private KeyStore mKeyStore;

private static final String KEY_ALIAS = "MyKey";
void testEncryption() throws NoSuchProviderException, NoSuchAlgorithmException, InvalidAlgorithmParameterException, KeyStoreException, IOException, CertificateException, BadPaddingException, IllegalBlockSizeException, InvalidKeyException, UnrecoverableEntryException, NoSuchPaddingException {

    mKeyStore = KeyStore.getInstance("AndroidKeyStore");
    mKeyStore.load(null);

    // Generate Key Pair -------------------------------------
    KeyPairGenerator kpg = KeyPairGenerator.getInstance(KeyProperties.KEY_ALGORITHM_RSA, "AndroidKeyStore");
    kpg.initialize(new KeyGenParameterSpec.Builder(
            KEY_ALIAS,
            KeyProperties.PURPOSE_ENCRYPT | KeyProperties.PURPOSE_DECRYPT)
            .setDigests(KeyProperties.DIGEST_SHA256, KeyProperties.DIGEST_SHA512)
            .setEncryptionPaddings(KeyProperties.ENCRYPTION_PADDING_RSA_OAEP)
            .setKeySize(2048)
            .build());
    KeyPair kp = kpg.generateKeyPair();

    // Encrypt -----------------------------------------------
    KeyStore.PrivateKeyEntry privateKeyEntry = (KeyStore.PrivateKeyEntry)mKeyStore.getEntry(KEY_ALIAS, null);
    PublicKey publicKey = (PublicKey) privateKeyEntry.getCertificate().getPublicKey();
    Cipher cipher = Cipher.getInstance("RSA/ECB/OAEPWithSHA-256AndMGF1Padding");
    cipher.init(Cipher.ENCRYPT_MODE, publicKey);
    String x = "It doesn't have to be perfect, it's just for demonstration.";

    byte [] vals = cipher.doFinal(x.getBytes("UTF-8"));

    byte[] encryptedBytes = Base64.encode(vals, Base64.DEFAULT);
    String encryptedText = new String(encryptedBytes, "UTF-8");


    // Decrypt -----------------------------------------------
    PrivateKey privateKey = privateKeyEntry.getPrivateKey();

    Cipher output = Cipher.getInstance("RSA/ECB/OAEPWithSHA-256AndMGF1Padding");
    output.init(Cipher.DECRYPT_MODE, privateKey/*, spec */);

    byte[] bxx = Base64.decode(encryptedText, Base64.DEFAULT);
    byte[] bytes = output.doFinal(bxx);  // <= throws IllegalBlocksizeException

    String finalText = new String(bytes, 0, bytes.length, "UTF-8");
}

I tried other padding algorithms too. "RSA/ECB/OAEPWithSHA-1AndMGF1Padding" works and also "RSA/ECB/PKCS1Padding" works fine. As a workaround I could change the padding but this might cause problems when updating from a previous version of the App which used "RSA/ECB/OAEPWithSHA-256AndMGF1Padding" because stored data could not be read anymore.

Does anybody have the same problem and maybe an idea how to fix it without changing the padding?

Thanks in advance.

Dollfuss

Why does your title say something different from the body of the question? – Md

Hmm? Sorry, but it's not completely different. Got the same result with SHA-256 as with SHA-512. – Dollfuss

Yes, so do I. I think it has something to do with the AndroidKeyStore. I would send this as a bug report to Android if you can't figure it out soon. – Md

We are experiencing the very same problem. We also use "RSA/ECB/OAEPWithSHA-256AndMGF1Padding" and get the IllegalBlocksizeException – Hyla

did you get solution for above issue? – Factorize

Answer 1

A possible solution is described here in the comment #15 from Sep 8, 2017 07:08PM:

https://issuetracker.google.com/issues/36708951#comment15

I changed the Cipher-Initialisation from

Cipher cipher = Cipher.getInstance("RSA/ECB/OAEPWithSHA-256AndMGF1Padding");
cipher.init(Cipher.DECRYPT_MODE, this.getPrivateKey(context));

to

OAEPParameterSpec sp = new OAEPParameterSpec("SHA-256", "MGF1", new MGF1ParameterSpec("SHA-1"), PSource.PSpecified.DEFAULT);
Cipher cipher = Cipher.getInstance("RSA/ECB/OAEPWithSHA-256AndMGF1Padding");
cipher.init(Cipher.DECRYPT_MODE, this.getPrivateKey(context), sp);

I tested this on Android 6 up to Android 8 (emulator) and the issue seems to be gone. You should also change the Cipher.ENCRYPT_MODE-Implementation.

Hemocyte

saved my day, nice answer, I guess it should be accepted – Denouement

Just run into this problem on Android 9 device while Android 5 was working fine. Thanks a lot, saved my day. – Portiaportico

Hi. Maybe I misunderstanding something but why do you set "SHA-1" for MGF1ParameterSpec and not SHA-256? – Nicaragua

@LeandroOcampo KeyStore uses SHA-1 – Biodynamics