last block incomplete with CipherInputStream/CipherOutputStream, even with padding AES/CBC/PKCS5Padding

Question

Actually, I searched lot from internet and in stackoverflow too for this,

Initially I don't used padding in my encryption and decryption,

But Finally I got solution from here

https://mcmap.net/q/745214/-java-missing-final-characters-when-encrypting-using-blowfish

and I updated my code with padding as AES/CBC/PKCS5Padding and the same error is coming, and last block is not decrypted...

I'm working on this for last two day, but no solution found

my Crypter Code:

package mani.droid.browsedropbox;

import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.math.BigInteger;
import java.security.InvalidAlgorithmParameterException;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import javax.crypto.Cipher;
import javax.crypto.CipherInputStream;
import javax.crypto.CipherOutputStream;
import javax.crypto.NoSuchPaddingException;
import javax.crypto.SecretKey;
import javax.crypto.spec.IvParameterSpec;
import javax.crypto.spec.SecretKeySpec;

public class Crypter {

    Cipher encipher;
    Cipher decipher;
CipherInputStream cis;
CipherOutputStream cos;
FileInputStream fis;
byte[] ivbytes = new byte[]{(byte)'a', (byte)'b', (byte)'c', (byte)'d', (byte)'e',                                                                                          (byte)'f', (byte)'g', (byte)'h', (byte)'i', (byte)'j', (byte)'k', (byte)'l', (byte)'m',     (byte)'n', (byte)'o', (byte)'p'};
    IvParameterSpec iv = new IvParameterSpec(ivbytes);

public boolean enCrypt(String key, InputStream is, OutputStream os)
{
    try {
        byte[] encoded = new BigInteger(key, 16).toByteArray();
        SecretKey seckey = new SecretKeySpec(encoded, "AES");
        encipher = Cipher.getInstance("AES/CBC/PKCS7Padding");
        encipher.init(Cipher.ENCRYPT_MODE, seckey, iv);
        cis = new CipherInputStream(is, encipher);
        copyByte(cis, os);
        return true;
    } 
    catch (InvalidKeyException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (NoSuchAlgorithmException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (NoSuchPaddingException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (InvalidAlgorithmParameterException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }
    return false;
}

public boolean deCrypt(String key, InputStream is, OutputStream os)
{
    try {
        byte[] encoded = new BigInteger(key, 16).toByteArray();
        SecretKey seckey = new SecretKeySpec(encoded, "AES");
        encipher = Cipher.getInstance("AES/CBC/PKCS7Padding");
        encipher.init(Cipher.DECRYPT_MODE, seckey, iv);
        cos = new CipherOutputStream(os, encipher);
        copyByte(is, cos);
        //cos.close();
        return true;
    } 
    catch (InvalidKeyException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (NoSuchAlgorithmException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (NoSuchPaddingException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (InvalidAlgorithmParameterException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }
    return false;
}

public void copyByte(InputStream is, OutputStream os) throws IOException
{
    byte[] buf = new byte[8192];
    int numbytes;
    while((numbytes = is.read(buf)) != -1)
    {
        os.write(buf, 0, numbytes);
        os.flush();
    }
    os.close();
    is.close();
}
}

Baptlsta

Is the rest of the cipher text ok? The cipher input streams have the obnoxious habit of shuffling Exceptions beneath the carpet, including instances of BadPaddingException. – Uranian

Check the size of the file(?) you are writing to and make sure you are writing everything properly. – Push

@owlstead rest of the cipher are perfectly decrypting, but that last block is missing, to check the code I printed numbytes in copyByte, the result is when encryption, all blocks are read, 16 bits even for that last incomplete block, but for decryption last block value is not 16, less than 16..., So My guess is, its padded with null or 0, so its reading full block at the time of encryption, so the encrypted file size is same as my original file size not in multiple of 16bytes... – Baptlsta

@NikolayElenkov the encrypted file size is same as my original file size, not in multiple of 16bit..., plz refer my previous comment... – Baptlsta

Try calling doFinal() on the Cipher directly to get the encrypted bytes and eliminated any stream issues. – Push

@NikolayElenkov Actually, I want to do encryption and decryption in files, can you plz tell me the procedure to use doFinal() with files – Baptlsta

Use a small file for testing, read it into a byte array, pass it to do Final(). Or use update() and read the file in chunks if it is larger, pass the last chunk to doFinal(). – Push

@NikolayElenkov I did as you said, read byte[] from file and dofinal and write the output byte[] in file, its again works in encryption, and for decryption, same IllegalBlockException last block incomplete in decryption Code for Read and Write byte from/to file, Read: ByteArrayOutputStream byteArray = new ByteArrayOutputStream(); copyByte(is, byteArray); byte[] enData = byteArray.toByteArray();, Write: byte[] deData = decipher.doFinal(enData); BufferedOutputStream bos = new BufferedOutputStream(os); bos.write(deData); bos.flush(); bos.close(); – Baptlsta

This is the function copyByte Code: public void copyByte(InputStream is, OutputStream os) throws IOException { byte[] buf = new byte[8192]; int numbytes; while((numbytes = is.read(buf)) != -1) { Log.e("tamil", Integer.toString(numbytes)); os.write(buf, 0, numbytes); } os.flush(); is.close(); os.close(); } – Baptlsta

Answer 1

I've had exactly the same problem. The accepted solution works because you've used cipher mode that does not require padding, but this is not a way crypto-related issues are fixed.

According to the CipherOutputStream documentation, you have to call close() method in order to finalize encryption properly (i.e., padding block is added).

This method invokes the doFinal method of the encapsulated cipher object, which causes any bytes buffered by the encapsulated cipher to be processed. The result is written out by calling the flush method of this output stream.

This method resets the encapsulated cipher object to its initial state and calls the close method of the underlying output stream.

If you want to preserve OutputStream open even after calling CipherOutputStream.close() method, you can wrap OutputStream to the stream that does not close it. For example:

public class NotClosingOutputStream extends OutputStream {
  private final OutputStream os;

  public NotClosingOutputStream(OutputStream os) {
    this.os = os;
  }

  @Override
  public void write(int b) throws IOException {
    os.write(b);
  }

  @Override
  public void close() throws IOException {
    // not closing the stream.
  }

  @Override
  public void flush() throws IOException {
    os.flush();
  }

  @Override
  public void write(byte[] buffer, int offset, int count) throws IOException {
    os.write(buffer, offset, count);
  }

  @Override
  public void write(byte[] buffer) throws IOException {
    os.write(buffer);
  }
}

Then you can use:

...
cos = new CipherOutputStream(new NotClosingOutputStream(os), encipher);
copyByte(is, cos);
cos.close();
...

Note the os stream does not get closed, you need to do it on your own when appropriate.

Microseism

This should be the accepted answer to the question for future readers. – Executory

Answer 2

Finally I got answer for my own question, with trial and error Actually here Conflict is I set Padding in encipher = Cipher.getInstance("AES/CBC/PKCS7Padding");

and Set IV with some values.....,

Finally I got Answer only just replaced the Algorithm

From:

AES/CBC/PKCS7Paddinng

To:

AES/CFB8/NoPadding

and its worked like charm...., So I suggest this answer for others who struggling with this problem, if you solved you problem, mention here for others...

Baptlsta

Do you understand why? Or have a link to an explanation?? – Shirberg

I am also facing this issues, I need link for understanding, how this issue is resolved.Can you please give me the link? – Celebrant

Some algorithm like DESeade or AES requires input data to have the good size, which usually is a multiple of Cipher.getBlockSize(). You have the choice between using a padding in the algortihm ("PKCS7Paddinng") or not usign a padding, and it's up to you to be sure that input data has the good size. If it's not, you get that exception. – Creatural

Answer 3

I've seen CipherInputStream fail with padding problems too. This behaviour varied across different versions of the JVM. Eg 7u55 32 bit my code worked fine, 7u55 64 bit same code failed... and I also saw failures on later 32 bit JVMs. Workaround was to use byte array methods and avoid CipherInputStream.

Photoconductivity

Answer 4

Not sure if this is relevant to OP's problem, but this may help someone.

When you repeatedly get that java.io.IOException: last block incomplete in decryption regardless of what you change, check if you are still using the file from some previous run. If your read/write test code appends to that file, you will always get that exception -- unless you delete the corrupt file that you write to.

Cleora

Answer 5

Using CipherInputStream with padding is possible, switching to NoPadding is a workaround but not a solution.

Padding is applied when CipherInputStream reaches the end of the stream. The important point is that you have to call the read() method of the CipherInputStream at least twice for getting all the data.

The following example demonstrates reading a CipherInputStream with padding:

public static void test() throws Exception {
    Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");

    SecureRandom rnd = new SecureRandom();
    byte[] keyData = new byte[16];
    byte[] iv = new byte[16];
    rnd.nextBytes(keyData);
    rnd.nextBytes(iv);
    SecretKeySpec key = new SecretKeySpec(keyData, "AES");

    cipher.init(Cipher.ENCRYPT_MODE, key, new IvParameterSpec(iv));

    ByteArrayOutputStream buffer = new ByteArrayOutputStream();
    CipherOutputStream out = new CipherOutputStream(buffer, cipher);

    byte[] plain = "Test1234567890_ABCDEFG".getBytes();
    out.write(plain);
    out.flush();
    out.close();
    byte[] encrypted = buffer.toByteArray();
    System.out.println("Plaintext length: " + plain.length);
    System.out.println("Padding length  : " + (cipher.getBlockSize() - (plain.length % cipher.getBlockSize())));
    System.out.println("Cipher length   : " + encrypted.length);

    cipher.init(Cipher.DECRYPT_MODE, key, new IvParameterSpec(iv));
    CipherInputStream in = new CipherInputStream(new ByteArrayInputStream(encrypted), cipher);
    buffer = new ByteArrayOutputStream();
    byte[] b = new byte[100];
    int read;
    while ((read = in.read(b)) >= 0) {
        buffer.write(b, 0, read);
    }
    in.close();

    // prints Test1234567890_ABCDEFG
    System.out.println(new String(buffer.toByteArray()));
}

Executory

Answer 6

For those who are struggling with aes encryption / decryption with image file, here is my example, and it works like a charm.

public static String decrypt(String textToDecrypt)  {
    byte[] dataDecrypted = null;

    try {
        Cipher cipher = getCipher();
        SecretKey key = getKey();
        IvParameterSpec iv = getIV();
        if(cipher == null) {
            return null;
        }
        cipher.init(Cipher.DECRYPT_MODE, key, iv);

        byte[] dataToDecrypt = Base64.decode(textToDecrypt, Base64.NO_WRAP);
        ByteArrayInputStream bais = new ByteArrayInputStream(dataToDecrypt);
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        // Wrap the output stream
        CipherOutputStream cos = new CipherOutputStream(baos, cipher);

        // Read bytes
        int count = 0;
        byte[] buffer = new byte[DEFAULT_BYTE_READ_WRITE_BLOCK_BUFFER_SIZE];
        while ((count = bais.read(buffer)) != -1) {
            cos.write(buffer, 0, count);
        }
        cos.close();    // manually do close for the last bit

        dataDecrypted = baos.toByteArray();

        // Close streams.
        baos.close();
        bais.close();
    } catch (Exception e) {
        e.printStackTrace();
    }

    return (dataDecrypted == null ? "" : Base64.encodeToString(dataDecrypted, Base64.NO_WRAP));
}

---

public static String encrypt(String textToEncrypt) {
    byte[] dataEncrypted = null;

    try {
        Cipher cipher = getCipher();
        SecretKey key = getKey();
        IvParameterSpec iv = getIV();
        if (cipher == null) {
            return null;
        }
        cipher.init(Cipher.ENCRYPT_MODE, key, iv);

        byte[] dataToEncrypt = Base64.decode(textToEncrypt, Base64.NO_WRAP);
        ByteArrayInputStream bais = new ByteArrayInputStream(dataToEncrypt);
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        // Wrap the output stream
        CipherOutputStream cos = new CipherOutputStream(baos, cipher);

        // Read bytes
        int count = 0;
        byte[] buffer = new byte[DEFAULT_BYTE_READ_WRITE_BLOCK_BUFFER_SIZE];
        while ((count = bais.read(buffer)) != -1) {
            cos.write(buffer, 0, count);
        }
        cos.close();    // manually do close for the last bit

        dataEncrypted = baos.toByteArray();

        // Close streams.
        baos.close();
        bais.close();
    } catch (Exception e) {
        e.printStackTrace();
    }

    return (dataEncrypted == null ? "" : Base64.encodeToString(dataEncrypted, Base64.NO_WRAP));
}

Elegy