zLib inflate() hangs while uncompressing buffer

Question

I use zLib 1.2.7, taken from here. I have compiled it in Microsoft Visual Studio 2010 as a static library and added it to my project.

I need to decompress some binary data compressed with deflate algorithm. Here it is:

unsigned char rawData[114] =
{
    0x00, 0x00, 0x00, 0x00, 0x15, 0x82, 0x05, 0x9D, 0x62, 0x91, 0x9A, 0x86, 0x26, 0xF3, 0x45, 0xBF, 
    0xE1, 0x69, 0x19, 0xA8, 0x80, 0x21, 0x08, 0x43, 0xF1, 0xEF, 0xCC, 0x01, 0x68, 0x4E, 0x3C, 0x06, 
    0x59, 0x6D, 0x90, 0xB2, 0x1F, 0xC3, 0x87, 0xC2, 0xBF, 0xC0, 0x90, 0xBE, 0x1F, 0x11, 0xB6, 0xD7, 
    0xB7, 0x06, 0x18, 0x32, 0x5F, 0x80, 0x8F, 0x09, 0xF1, 0x81, 0xF2, 0xB8, 0xC8, 0x9E, 0x71, 0xB7, 
    0xC9, 0x73, 0x7E, 0x88, 0x02, 0xD0, 0x9C, 0x65, 0xB0, 0x34, 0xD3, 0x97, 0x33, 0xE8, 0x80, 0x2D, 
    0x09, 0xC6, 0x5B, 0x03, 0x4D, 0x39, 0x73, 0x74, 0x1B, 0xAD, 0x19, 0x9D, 0xF0, 0xCA, 0x6F, 0xBD, 
    0xA4, 0xD5, 0x33, 0x6E, 0xDF, 0x1F, 0x11, 0x8A, 0xC5, 0xA2, 0x1C, 0x99, 0xE2, 0xDB, 0xBF, 0x7C, 
    0x0E, 0x8B
};

This block of data was captured from SPDY session. And this is my uncompress code (SPDY_dictionary_txt can be found on previous link):

INT APIENTRY WinMain( __in HINSTANCE hInstance, __in_opt HINSTANCE hPrevInstance, __in LPSTR lpCmdLine, __in int nShowCmd )
{
    z_stream zStream = { 0 };
    DWORD Length = sizeof(rawData);

    zStream.zalloc = Z_NULL;
    zStream.zfree  = Z_NULL;
    zStream.opaque = Z_NULL;

    inflateInit(&zStream);

    zStream.avail_in = Length;
    zStream.next_in = rawData;

    DWORD dwUsed = 0;
    DWORD dwSize = 5 * 1024;
    LPBYTE lpDecompressed = NULL;
    BOOL triedDictionary = FALSE;
    BOOL uncompressed = TRUE;

    lpDecompressed = (LPBYTE)calloc(dwSize, 1);

    do
    {
        zStream.next_out = (LPBYTE)((DWORD_PTR)lpDecompressed + dwUsed);
        zStream.avail_out = dwSize - dwUsed;

        int zlib_rv = inflate(&zStream, Z_NO_FLUSH); // <-- THIS HANGS!

        if (zlib_rv == Z_NEED_DICT) 
        {
            if (triedDictionary)
            {
                uncompressed = FALSE;
                break;
            }

            triedDictionary = TRUE;
            inflateSetDictionary(&zStream, SPDY_dictionary_txt, sizeof(SPDY_dictionary_txt));
        }

        if (zlib_rv < 0)
        {
            uncompressed = FALSE;
            break;
        }

        dwUsed += dwSize - dwUsed - zStream.avail_out;
    }
    while (zStream.avail_in);

    if(!uncompressed)
    {
        OutputDebugString("Could not decompress buffer.\n");
    }
    else
    {
        OutputDebugString("Buffer was decompressed.\n");
    }

    Sleep(1000);

    return 0;
}

I started to debug this code and found out that it hangs on the first inflate() call. What is this? Is this a bug in zLib or maybe my code is wrong?

Bandage

I tried the code on Linux, and it doesn't hang. Instead, zlib_rv is -3 (Z_DATA_ERROR) and the code prints "Could not decompress buffer." – Triplicity

OK, maybe I captured wrong data. But it means that 1.2.7 version has bug which can be used for DoS attack. – Bandage

You are correct. If you use zlib correctly indeed, it means you discovered a DoS attack vector. I would consider sending mail to [email protected] and explain your findings in enough detail that they can be reproduced. This is an important enough safety related thing that you shouldn't just leave the problem there. – Triplicity

OK, but maybe it is not zLib problem; I've downloaded it from another site, it is Visual Studio project with some hacks for compiling it on both x86 and x86_64 architectures. – Bandage

But I have traced my code, step by step. If I put two OutputDebugString() calls. with "message1" and "message2" before and after inflate() respectively - it displays "message1" and DOES NOT display "message2". – Bandage

Answer 1

The code you downloaded is not the original zlib code. It was modified by someone to compile "without warnings and errors" and bugs may have been introduced in the process. You need to download the original and correct code from zlib.net. The code you downloaded has, for example, a commit on May 31, 2014 with the log message "Corrected infinite loop errors in inflate.c and infback.c". You should be more careful about downloading code from strangers.

Note that the data provided in the question is not the result of zlib compression, and would be immediately rejected by inflate() upon reading the first two bytes as not even being a zlib header. You must be using some other input data to get your code to "hang". You need to provide the actual data that caused the issue in order for anyone to be able to help you.

About your code: you do not need to update next_out and avail_out inside the loop as you are doing, since inflate() already does that. You can compute dwUsed when the loop exits as dwSize - zStream.avail_out. You also need to check the return code from inflate() to make sure that it returns Z_STREAM_END when done, otherwise the stream was not complete. You should not abort on Z_BUF_ERROR, but rather provide more output space or more input. See this example for how inflate() and deflate() are used, and read the documentation in zlib.h.

Bonesetter

I tried with dwSize = 5 * 1024 * 1024 and had the same effect. Yes, my data is wrong, because I have captured wrong data. Now I understand it. But I trace my code - and see that execution hangs after the first call to inflate(), e.g. it does not return and execute if (zlib_rv == Z_NEED_DICT). Disassembly shows something like l1: test edi, edi / jne l1. – Bandage

And my code could not go into infinite loop, because I have an error check if (zlib_rv < 0) and break after it. – Bandage

OK, thank you, I didn't notice last changes in that code and didn't clone sources, I took code from "all downloads" (year 2012). Can I compile original zLib code as a static library in Microsoft Visual Studio 2010 for x86_64? – Bandage