malloc() returning address that I cannot access

Question

I have a C++ program that calls some C routines that are generated by Flex / Bison.

When I target a Windows 8.1 64-bit platform, I hit the following exception at runtime:

Unhandled exception at 0x0007FFFA70F2C39 (libapp.dll) in application.exe: 0xC0000005: 
Access violation writing location 0x000000005A818118.

I traced this exception to the following piece of code:

YY_BUFFER_STATE yy_create_buffer( FILE *file, int size )
{
   YY_BUFFER_STATE b;
   b = (YY_BUFFER_STATE) yy_flex_alloc( sizeof( struct yy_buffer_state ) );
   if ( ! b )
      YY_FATAL_ERROR( "out of dynamic memory in yy_create_buffer()" );
   b->yy_buf_size = size; // This access is what throws the exception
}

For reference, elsewhere in the code (also generated by Flex / Bison), we have:

typedef struct yy_buffer_state *YY_BUFFER_STATE;
struct yy_buffer_state
{
   FILE *yy_input_file;
   char *yy_ch_buf;
   char *yy_buf_pos;
   yy_size_t yy_buf_size;
   // ... other fields omitted,
   // total struct size is 56 bytes
}

static void *yy_flex_alloc( yy_size_t size )
{
   return (void *) malloc( size );
}

I traced back to the malloc call and observed that malloc itself is returning the address 0x000000005A818118. I also checked errno, but it is not set after the call to malloc.

My question is: why does malloc give me an address that I don't have access to, and how can I make it give me a correct address?

Note: I only observe this behavior in Windows 8.1 64-bit. It passes with other 32-bit Windows variants, as well as Windows 7 32-bit.

Compilation information: I am compiling this on a 64-bit Windows 8.1 machine using Visual Studio 2012.

If it helps, here is the disassembled code:

// b = (YY_BUFFER_STATE) yy_flex_alloc( ... )
0007FFFA75E2C12  call   yy_flex_alloc (07FFFA75E3070h)
0007FFFA75E2C17  mov    qword ptr [b],rax
// if ( ! b ) YY_FATAL_ERROR( ... )
0007FFFA75E2C1C  cmp    qword ptr [b],0
0007FFFA75E2C22  jne    yy_create_buffer+30h (07FFFA75E2C30h)
0007FFFA75E2C24  lea    rcx,[yy_chk+58h (07FFFA7646A28h)]
0007FFFA75E2C2B  call   yy_fatal_error (07FFFA75E3770h)
// b->yy_buf_size = size
0007FFFA75E2C30  mov    rax,qword ptr [b]
0007FFFA75E2C35  mov    ecx,dword ptr [size]
0007FFFA75E2C39  mov    dword ptr [rax+18h],ecx

Thanks!

Disjuncture

I reran this segment several times, malloc either returns 00000000-------- or FFFFFFFF-------- for the address of the "allocated" space. – Disjuncture

Most likely, some other, seemingly unrelated part of the code has managed to corrupt the heap (e.g. by overrunning a heap-allocated buffer). The part of code you are looking at is an innocent victim. – Diagnostician

Your program is experiencing the Heap Corruption scenario. You may refer the post on this article : https://mcmap.net/q/1331017/-heap-corruption-while-using-createwindowexw – Kingery

Does the file where yy_flex_alloc is defined include stdlib? If not, it's likely that it treats the return value from malloc as an int and the cast hides it. – Didymous

@Art: the file where yy_flex_alloc() is defined indeed contains #include <stdlib.h>. This file is generated automatically by Flex. – Disjuncture

Some versions of Windows use lazy allocation (IDK whether Windows 8.1 does). When lazy allocation is enabled, a malloc can succeed even if there is not enough virtual memory to answer the request. But you only get an access violation when you actually write to a part of the requested memory that wasn't allocated yet and still can't be allocated. – Ferdy

Make sure that you are not doing free in the C++ part of your program for memory that was allocated by malloc in the C part, and vice versa – Ferdy

return (void *) malloc( size ); should be return malloc(size); . Make sure that the file actually does #include <stdlib.h> . Also make sure that you do not try to compile C files with the C++ compiler. – Ferdy

Answer 1

The real answer is:

When you are compiling flex-generated .c source in Visual Studio, it doesn't include stdlib.h (where malloc defined as returning void*) and Visual Studio takes some own definition, where malloc returns int. (I think it's for some kind of compatibility)

Visual studio prints:

'warning C4013: 'malloc' undefined; assuming extern returning int' sizeof(int)==4, but values in pointers on x64 systems often exceed 4 bytes

So your pointer just cut to low 4 bytes.

It seems this problem appears only in x64 bits visual studio in .c files.

So, solution will be - just include stdlib.h by yourself, or define some macros, which will lead in flex-generated source to including stdlib.h.

Caritacaritas

I can only imagine how many hours you've just saved me with this answer. – Aloysius

thankyou hostagebrian and laughing-man(Morales) : very nice answer – Fur

This is definitely the issue I met -- after I struggled almost all day long. Thank you so much! – Roby

Answer 2

Under normal circumstances malloc() will return a pointer to valid, accessible memory or else NULL. So, your symptoms indicate that malloc() is behaving in an unspecified way. I suspect that, at some point earlier, your program wrote outside of its valid memory, thereby corrupting the data structures used internally by malloc().

Examining your process with a run-time memory analysis tool should help you identify the source of the issue. [See this post for suggestions on memory analysis tools for Windows: Is there a good Valgrind substitute for Windows? ]

Bibliomania

Thanks, this is a good suggestion. I will look into it as soon as I can install a Windows 8 compatible leak detector... I did run Valgrind on the Linux version of this application though, and there didn't seem to be anything suspicious. By the time this part of the code is run, I lost only 4 bytes (due to a library function that I can't change...) – Disjuncture