Valgrind reporting a segment overflow

Question

When running my program with valgrind / callgrind I get the following message a lot:

==21734== brk segment overflow in thread #1: can't grow to 0x4a39000 (with different addresses)

Note that it is not preceded by a stack overflow message.

I can't find any documentation on this message and I have no idea what is overflowing exactly.

Can anybody help me figure out what the problem is? Is this a problem of valgrind, or of my program?

Gott

brk is a system call that allocates memory for a process by changing the size of the data segment. Failure to grow the data segment implies to me, that this error is about memory running out. But I'm not sure. – Slifka

Does this cause your program to fail? – Historicism

@Historicism If I'm right the program executes normally - though callgrind makes it awefully slow. – Gott

Do I need to provide valgrind with some command line arguments to make the amount of memory allocated to the program higher? – Gott

repo.or.cz/valgrind.git/blob/HEAD:/coregrind/m_syswrap/… line 1322, hopefully the surrounding comments can shed some light onto this (just a suggestion, i'm no expert on valgrind) – Prospect

also, this seems to be the commit where the message first appeared: sourceforge.net/p/valgrind/mailman/message/34068401 the commit message reads "Issue an error message if then brk segment overflows." – Prospect

Ah my RAM was swamped. I closed some apps and changed the free RAM from 64k to 1300k. I'm still getting the warning messages though; perhaps I need to restart the valgrind run. – Gott

Answer 1

Line 1327 from the valgrind source code points to the user manual, "see section Limitations in user manual":

Limits section item 1:

On Linux, Valgrind determines at startup the size of the 'brk segment' using the RLIMIT_DATA rlim_cur, with a minimum of 1 MB and a maximum of 8 MB. Valgrind outputs a message each time a program tries to extend the brk segment beyond the size determined at startup. Most programs will work properly with this limit, typically by switching to the use of mmap to get more memory. If your program really needs a big brk segment, you must change the 8 MB hardcoded limit and recompile Valgrind.

Thylacine

Has anyone found where to change this hardcoded limit to recompile? Also what are reasonable values to change it to? – Immunology

I still don't understand what a brk segment is and I still don't know whether this means there is a problem in my program or in valgrind or in the capabilities of my computer. – Gott

@TimKuipers See brk(2). It's a low level syscall for memory. You wouldn't normally use it. Check the man page out and it might help clear things up at least a bit. – Jael

Answer 2

Valgrind only allocates 8MB for the brk segment, which runs out. One reports that libc is then switching to a mmap-based memory allocation in the valgrind bugreport discussing this.

Radiculitis

So, to be a bit more specific: If your application runs on linux and uses the usual C malloc() function and/or C++ new operator for memory allocation, this valgrind message can safely be ignored, as the malloc library will automatically switch to mmap() instead. If you use your own allocator, that only supports brk() / cbrk(), then adding mmap() support to that allocator is maybe still easier than recompiling valgrind. – Fontaine

Answer 3

While this is not really an answer, it still satisfies OP's "couldn't find any docs" requirement:

1) http://repo.or.cz/valgrind.git/blob/HEAD:/coregrind/m_syswrap/syswrap-generic.c

contains the message discussed at line 1322

2) http://sourceforge.net/p/valgrind/mailman/message/34068401/

is the commit that introduced the feature, and the corresponding commit message reads

Author: florian
Date: Wed Apr 29 13:59:16 2015
New Revision: 15155

Log: Issue an error message if then brk segment overflows.

from where we can further relay this question on to those who can give a qualified answer to "what exactly does "a brk segment overflows" mean in this context"!

Prospect

Answer 4

Adding to Piwi's answer, sometimes your program will require Callgrind to use a bigger brk segment (up to GBs, depending on your implementation).

To modify the hardcoded limit, go to function VG_(ii_create_image) in coregrind/m_initimg/initimg-linux.c (around line 1000), change the following lines according to your needs

   //--------------------------------------------------------------
   // Setup client data (brk) segment.  Initially a 1-page segment
   // which abuts a shrinkable reservation.
   //     p: load_client()     [for 'info' and hence VG_(brk_base)]
   //--------------------------------------------------------------
   {
      SizeT m1 = 1024 * 1024;
      SizeT m8 = 8 * m1;
      SizeT dseg_max_size = (SizeT)VG_(client_rlimit_data).rlim_cur;
      VG_(debugLog)(1, "initimg", "Setup client data (brk) segment\n");
      if (dseg_max_size < m1) dseg_max_size = m1;
      if (dseg_max_size > m8) dseg_max_size = m8;
      dseg_max_size = VG_PGROUNDUP(dseg_max_size);

      setup_client_dataseg( dseg_max_size );
   }

and rebuild valgrind.

m8 is the max brk segment size that callgrind will try to allocate

There are similar pieces of code for FreeBSD and Solaris, but not macOS.

Aeniah

I ran into the same error. The above approach is not working. It either segfaults or reports the brk segment error. – Warmup

I did change it to SizeT m8 = 100 * m1, recompiled, reinstalled and it did in fact help to get rid of the error message (tested with valgrind 3.13.0). Note that it's m16 instead of m8 in valgrind 3.16.1. – Naresh

@Naresh No, the 16M limit is for the stack rlimit, not the data rlimit. – Scud

Answer 5

Is this a problem of valgrind, or of my program?

I am unsure of the reason, but I think you can ignore it. At least it seems to be possible to trigger it with legal programs. I answered a similar/dublicate with an example here:

Valgrind reporting "brk segment overflow in thread #1"

Overrate