pthread_exit vs. return

Question

I have a joinable pthread runner function defined as below:

void *sumOfProducts(void *param)
{
...
pthread_exit(0);
}

This thread is supposed to join the main thread.

Whenever I ran my program through Valgrind I would get the following leaks:

LEAK SUMMARY:
   definitely lost: 0 bytes in 0 blocks
   indirectly lost: 0 bytes in 0 blocks
     possibly lost: 0 bytes in 0 blocks
   still reachable: 968 bytes in 5 blocks
        suppressed: 0 bytes in 0 blocks

ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 15 from 10)

I checked the man page for pthreads which said:

  The new thread terminates in one of the following ways:

   * It  calls  pthread_exit(3),  specifying  an exit status value that is
     available  to  another  thread  in  the  same  process   that   calls
     pthread_join(3).

   * It  returns  from  start_routine().   This  is  equivalent to calling
     pthread_exit(3) with the value supplied in the return statement.

   * It is canceled (see pthread_cancel(3)).

   * Any of the threads in the process calls exit(3), or the  main  thread
     performs  a  return  from main().  This causes the termination of all
     threads in the process.

Miraculously, when I replaced the pthread_exit() with a return statement, the leaks disappeared.

return(NULL);

My actual question is three-pronged:

1. Can someone explain why the return statement gave no leaks?
2. Is there some fundamental difference between both statements, in relation to exiting from threads?
3. If so, when should one be preferred over the other?

Tea

Are you really using C++? C++ uses scope to destroy objects and return would "leave" that scope while pthread_exit won't. – Allegorize

I'm sorry but I never mention C++ anywhere in my question. I'm doing everything in C as of now. – Tea

I know you didn't mention it, but it was a guess, that's why I asked. :) Could you provide a complete test case? – Allegorize

Answer 1

The following minimal test case exhibits the behaviour you describe:

#include <pthread.h>
#include <unistd.h>

void *app1(void *x)
{
    sleep(1);
    pthread_exit(0);
}

int main()
{
    pthread_t t1;

    pthread_create(&t1, NULL, app1, NULL);
    pthread_join(t1, NULL);

    return 0;
}

valgrind --leak-check=full --show-reachable=yes shows 5 blocks allocated from functions called by pthread_exit() that is unfreed but still reachable at process exit. If the pthread_exit(0); is replaced by return 0;, the 5 blocks are not allocated.

However, if you test creating and joining large numbers of threads, you will find that the amount of unfreed memory in use at exit does not increase. This, and the fact that it is still reachable, indicates that you're just seeing an oddity of the glibc implementation. Several glibc functions allocate memory with malloc() the first time they're called, which they keep allocated for the remainder of the process lifetime. glibc doesn't bother to free this memory at process exit, since it knows that the process is being torn down anyway - it'd just be a waste of CPU cycles.

Stinkweed

Is there not a way to force free'ing memory allocated by glibc? Not that it is nessescary, but I think I've seen it mentioned somewhere on this site... – Kneehigh

@Christoffer: Valgrind, since about version 1.1 or so, calls a function named __libc_freeres which is supposed to do this. However, there are some versions of glibc which have bugs in this function (as it's generally not called unless the program is running under a memory debugger). It is usually called by default, unless valgrind is run with the argument --run-libc-freeres=no. – Allies

Thank you! This was driving me crazy. I thought it was a glibc oddity, but I wasn't sure. – Gisborne

Answer 2

Not sure if you're still interested in this, but I am currently debugging a similar situation. Threads that use pthread_exit cause valgrind to report reachable blocks. The reason seems to be fairly well explained here:

https://bugzilla.redhat.com/show_bug.cgi?id=483821

Essentially it seems pthread_exit causes a dlopen which is never cleaned up explicitly when the process exits.

Hypophyge

Answer 3

Are you actually using C++, by any chance? To clarify - your source file ends with a .c extension, and you are compiling it with gcc, not g++?

It seems reasonably likely that your function is allocating resources that you expect to be cleaned up automatically when the function returns. Local C++ objects like std::vector or std::string do this, and their destructors probably won't be run if you call pthread_exit, but would be cleaned up if you just return.

My preference is to avoid low-level APIs such as pthread_exit, and always just return from the thread function, where possible. They're equivalent, except that pthread_exit is a de-facto flow-control construct that bypasses the language you're using, but return doesn't.

Allies

And you're compiling with gcc, and there's no possibility that it or any code it calls could accidentally be using C++ features? – Allies

Rest assured, I AM compiling with gcc. Please look at the tags, before answering. – Tea

@crypto: I do look at the tags. It's not always clear whether people are using a pure C compiler, or whether they are using a C-like subset of C++. In any case, caf's answer is probably more relevant. – Allies

Answer 4

It looks like calling exit() (and, apparently, pthread_exit()) leaves automatically-allocated variables allocated. You must either return or throw in order to properly unwind.

Per C++ valgrind possible leaks on STL string:

@Klaim: I don't see where that document says that I am wrong, but if it does then it is wrong. To quote the C++ standard (§18.3/8): "Automatic objects are not destroyed as a result of calling exit()." – James McNellis Sep 10 '10 at 19:11

Since doing a "return 0" instead of "pthread_exit(0)" seemed to solve your problem (and mine.. thanks), I'm assuming that the behavior is similar between the two.

Marcasite

Answer 5

it's difficult for Valgrind to track pthread_exit variables that's why with pthread_exit it shows memory leak with still reachable tag; but not in the case with return.

Sergiosergipe

Answer 6

I have the experience that valgrind has difficulties of tracking the storage that is allocated for the state of joinable threads. (This goes in the same direction as caf indicates.)

Since it seems that you always return a value of 0 I guess that you perhaps need to join your threads from an application point of view? If so consider of launching them detached from the start, this avoids the allocation of that memory.

The downside is that you either have:

1. to implement your own barrier at the end of your main. If you know the number of threads beforehand, a simple statically allocated pthread_barrier would do.
2. or to exit you main with pthread_exit such that you don't kill the rest of the running threads that might not yet be finished.

Illtreat