What's the difference between strtok and strtok_r in C?

Question

What's the difference between strtok and strtok_r in C and when are we supposed to use which?

Linguini

One is reentrant the other is not. I think the manual states that. – Gretel

Not standard C, it is Posix. Strtok() is a bug factory, it uses a global variable inside the CRT to keep track of the string position. So you can't use it on multiple strings simultaneously. Also risky in threads, most CRTs do solve that. – Astronomical

To confuse you even more, C11 adds strtok_s. – Avis

Answer 1

strtok is equivalent to (and often defined as):

char *strtok(char *str, const char *delim) {
    static char *save;
    return strtok_r(str, delim, &save);
}

in general, you should use strtok_r directly rather than strtok, unless you need to make your code portable to pre-POSIX-2001 systems that only support strtok

Auster

Answer 2

The _r versions of functions are reentrant: you can call them from multiple threads simultaneously, or in nested loops, et cetera. Reentrant versions usually take an extra argument, this argument is used to store state between calls instead of using a global variable.

The non-reentrant versions often use global state, so if you call them from multiple threads, you are probably invoking undefined behavior. Your program could crash, or worse.

From the man pages (man 3 strtok):

The strtok_r() function is a reentrant version of strtok(). The context pointer last must be provided on each call. The strtok_r() function may also be used to nest two parsing loops within one another, as long as separate context pointers are used.

Adalia

Is there a list of such _r functions vs the original ones I may have used without knowing ? Since the _r are not standard C I can't just grep for them in *.h – Jocosity

@dargaud: They are in the *.h files. The functions are not “standard C” but they are part of the POSIX standard. The only other one I can think of is strerror_r. – Adalia

Answer 3

strtok save static pointer for reuse in the next time, when you give NULL as the first parameter, so you just can't parse 2 strings in parallel.

In the strtok_r you give also the pointer, as out parameter (pointer to pointer). so there is no static pointer in the function and you can move from one string to another and back...

Fredenburg

Answer 4

According the documentation, the strtok_r() function is a reentrant version of strtok().

char *strtok_r(char *s1, const char *s2, char **s3);

It gets the next token from string s1, where tokens are strings separated by characters from s2. To get the first token from s1, strtok_r() is called with s1 as its first parameter. Remaining tokens from s1 are obtained by calling strtok_r() with a null pointer for the first parameter. The string of delimiters, s2, can differ from call to call.

Topee