strtok_r for MinGW

Question

strtok_r is the reentrant variant of strtok. It is POSIX-conformant. However, it is missing from MinGW, and I'm trying to compile a program that is using it.

Is there any way I could add a standard implementation of this function, perhaps to the project's own code, or to MinGW's standard library functions?

Naphtha

Answer 1

Since there are some license questions about the code from another answer, here's one that's explicitly public domain:

/* 
 * public domain strtok_r() by Charlie Gordon
 *
 *   from comp.lang.c  9/14/2007
 *
 *      http://groups.google.com/group/comp.lang.c/msg/2ab1ecbb86646684
 *
 *     (Declaration that it's public domain):
 *      http://groups.google.com/group/comp.lang.c/msg/7c7b39328fefab9c
 */

char* strtok_r(
    char *str, 
    const char *delim, 
    char **nextp)
{
    char *ret;

    if (str == NULL)
    {
        str = *nextp;
    }

    str += strspn(str, delim);

    if (*str == '\0')
    {
        return NULL;
    }

    ret = str;

    str += strcspn(str, delim);

    if (*str)
    {
        *str++ = '\0';
    }

    *nextp = str;

    return ret;
}

Alfilaria

Answer 2

Here's the source code which you can simply add to your own library/function in your project:

char *strtok_r(char *str, const char *delim, char **save)
{
    char *res, *last;

    if( !save )
        return strtok(str, delim);
    if( !str && !(str = *save) )
        return NULL;
    last = str + strlen(str);
    if( (*save = res = strtok(str, delim)) )
    {
        *save += strlen(res);
        if( *save < last )
            (*save)++;
        else
            *save = NULL;
    }
    return res;
}

Bourguiba

This is the standard implementation of strtok_r. If str is a on heap then this is thread-safe. – Bourguiba

Is it reasonable to expect str to be on heap? Meaning, is the requirement part of the spec? – Naphtha

Is this your own local implementation or did it come from an open source project? If its the latter, is there any license associated with its use? – Tipperary

libc source. This is very simple. Anyone can implement without ever looking at the source. I don't think there's any license associated with this. – Bourguiba

Sorry if I'm being dim but do you have a link to the source? It seems unlikely to me that it'll have been released without any license but I expect I'll be proved wrong :-) – Tipperary

AFAIK strtok stores a statically allocated pointer to the current token. Every time you call strtok passing a non-null delim you are initializing this pointer, so you may affect another thread that may use strtok at the same time... From the OpenGroup specification: "The strtok() function need not be reentrant. A function that is not required to be reentrant is not required to be thread-safe." (pubs.opengroup.org/onlinepubs/009695399/functions/strtok_r.html). Excuse me if I'm wrong. – Thessalonians

(Continue) Actually, the implementation I posted in my answer does not use strtok. Actually, strtok is a particular case strtok_r that uses the statically allocated pointer I commented. – Thessalonians

Answer 3

Is the FreeBSD implementation any use to you?

Its liberally licensed but integrating it may have some requirements on your project documentation (adding an acknowledgement that the code has been included).

Tipperary

Answer 4

MINGW has no implementation of strtok_r. However you can find a thread-safe implementation in the link below:

http://www.raspberryginger.com/jbailey/minix/html/strtok__r_8c-source.html

Thessalonians