Need help in strtok function when the first token is null

Question

Need help in strtok function

#include<stdio.h>
#include<string.h>

int main()
{
    char string[100], *ptr = NULL;

    memset(string, 0, 100);
    strcpy(string, "abc#efg#xyz");

    ptr = strtok(string, "#");
    fprintf(stderr, "ptr = [%s]\n", ptr);

    ptr = strtok(NULL, "#");
    fprintf(stderr, "ptr = [%s]\n", ptr);

    ptr = strtok(NULL, "#");
    fprintf(stderr, "ptr = [%s]\n", ptr);
    return 0;
}

The output is

ptr = [abc]
ptr = [efg]
ptr = [xyz]

This is fine, but when if the first token is null then the first call to strtok returns the second token. My understanding is that it will return a null in the first call as the token is not present.

#include<stdio.h>
#include<string.h>

int main()
{
    char string[100], *ptr = NULL;

    memset(string, 0, 100);
    strcpy(string, "#efg#xyz");

    ptr = strtok(string, "#");
    fprintf(stderr, "ptr = [%s]\n", ptr);

    ptr = strtok(NULL, "#");
    fprintf(stderr, "ptr = [%s]\n", ptr);

    ptr = strtok(NULL, "#");
    fprintf(stderr, "ptr = [%s]\n", ptr);
    return 0;
}***

The output is

ptr = [efg]
ptr = [xyz]
ptr = [(null)]

Whinny

Read: strtok() source code – Bradberry

Answer 1

From http://www.cplusplus.com/reference/cstring/strtok/

To determine the beginning and the end of a token, the function first scans from the starting location for the first character not contained in delimiters

So I think that this is the correct behavior you are seeing. strtok() starts from your first character, see's it as a delimeter, ignores it, and seeks forward for the first non-delimiter character.

I guess the way around it is to check that the first character is not a delimiter yourself.

Checked C standard (ISO/IEC 9899:1999(E)) too and the definition is the same:

The first call in the sequence searches the string pointed to by s1 for the first character that is not contained in the current separator string pointed to by s2

In fact, the standard gives an example in 7.21.5.8, bullet point 8:

#include <string.h>
static char str[] = "?a???b,,,#c";
char *t;
t = strtok(str, "?"); // t points to the token "a"

Macronucleus

Answer 2

According to the strtok manual page (from man strtok):

A sequence of two or more contiguous delimiter characters in the parsed string is considered to be a single delimiter. Delimiter characters at the start or end of the string are ignored. Put another way: the tokens returned by strtok() are always nonempty strings.

Vaunt

I was to quote the same sntence at the same moment. Congratulations, you won :-) – Cordiform