Tokenizing multiple strings simultaneously

Question

Say I have three c-style strings, char buf_1[1024], char buf_2[1024], and char buf_3[1024]. I want to tokenize them, and do things with the first token from all three, then do the same with the second token from all three, etc. Obviously, I could call strtok and loop through them from the beginning each time I want a new token. Or alternatively, pre-process all the tokens, stick them into three arrays and go from there, but I'd like a cleaner solution, if there is one.

Boarder

What if the number of tokens per string don't match? – Allantois

you have 2 buf_1s. probably a mistake – Morale

@jrok, they all have the same number of tokens – Boarder

Answer 1

It sounds like you want the reentrant version of strtok, strtok_r which uses a third parameter to save its position in the string instead of a static variable in the function.

Here's some example skeleton code:

char buf_1[1024], buf_2[1024], buf_3[1024];
char *save_ptr1, *save_ptr2, *save_ptr3;
char *token1, *token2, *token3;

// Populate buf_1, buf_2, and buf_3

// get the initial tokens
token1 = strtok_r(buf_1, " ", &save_ptr1);
token2 = strtok_r(buf_2, " ", &save_ptr2);
token3 = strtok_r(buf_3, " ", &save_ptr3);

while(token1 && token2 && token3) {
    // do stuff with tokens

    // get next tokens
    token1 = strtok_r(NULL, " ", &save_ptr1);
    token2 = strtok_r(NULL, " ", &save_ptr2);
    token3 = strtok_r(NULL, " ", &save_ptr3);
}

Ced