Split string into tokens and save them in an array

Question

How to split a string into an tokens and then save them in an array?

Specifically, I have a string "abc/qwe/jkh". I want to separate "/", and then save the tokens into an array.

Output will be such that

array[0] = "abc"
array[1] = "qwe"
array[2] = "jkh"

please help me

Calctufa

Answer 1

#include <stdio.h>
#include <string.h>

int main ()
{
    char buf[] ="abc/qwe/ccd";
    int i = 0;
    char *p = strtok (buf, "/");
    char *array[3];

    while (p != NULL)
    {
        array[i++] = p;
        p = strtok (NULL, "/");
    }

    for (i = 0; i < 3; ++i) 
        printf("%s\n", array[i]);

    return 0;
}

Krystin

If i want to compare array[0] with array [1], what should i do? – Calctufa

You use another function from <string.h>: if (strcmp(array[0], array[1]) == 0) { // array[0] = array[1] } – Krystin

This assumes the number of tokens is know. – Incorporated

What is the purpose of the line p = strtok (NULL, "/");? – Zingaro

strtok() does not work well in multi-threaded programs, libraries nor in some other programs that may call strtok() a second time with a different strings without finishing the loop before. strtok_r() could be a better solution or write your own function when strtok_r() is not available. – Bandoline

Answer 2

You can use strtok()

char string[] = "abc/qwe/jkh";
char *array[10];
int i = 0;

array[i] = strtok(string, "/");

while(array[i] != NULL)
   array[++i] = strtok(NULL, "/");

Brachium

A null pointer may be specified as the first argument of strtok(), in which case the function continues scanning where a previous successful call to the function ended. – Jodyjoe

Worked pretty good. just how to give a dynamic count if we dont know the numbers of slpit element.. – Porscheporsena

Cant arrays not be initialized with a variable? – Gawk

This puts 4 elements into the array, with NULL as the last one. – Mechanics

strtok() is not a good solution, because it requires variables that are not on the stack nor heap. – Bandoline

@Bandoline And where are those variables then? 🤣 – Neper

@Neper static variables, this causes problems when strtok() is called from multiple places. See my answer. – Bandoline

Answer 3

Why strtok() is a bad idea

Do not use strtok() in normal code, strtok() uses static variables which have some problems. There are some use cases on embedded microcontrollers where static variables make sense but avoid them in most other cases. strtok() behaves unexpected when more than 1 thread uses it, when it is used in a interrupt or when there are some other circumstances where more than one input is processed between successive calls to strtok(). Consider this example:

#include <stdio.h>
#include <string.h>

//Splits the input by the / character and prints the content in between
//the / character. The input string will be changed
void printContent(char *input)
{
    char *p = strtok(input, "/");
    while(p)
    {
        printf("%s, ",p);
        p = strtok(NULL, "/");
    }
}

int main(void)
{
    char buffer[] = "abc/def/ghi:ABC/DEF/GHI";
    char *p = strtok(buffer, ":");
    while(p)
    {
        printContent(p);
        puts(""); //print newline
        p = strtok(NULL, ":");
    }
    return 0;
}

You may expect the output:

abc, def, ghi,
ABC, DEF, GHI,

But you will get

abc, def, ghi,

This is because you call strtok() in printContent() resetting the internal state of strtok() generated in main(). After returning, the content of strtok() is empty and the next call to strtok() returns NULL.

What you should do instead

You could use strtok_r() when you use a POSIX system, this versions does not need static variables. If your library does not provide strtok_r() you can write your own version of it. This should not be hard and Stackoverflow is not a coding service, you can write it on your own.

Bandoline

the OP never mentioned that they are in a multithreaded environment – Egg

@sijanec He also did not say he isn't. Please keep in mind that libraries should be multi threading save and putting a strtok() in a library is a bad idea. It is not only a problem in multithreading applications, as i wrote in my answer. Please read before you comment, thank you. – Bandoline