Implicit Declaration of Function ‘strtok_r’ Despite Including <string.h>

Question

I have the following code to tokenize a string containing lines separated by \n and each line has integers separated by a \t:

void string_to_int_array(char file_contents[BUFFER_SIZE << 5], int array[200][51]) {
  char *saveptr1, *saveptr2;
  char *str1, *str2;
  char delimiter1[2] = "\n";
  char delimiter2[2] = " ";
  char line[200];
  char integer[200];
  int j;
  for(j = 1, str1 = file_contents; ; j++, str1 = NULL) {
    line = strtok_r(str1, delimiter1, &saveptr1);

    if (line == NULL) {
      break;
    }


    for (str2 = line; ; str2 = NULL) {
      integer = strtok_r(str2, delimiter2, &saveptr2);
      if (integer == NULL) {
        break;
      }
    }
  }
}

(Have included only the relevant function here, the complete, if required, is here.)

However, when I try to compile this code using:

gcc -m64 -std=c99 -pedantic -Wall -Wshadow -Wpointer-arith -Wcast-qual -Wstrict-prototypes -Wmissing-prototypes file_read.c

I get the following warnings:

file_read.c:49:5: warning: implicit declaration of function ‘strtok_r’ [-Wimplicit-function-declaration]
     line = strtok_r(str1, delimiter1, &saveptr1);
     ^
file_read.c:49:10: error: incompatible types when assigning to type ‘char[200]’ from type ‘int’
     line = strtok_r(str1, delimiter1, &saveptr1);
          ^
file_read.c:59:15: error: incompatible types when assigning to type ‘char[200]’ from type ‘int’
       integer = strtok_r(str2, delimiter2, &saveptr2);
               ^

Line nos 49 and 59 correspond to the strtok_r call.

As you can see, I have included string.h in my file (which is where strtok_r is declared), still I get the implicit declaration warning for strtok_r.

Any insights as to how I can remove the warning is appreciated.

I am using gcc 4.8.2 on ubuntu 14.04 64-bit desktop.

Gaunt

Regarding your implicit declaration, strtok_r should be brought in correctly using string.h, but it is a POSIX-2001 feature, and you should ensure it is included. Either #define _POSIX_C_SOURCE 200112L before including any of the standard headers, or provide it via -D option on your command line switches for your compiler. This, of course, assuming your implementation supports it, and I certainly would expect yours would. – Metry

@WhozCraig: I don't see using #define _POSIX_C_SOURCE in the manual page for strtok_r, how do I know I should be doing that? – Gaunt

See the Feature Test Macro Requirements in the man page. I assume you're using glibc on your platform. There are other ways it can be sucked in, some of which may be predefine by default when you set your standard switch (-std=c99 likely won't turn it on, for example, while -std=c11 probably would. haven't tested that but I wouldn't be surprised). – Metry

@WhozCraig: Thanks! Your comment along with nos' answer helped resolve this. Thanks a lot! :) – Gaunt

possible duplicated question. check this #2247118 and this #19389274 – Metamorphic

@JohnSmith: Doesn't seem to look like that, its specifically for kernel modules. – Gaunt

Answer 1

strtok_r is not a standard C function. You have asked for only C99 by using the -std=c99compiler flag, so the header files (of glibc) will only make the standard C99 functions in string.h available to you.

Enable extensions by using -std=gnu99 , or by defining one of the extensions, shown in the manpage of strtok , that supports strtok_r before including string.h. E.g.

#define _GNU_SOURCE
#include <string.h>

Note that the code have other problems too, strtok_r returns a char * , but you are trying to assign that to a char array in integer = strtok_r(str2, delimiter2, &saveptr2);. Your integer variable should be a char *

Buckie

Thanks! Does that mean strtok_r reduces the portability and is available only on linux? – Gaunt

Yes, it might reduce portability. However strtok_r is defined by posix, so at least pretty much all unix like system will have it. – Buckie

@i08in strtok_r definitely reduces portability, as it is a *nix only function, the windows equivalent being strtok_s. – Biodynamics

Answer 2

Same problem with GCC 7.4.2 on Debian

Solved using __strtok_r or -std=gnu99 or adding a protoype after includes:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#define BUFFER_SIZE 1024

extern char *strtok_r(char *, const char *, char **);

Speculum

Answer 3

I am using 4.8.2 and ubuntu 14.04 64bit too. But I got different errors.

    incompatible types when assigning to type 'char[200]' from type 'char *' 
        line = strtok_r(str1, delimiter1, &saveptr1);
             ^

here, line is declared 'char[200]' but strtok_r returns char*.

Similar errors at line for integer = ....

To get rid of these compile errors, you can declare something like char *line;.

For the warning about implicit declaration, check this

Can I use strtok() in a Linux Kernel Module?

Metamorphic

line and integer were char *s previously. I had changed that assuming the seg faults were because of assignment to a char pointer, but changing that didn't help: you can try changing line to a char* instead of char line[200]. Don't you get warnings for implicit declaration of strtok_r when you compile them using the gcc options I have mentioned in the question? – Gaunt

I hope you have used the gcc options I have mentioned in the question, haven't you? – Gaunt

Any reasons you skipped that? I included that as part of what I had read as recommended options I should use while compiling C programs here on SO. – Gaunt

Thanks, you gave a great insight as to the cause, but nos answered helped it solve. – Gaunt

Answer 4

the problem is that the function strtok_r returns a pointer to char, witch you are trying to assign to an array of char, so in order to make it work you have to declare your line and integer variables as a pointer of char, then allocate the memory with malloc.

void string_to_int_array(char file_contents[BUFFER_SIZE << 5], int array[200][51]) {
  char *saveptr1, *saveptr2;

  char *str1, *str2;

  char delimiter1[2] = "\n";
  char delimiter2[] = " ";
  char *line;
  char *integer;
  int j;
  line = (char*)malloc(200);
  integer = (char*)malloc(200);
  for(j = 1, str1 = file_contents; ; j++, str1 = NULL) {
    line = strtok_r(str1, delimiter1, &saveptr1);
    printf("%s\n", line);
    if (line == NULL) {
      break;
    }
    printf("end of first\n");

    for (str2 = line; ; str2 = NULL) {
    printf("begin of second\n");
      printf("asdf%s\n", line);
      integer = strtok_r(str2, delimiter2, &saveptr2);
      if (integer == NULL) {
        break;
      }
      printf("%s\n", integer);
    }
  }
}

Boltonia