c++ strtok in function changes original string value as parameter

Question

when I use strtok to tokenize a c++ string, it happens a confusing problem, see the simple code below:

void a(string s){
    strtok((char*)s.c_str(), " ");
}
int main(){
    string s;
    s = "world hello";
    a(s);
    cout<<s<<endl;
    return 0;
}

the program outputs "world". Shouldn't it output "world hello"? Because I pass the string as a value parameter to function a, the strtok shouldn't modify the original s... Can anyone explain this trick. thank you.

Nissie

Don't do that, you're asking for memory corruption issues. Yes, strtok does alter it's input. cplusplus.com/reference/clibrary/cstring/strtok – Viens

Answer 1

The problem is (char*)s.c_str(), you are casting the constness away and modified the string contents in a way that you are not supposed to. While the original s should not be modified, I pressume you may have been hit by a smart optimization that expects you to play by the rules. For instance, a COW implementation of string would happen to show that behavior.

Quevedo

thx. Btw, do you know how to convert a c++ string to char* safely and efficiently, so i can use char* in the strtok without problems? – Nissie

@zwx: If you intend to modify the char* then you need to make a copy of it. – Quevedo

Answer 2

c_str() returns a const pointer, which is a promise to the compiler that the thing being pointed at won't be modified. And then you're calling strtok which modifies it.

When you lie to the compiler, you will be punished.

Vend

An std::string has no problems with nulls. – Quevedo

Answer 3

That's the way strtok() works. It use the first parameter as a buffer. By casting it to a char*, you allow it to modify the string. strtok() does not known about the original std::string. It also store the string pointer in a static variable, that's why you have to call it with a null pointer the next times to continue to parse the same string.

By the way, in c++, you should use std::istringstream instead. It does not use an internal static variable, which is not thread-safe. And you can extract the parameters directly into int, double, etc like we do with cin. std::ostringstring replace sprintf().

Slavery