non-copying istringstream

Question

So istringstream copies the contents of a string when initialised, e.g

string moo("one two three four");
istringstream iss(moo.c_str());

I was wondering if there's a way to make std::istringstream use the given c_str as its buffer without copying things. This way, it won't have to copy large bits of memory before passing the std::istringstream& to functions that take istream& as an argument.

What I've been trying to do is convert some functions which only take std::ifstream& arguments (they're mostly parsers) into taking istream& as well. Would I have to make my own istream subclass for this?

Gibert

See this answer for a way to do it (basically identical to @Charles's solution, but wrapping a istream around it for convenience). – Lookeron

Answer 1

It's fairly trivial to write a basic std::streambuf class that reads from a given memory area. You can then construct an istream from this and read from that.

initializing a C++ std::istringstream from an in memory buffer?

Note that the lifetime of the buffer pointed to be c_str() is very limited, though, and there's no guarantee that a call to c_str() want cause some copying although I don't know of any implementations where it does.

Lode

Ah, thanks for that, will give that a try. It looks a lot simpler than what I imagined it would be :) – Gibert

In my case this definitely doesn't work the same as istringstream, solution by @Ozan bellow does work though as expected. In my case code updates get pointer in the istringstream while reading using seekg. In other words proper solution would require seekpos overload as well to match behavior of ostreingstream – Karilynn

Answer 2

Using istringstream is not a satisfactory solution, because this copies the entire buffer.

A previous answer suggests the deprecated istrstream, but as this generates warnings and may be removed in future, a better solution is to use boost::iostreams:

boost::iostreams::stream<boost::iostreams::array_source> stream(moo.c_str(), moo.size());

This avoids copying the buffer in the same way istrstream did, and saves you having to write your own stream class.

Ozan

Answer 3

the deprecated istrstream supports this feature.

#include <string>
#include <strstream>
using namespace std;

int main(int argc, char* argv[])
{
    string moo = "one two three four";
    istrstream istr(const_cast<char*>(moo.c_str()),moo.size());
    std::string line;
    while(!istr.fail() && !istr.eof()){
        getline(istr,line,' ');
        cout << line << "_";
    }
    // prints: one_two_three_four_
}

Reichard

@sbi: my bad, i did some tests with strstream first, was a leftover from that, well its fixed now. – Reichard

Answer 4

There's only a copy because the parameter you pass, a const char*, requires conversion to the argument type of the istringstream constructor.

Just pass in the string without calling c_str().

istringstream iss(moo);

Well ok, that doesn't prevent copying completely, but it eliminates an unnecessary copy. To completely eliminate the copy, you'd have to rewrite std::stringbuf, which specifically avoids working directly on the string you give it.

Hawser

Answer 5

It depends on what a std::string does. According to 27.2.1/1 The class basic_istringstream<charT,traits,Allocator> ... uses a basic_stringbuf<charT,traits,Allocator> object to control the associated storage. Since the class must use an object it must copy construct the string into that object.

So the real question is not whether a stringstream copies the contents, but whether copy constructing a string will copy the contents or implement some sort of copy-on-write scheme.

Swollen

Answer 6

After a few days of searching and trying various methods including scnlib, sscanf, istringstream, etc., maybe I found the answer. If you care about parsing performance, you should use the from_chars family of functions in charsconv header that were added to C++17. This is the only reasonable solution for fast parsing of string_view in C++.

According to cppreference:

Unlike other parsing functions in C++ and C libraries, std::from_chars is locale-independent, non-allocating, and non-throwing. Only a small subset of parsing policies used by other libraries (such as std::sscanf) is provided. This is intended to allow the fastest possible implementation that is useful in common high-throughput contexts such as text-based interchange (JSON or XML).

The problems with the other methods are as follows:

scnlib: I've found that its performance is related to the length of the incoming string, and that even when you're only parsing an integer, passing in a long string causes a linear performance degradation. This is unacceptable for scenarios requiring high performance.

istringstream: As the question says, it can't handle scopes and inevitably requires a copy at initialization time.

sscanf: A C-style null terminated string is necessary, while some implementations may likewise cause performance problems according to cppreference.

Towers