Block-level copying of data between streambuffers

Question

I would like to copy data efficiently between std::streambuf instances. That is, I would like to shovel blocks of data between them, as opposed to perform character-by-character copying. For example, this is not what I am looking for:

stringbuf in{ios_base::in};
stringbuf out{ios_base::out};
copy(istreambuf_iterator<char>{in},
     istreambuf_iterator<char>{},
     ostreambuf_iterator<char>{out});

There exists syntactic sugar for this, with a bit more error checking:

ostream os{&out};
os << &in;

Here's a snippet of the implementation of operator<<(basic_streambuf<..>*) in my standard library (Mac OS X, XCode 7):

                typedef istreambuf_iterator<_CharT, _Traits> _Ip;
                typedef ostreambuf_iterator<_CharT, _Traits> _Op;
                _Ip __i(__sb);
                _Ip __eof;
                _Op __o(*this);
                size_t __c = 0;
                for (; __i != __eof; ++__i, ++__o, ++__c)
                {
                    *__o = *__i;
                    if (__o.failed())
                        break;
                }

The bottom line is: this is still per-character copying. I was hoping the standard library uses an algorithm that relies on the block-level member functions of streambuffers, sputn and sgetn, as opposed to per-character transport. Does the standard library provide such an algorithm or do I have to roll my own?

Chronopher

The problem is that this is based on an interface with virtual functions. You never know when *__o = *__i will fail to output, so you cannot read ahead and risk losing those characters. – Katleen

did you find the method? – Cog

Answer 1

I'm afraid that the answer is: it is not possible with the current design of the standard library. The reason is that streambuffers completely hide the character sequence they manage. This makes it impossible to directly copy bytes from the get area of one streambuffer to the put area of another.

If the "input" streambuffer would expose its internal buffer, then the "output" streambuffer could just use sputn(in.data(), in.size()). Or more obviously: if the output buffer also exposed its internal buffer, then one could use plain memcpy to shovel bytes between the two. Other I/O libraries operate in this fashion: the stream implementation of Google's Protocol Buffers, for example. Boost IOStreams has an optimized implementation to copy between streams. In both cases, efficient block-level copying is possible because the streambuffer equivalent provides access to its intermediary buffer.

In fact, streambuffers ironically do not even need to have a buffer: when operating unbuffered, each read/write goes directly to the underlying device. Presumably this is one reason why the standard library does not support introspection. The unfortunate consequence is that no efficient copying between input and output streambuffers is possible. Block-level copying requires an intermediary buffer, and a copy algorithm would operate as follows:

1. Read from the input streambuffer via sgetn into the intermediary buffer.
2. Write from the intermediary buffer into the output streambuffer via sputn.
3. Go to 1. until input is exhausted or writes to the output streambuffer fail

Chronopher