Param syntax for substituting boost filtering_stream for std::ofstream

Question

Some basic questions about boost filtering_streams. I have dozens of functions that take a parameter of std::ofstream&

void foo(std::ofstream& outStream)
{
    // lots of operations, like this:
    outStream << "various bits of text";
}

void StreamSomeTextToFile(char* fileName)
{
    ofstream myFileStream(fileName, ios::out | ios::app | ios::binary);
    foo(myFileStream);
    myFileStream.close();
}

Now I'd like to use the boost filtering_stream to output to a compressed ZIP file. The commonly cited boost filtering_streams test code for packing and unpacking compiled, linked, and worked perfectly for me. I'd like to substitute the filtering_stream:

void StreamSomeCompressedTextToFile(char* fileName)
{
    ofstream myFileStream(destPath, std::ios_base::out | std::ios_base::app | std::ios_base::binary);
    boost::iostreams::filtering_streambuf<boost::iostreams::output> myCompressedFileStream;
    myCompressedFileStream.push(boost::iostreams::zlib_compressor());
    myCompressedFileStream.push(myFileStream);

    foo(myCompressedFileStream);    // I can't just pass myCompressedFileStream to foo(std::ofstream&), right?
    myFileStream.close();
}

THREE QUESTIONS:

1) Do all my functions that previously accepted std::ofstream& outStream need to now accept a parameter of type boost::iostreams::filtering_streambuf& ? Or is there a proper parameter type so those numerous ("foo") functions could work with EITHER type of stream type?

2) In my simple test cases, I was not able to use stream operator syntax with the filtering_streambuf:

myCompressedFileStream << "some text";

this generated the the error: no match for 'operator<<'. I similarly had compile errors with write():

error: 'class boost::iostreams::filtering_streambuf<boost::iostreams::output, char, std::char_traits<char>, std::allocator<char>, boost::iostreams::public_>' has no member named 'write'

3) In the common test case example code (below), I was confused that I could not locate the file "hello.z" after it had been created. The unpack code (also below) clearly references it -- so where can it be found? NOTE: the location was finally discovered: it was in the /Library/Preferences/

void pack()
{            
    std::ofstream file("hello.z", std::ios_base::out | std::ios_base::binary);

    boost::iostreams::filtering_streambuf<boost::iostreams::output> out;
    out.push(boost::iostreams::zlib_compressor());
    out.push(file);       
    char data[5] = {'a', 'b', 'c', 'd', 'e'};    
    boost::iostreams::copy(boost::iostreams::basic_array_source<char>(data, sizeof(data)), out);
    file.close();
}

void unpack()
{
    std::fstream file("hello.z", std::ios_base::in | std::ios_base::binary);
    boost::iostreams::filtering_streambuf<boost::iostreams::input> in;
    in.push(boost::iostreams::zlib_decompressor());
    in.push(file);
    boost::iostreams::copy(in, std::cout);
}

BTW: XCode 3.2.6, GNU 4.0, OS X 10.6.8

Mechling

Answer 1

Taking the questions in order:

1: Stream buffer objects (like boost::iostream::filtering_streambuf or std::streambuf) are not interchangeable with stream objects (such as std::ostream or boost's implementation). That being said, you can pass a streambuf object like "myCompressedFileStream" to a constructor of an ostream object (this boost iostream tutorial provides a decent explanation with examples). And because boost's streambufs are compatible with those in the standard library, you need not change any of the functions accepting std::ostream/ofstream references. You just can't pass streambufs as streams.

2: Same as above, the insertion operator is defined for streams, not streambufs.

3: Normally, files without a preceding directory name are created in the directory of the executable. That being said, I've found at times Finder has been somewhat slow reflect files updated/created by non-Finder processes. I didn't experience those problems in Terminal using ls. No idea if that's related to your problem, though.

Lylalyle

This was great, and VERY helpful -- but there's still one aspect I'm still confused about: constructing an ostream object from the filtering_streambuf: the filtering_streambuf (myCompressedFileStream) requires an ofstream defined first (myFileStream) so it can be pushed. If I simply construct an ostream from myCompressedFileStream, and then output to that newly-constructed ostream, then the file is empty. – Mechling

Answer 2

SUCCESS!

A combination of hints from Paul Schellin (above) and several on Boost-users resulted in the answer:

1) Boost User Frédéric pointed out that "nothing happen[s] until output_file [filtering_ostream] is destroyed. So enclose in { }". This was the essential missing piece, because I was trying to do file.close() on my ofstream BEFORE my filtering_streambuf was destroyed. That explained why the file was empty!

Re-reading the documentation revealed:

  "By default, if the Device at the end of the chain is popped
   or if the filtering_stream is complete when it is destroyed,
   all the filters and devices in the chain are closed using the
   function close. This behavior can be modified using the member
   function set_auto_close"

This states is there is no need to "pop" the compressor() or the ofstream (file) off the filtering_stream's stack, nor to call close(). Just destruct the filtering_stream object, and everything gets written out and cleaned up. An obscure detail, and one that goes counter to what one might expect.

3) Boost User Holger Gerth questioned why I was using filtering_streambuf when I could've been using filtering_stream. Truth is, I wasn't sure, however, in my experiments I could neither construct the ostream (which I required to pass to other functions) from the filtering_stream, nor could I pass the filtering_stream in place of the ostream I required.

Even after reading several articles on filtering_streambuf vs filtering_stream, I'm still mystified how and why (FOR MY PURPOSE) I would use the filtering_stream over constructing an ostream from a filtering_streambuf.

SO, TO RECAP:

1) Construct a separate ostream from the filtering_streambuf, and pass THAT to foo() or to the Stream Insertion operator (i.e. <<).

2) Don't call myFileStream.close();

Mechling