How to bind program termination with end-of-stream in Boost.Process 0.5?

Question

In this simple example of Boost.Process 0.5 ( http://www.highscore.de/boost/process0.5/index.html) the output of a program (ls) is feeding a stream. The stream works fine but contrary to the expectation the stream doesn't become invalid (e.g. end-of-stream) after the program finishes (similar to previous version of Boost.Process, e.g. http://www.highscore.de/boost/process/index.html)

What am I missing in order to make the stream (is in the example) automatically invalid after child program exits?

Perhaps is it an option that I have to set in the Boost.Streams stream of file_descriptor?

#include <boost/process.hpp> // version 0.5 from http://www.highscore.de/boost/process0.5/process.zip
#include <boost/iostreams/device/file_descriptor.hpp>
#include <boost/iostreams/stream.hpp>
#include <string>
using namespace boost::process;
using namespace boost::process::initializers;
using namespace boost::iostreams;
int main(){
    boost::process::pipe p = create_pipe();
    file_descriptor_sink sink(p.sink, close_handle);
    child c = execute(run_exe("/usr/bin/ls"), bind_stdout(sink));
    file_descriptor_source source(p.source,  close_handle);
    stream<file_descriptor_source> is(source);
    std::string s;
    while(std::getline(is, s)){
        std::cout << "read: " << s << std::endl;
    }
    std::clog << "end" << std::endl; // never reach
}

Keeping

Just an unrelated tip: You don't need std::getline(is, s) and is in the loop condition, just std::getline(is, s) is enough. – Cenogenesis

@Joachim, thanks, I left it there after trying different tricks, I edited the question, so it is less confusing. – Keeping

Answer 1

---

UPDATE 2020: Boost.Process is now part of Boost https://www.boost.org/doc/libs/1_74_0/doc/html/process.html and this answer might be completely out-of-date. It only applies to an experimental version "0.5" http://www.highscore.de/boost/process0.5/index.html.

---

I had a private (actually through Nabble) communication with Boris Schaeling, the author of the library. After discarding several possibilities, like bugs in posix/boost.iostreams, he gave me a slight modification of the code that works. Basically, what I can deduce is that the file_descriptor sink must be out of scope (destroyed) in order for the stream to return an EOF. The working code simply adds a specific scope for sink (listed at the end). I think this makes easy to encapsulate all in a pistream kind of class. (Next step in my list will be to allow also output to the process.)

Works with Boost 1.48 (Fedora 17).

#include <boost/process.hpp> // version 0.5
#include <boost/iostreams/device/file_descriptor.hpp>
#include <boost/iostreams/stream.hpp>
#include <string>

using namespace boost::process;
using namespace boost::process::initializers;
using namespace boost::iostreams;

int main() {
    pipe p = create_pipe();
    {
        // note the scope for sink
        file_descriptor_sink sink(p.sink, close_handle);
        /*  child c = */ // not necessary to hold a child object, it seems.
        execute(run_exe("/usr/bin/ls"), bind_stdout(sink));
    }   // note the scope for sink

    file_descriptor_source source(p.source,  close_handle);
    stream<file_descriptor_source> is(source);
    std::string s;
    while(std::getline(is, s)) {
        std::cout << "read: " << s << std::endl;
    }
    std::clog << "end" << std::endl; // never reach
}

Compiles with c(lang)++ -lboost_system -lboost_iostreams

EDIT: This seems to work as well, which avoid the artificial scope, but can be confusing because the sink has to be a temporary:

    ...
    pipe p = create_pipe();
    execute(run_exe("/usr/bin/ls"), bind_stdout(        
        file_descriptor_sink(p.sink, close_handle)
    ));
    file_descriptor_source source(p.source,  close_handle);
    ...

Keeping

This is truly subtle behaviour. Thanks for sharing that knowledge. – Hydroxide

Answer 2

This works on POSIX-like systems:

#include <boost/process.hpp> // version 0.5 from http://www.highscore.de/boost/process0.5/process.zip
#include <boost/iostreams/device/file_descriptor.hpp>
#include <boost/iostreams/stream.hpp>
#include <boost/asio.hpp>
#include <string>

using namespace boost::process;
using namespace boost::process::initializers;
using namespace boost::iostreams;

int main()
{
    boost::asio::io_service io_service;
    int status = 1;
    boost::asio::signal_set set(io_service, SIGCHLD);
    set.async_wait(
        [&status](const boost::system::error_code&, int) { ::wait(&status); }
    );

    boost::process::pipe p = create_pipe();
    file_descriptor_sink sink(p.sink, close_handle);
    child c = execute(run_exe("/bin/ls"), bind_stdout(sink));
    file_descriptor_source source(p.source,  close_handle);
    stream<file_descriptor_source> is(source);
    std::string s;
    while(status && std::getline(is, s))
    {
        std::cout << "read: " << s << std::endl;
    }
    std::clog << "end" << std::endl; // never reach
}

Note that it handles SIGCHLD to asynchronously set status. It was taken from http://www.highscore.de/boost/process0.5/boost_process/tutorial.html#boost_process.tutorial.starting_a_program. This page also shows the Windows style of doing the same:

#if defined(BOOST_WINDOWS_API)
    DWORD exit_code;
    boost::asio::windows::object_handle handle(io_service, c.process_handle());
    handle.async_wait(
        [&handle, &exit_code](const boost::system::error_code&)
            { ::GetExitCodeProcess(handle.native(), &exit_code); }
    );
#endif

    io_service.run();

Hydroxide

This Boost.Process library is getting overly complicated, so I have to run the process asynchronously and check the status of the program separately from the stream. That is crazy! it also makes the syntax of the loops incompatible with other streams. Not only that, now it is necessary to link with boost_streams and boost_asio. It was much simpler in Boost.Process 0.3, not to mention the good old pstreams.sourceforge.net. – Keeping

Erm. I can't really help that, I'm afraid. Also, you could just hide the extra complexity if you need to. Also, Boot Asio is header-only, and as such does not introduce an extra dependency. (wrap it in a class?). Sorry if my answer made you angry. Feel free to use something else entirely, of course :) – Hydroxide

As for wrapping it will be lots of work because I have to create a class that behaves almost like the stream that is wrapping, this will be a headache and because I am unfamiliar with the science behind streams I'll do a poor job probably. I'll stick with Boos.Process 0.3 (or GSOC2010) for the moment, maybe this feature I am asking for is an involuntary omission in the last version of the library as it seems. The historical behaviour is that exiting programs return EOF state (think of any posix pipe). – Keeping

what version of Boost/Boost.Iostream are you using to compile your code? – Keeping

@Keeping I just tried with 1_51, could have been 1.46-1.51 earlier, though – Hydroxide

I found the correct solution (see accepted answer). I awarded the bounty to your answer for the effort. I can't vote up because I wasn't able to test it as I couldn't install/link boost-asio easily in Fedora 17. – Keeping

Cheers. Thanks for putting in more effort and coming up with a less contrived solution. I might be using Boost Process in the future. It looks nice. – Hydroxide

+1 I finally was able to test the code and it works (compiled with c(lang)++ -std=c++11 -lboost_system -lboost_iostreams -lpthread. (gcc 4.9.2, clang 3.5, boost 1.55, Fedora 22) – Keeping