boost::process::child will not exit after closing input stream

Question

In the following example I try to write some data to a child process, which processes the data and writes it to a file. After closing the stream the parent process waits indefinitely for the child to finish. I am at a loss to know how to indicate that I’m done writing the data and would like the child process to stop reading and finish whatever it is doing. According to the documentation calling terminate would send a SIGKILL which I don’t think is what I want.

What am I missing? I checked this question but I would rather try to make the actual code work with synchronous IO first.

#include <boost/process.hpp>
#include <iostream>


namespace bp = boost::process;


int main(int argc, char **argv)
{
    boost::process::opstream in{};
    boost::process::child child("/path/to/test.py", bp::std_in < in);

    in << "test1\n";
    in << "test2\n";
    in << "test3\n";
    in << std::flush;

    std::cerr << "Closing the stream…\n";
    in.close();
    std::cerr << "Waiting for the child to exit…\n";
    child.wait(); // Parent seems to hang here.

    return 0;
}

test.py just writes the data to a file like so:

#!/usr/local/homebrew/opt/[email protected]/bin/python3

import sys

with open("/tmp/test.txt", "w") as f:
    for line in sys.stdin:
        f.write(line)

Unripe

Answer 1

After inspecting the source code, I found out that closing the stream did not close the associated pipe at least in this case. Doing that manually did solve the issue:

...
in.close();
in.pipe().close();
child.wait(); // Does not hang.

Unripe

Answer 2

The documentation warns that using synchronous IO to child processes is prone to deadlock.

Here's a minimal reword to async IO:

#include <boost/process.hpp>
#include <iostream>

namespace bp = boost::process;

int main() {
    boost::asio::io_context ioc;
    bp::async_pipe in{ioc};
    bp::child child("./test.py", bp::std_in < in, bp::std_out.close());

    for (auto msg : { "test1\n", "test2\n", "test3\n" }) {
        write(in, bp::buffer(msg, strlen(msg)));
    }

    std::cerr << "Closing the pipe…\n";
    in.close();
    std::cerr << "Waiting for the child to exit…\n";
    ioc.run(); // already awaits completion

    child.wait(); // Parent seems to hang here.
}

You can make it more realistic by doing some delays:

#include <boost/process.hpp>
#include <iostream>

using namespace std::chrono_literals;
namespace bp = boost::process;

int main() {
    boost::asio::io_context ioc;
    bp::async_pipe in{ioc};
    bp::child child("./test.py", bp::std_in < in, bp::std_out.close());

    std::thread th([&] {
        for (auto msg : { "test1\n", "test2\n", "test3\n" }) {
            write(in, bp::buffer(msg, strlen(msg)));
            std::this_thread::sleep_for(1s);
        }

        std::cerr << "Closing the pipe…\n";
        in.close();
    });

    std::cerr << "Waiting for the child to exit…\n";
    ioc.run(); // already awaits completion
    th.join();

    child.wait(); // Parent seems to hang here.
}

For fullblown async IO see other examples:

• simultaneous read and write to child's stdio using boost.process
• How to retrieve program output as soon as it printed?
• Running a process using boost process in async mode with timeout

Gayden

Thank you for your thorough answer! I actually found out a moment ago that the problem was caused by a pipe not being closed. I’ll see if my solution works and then perhaps contact the developers to verify that this is intended behaviour. – Unripe

I'm pretty sure it is, because I've seen this behavior change/be fixed many times over in the past. It needs better documentation, I feel. – Gayden