Piping (or command chaining) with QProcess

Question

I'm using Qt and bash over it, need to execute something like:

bash: cat file | grep string

in Qt:

QString cmd = "cat file | grep string";
QProcess *process = new QProcess;
process->start(cmd);
process->waitForBytesWritten();
process->waitForFinished();
qDebug() << process->readAll();

The problem is in pipe ("|"), and process returs nothing. If there is no ("|"), like

"cat file" 

everything is ok. I tried smth. like

"cat file \\| grep string", 
"cat file \| grep string" 

but result is the same. If I copy the command and run it in bash everything is ok.

QString::toAscii().data()

and other transforms also have bad result.

Clove

Try cmd = "bash -c 'cat file | grep string'"; – Claud

@LaszloPapp No argument there... Since there is a dedicated API your answer is better. On the other hand, invoking a shell may allow for more complex commands, e.g. using process substitution, shell globbing etc, so it may have some advantages under some conditions. – Claud

Answer 1

The problem is you cannot run a system command with QProcess, but only a single process. So the workaround will be to pass your command as an argument to bash:

process.start("bash", QStringList() << "-c" << "cat file | grep string");

Caoutchouc

Why not use QProcess for piped commands with the high-level API as per my answer? :-) – Pleomorphism

Your answer is perfectly OK, this is just a workaround. – Caoutchouc

What do you mean by "system command"? You probably meant something more specific, like "shell command". – Naman

I meant system call like os.system() in Python or system from <cstdlib>. – Caoutchouc

@DmitryMarkin: OT, but do not use os.system() in python. People should use the subprocess module as per documentation. – Pleomorphism

Answer 2

The quick and dirty hack would be this:

QString cmd = "/bin/sh -c \"cat file | grep string\"";

You could also avoid the escaping in there with C++11's R"", but the point is that do not use bash in there because that will make it only work with bash. It will not work on embedded with busybox without bash, just ash, or any other common desktop shell.

/bin/sh is usually a symlink to the shell interpreter used, so that will eventually work.

BUT!

I think you are thinking a bit too low-level when using a high-level C++/OOP framework such as Qt. I would not recommend to invoke the commands in the low-level way when you run it from bash. There is some dedicated high-level convenience API for this use case.

Based on the official documentation, QProcess is supposed to work for pipe'd commands:

void QProcess::setStandardOutputProcess(QProcess * destination)

Pipes the standard output stream of this process to the destination process' standard input.

In other words, the command1 | command2 shell command command can be achieved in the following way:

QProcess process1;
QProcess process2;

process1.setStandardOutputProcess(&process2);

process1.start("cat file");
process2.start("grep string");
process2.setProcessChannelMode(QProcess::ForwardedChannels);

// Wait for it to start
if(!process1.waitForStarted())
    return 0;

bool retval = false;
QByteArray buffer;
while ((retval = process2.waitForFinished()));
    buffer.append(process2.readAll());

if (!retval) {
    qDebug() << "Process 2 error:" << process2.errorString();
    return 1;
}

qDebug() << "Buffer data" << buffer;

This is not the main point, but a useful suggestion: do not use QString::toAscii(). That API has been deprecated in Qt 5.

Udder

I wish if I can use this method, but the problem for me is that it did not work the way the accepted answer did. I'm not piping commands, but simply trying to perform two consecutive commands: gcc *.c; ./a.out argv[1] ... when I use your code with setStandardOutputProcess, I get weird results. – Laconic

Any thoughts on long-running processes? What if process1 is a telnet session and I want process2 to rewrite the output of that process1 telnet session. I would still send commands to process1, but need it to pipe output to process2? Code welcome. – Kossuth

Could this line while ((retval = process2.waitForFinished())); buffer.append(process2.readAll()); be while (retval = process2.waitForFinished()){ buffer.append(process2.readAll())};? – Cymose

Answer 3

The problem is that when you call process->start(cmd), the commands following the the call to cat are all interpreted as arguments to cat, so the pipe is not doing what you're expecting. If you start with a call to bash with a parameter of a string, you should get what you want: -

QString cmd = "bash -c \"cat file | grep string\"";

Alternatively, you could just call "cat file" and do the search on the returned QString when you read the output from the QProcess

Soundboard

the command runs fine, however the output i get is bit garbaged...for this command "netstat -i | grep enss33 | awk '{print $3}', it get result as 0x804d2045085 , it should be just 5085... – Rael

@krisdigitx, I suspect that may be due to how the awk command is being passed with the ' characters. If you remove the awk and just pipe to grep, does that work as expected? If that works, add the awk, but escape the quotes. – Soundboard

I have escaped the single quotes...still the same output... fpaste.org/111471/32714861 – Rael

@Rael Using stdout as a variable name is probably a bad idea. This works for me: pastebin.com/1jZWDL0C – Soundboard

Qdebug works for me also, using "output" variable made no difference with the output using cout also.....so I think the problem is with cout handling the output.... – Rael

With Qt, I don't see a reason to use cout. You could have also used fprintf and see if that makes a difference. Glad it's working now ;O) – Soundboard

the cout output is actually correct, there something else in the program which is causing this...fpaste.org/111490/03274758 – Rael

Let us continue this discussion in chat. – Soundboard

Answer 4

how about this :

QString program = "program";
QStringList arguments;

download = new QProcess(this);
download->start(program, arguments);

Kiwanis

Consider to add more details to make it be better answer than current one. – Histoplasmosis

Answer 5

If Google brought you here and you are using PyQt5 or PySide2

        process1 = QProcess()
        process2 = QProcess()
        process1.setStandardOutputProcess(process2)
        process1.start(cat, [file])
        process2.start(grep, [string])   

Plonk