How can I run an external program from C and parse its output?

Question

I've got a utility that outputs a list of files required by a game. How can I run that utility within a C program and grab its output so I can act on it within the same program?

UPDATE: Good call on the lack of information. The utility spits out a series of strings, and this is supposed to be portable across Mac/Windows/Linux. Please note, I'm looking for a programmatic way to execute the utility and retain its output (which goes to stdout).

Dependency

see also #126328 – Suicidal

If you want stderr as well you can redirect, for example call ls nonexistant-name 2>&1 – Thong

This is not a duplicate; the linked question is C++ specific, while this question is about C. – Fuliginous

I'm voting to reopen this question as it not a duplicate, it's about C not C++ – Appoint

see also #9406485 – Macaulay

See also: C: Run a System Command and Get Output? – Pantisocracy

Answer 1

For simple problems in Unix-ish environments try popen().

From the man page:

The popen() function opens a process by creating a pipe, forking and invoking the shell.

If you use the read mode this is exactly what you asked for. I don't know if it is implemented in Windows.

For more complicated problems you want to look up inter-process communication.

Odine

some examples of how to properly use popen() would be nice. – Tantalic

I've added a link to the Open Group man page for the function, which includes an example. Basically popen geves you a file pointer very much like fopen. It's just that it lets you write to the standard input or read from the standard output of a program instead of interacting with a disk file. That's unix's "everything is a file" philosophy at work. Some implementations extend the standard by allowing bi-directional communication. – Odine

Thi s answer indicates that popen is available on Windows. – Mcwhorter

For the solution using the raw file handles (if you need asynch I/O for example), see #9406485 – Macaulay

Answer 2

As others have pointed out, popen() is the most standard way. And since no answer provided an example using this method, here it goes:

#include <stdio.h>

#define BUFSIZE 128

int parse_output(void) {
    char *cmd = "ls -l";    
    
    char buf[BUFSIZE] = {0};
    FILE *fp;

    if ((fp = popen(cmd, "r")) == NULL) {
        printf("Error opening pipe!\n");
        return -1;
    }

    while (fgets(buf, BUFSIZE, fp) != NULL) {
        // Do whatever you want here...
        printf("OUTPUT: %s", buf);
    }

    if (pclose(fp)) {
        printf("Command not found or exited with error status\n");
        return -1;
    }

    return 0;
}

Sample output:

OUTPUT: total 16
OUTPUT: -rwxr-xr-x 1 14077 14077 8832 Oct 19 04:32 a.out
OUTPUT: -rw-r--r-- 1 14077 14077 1549 Oct 19 04:32 main.c

Fellini

Shouldn't if(pclose(fp)) be if(pclose(fp) == -1)? – Fug

@CoolGuy: depends on what you want. pclose() return -1 if there was an error obtaining the command's exit status, otherwise it returns the exit status itself. As commands traditionally set exit status 0 to indicate success, by checking if(pclose(fp)) we're testing for errors in either pclose() or the command. – Fellini

Thank you so much for this example, it pipes the output of another program to the output of the current program. Thank you! – Trexler

Another example: Is there a way to obtain the output of a linux command (like ifconfig) on a .txt file using a C program? – Pantisocracy

Any advantage to using fgets() here instead of just fread()? It seems like fread() could do the same thing without the while loop, no? They will both stop if the end of the file is reached. – Pantisocracy

@GabrielStaples: fgets() is more suitable when reading text output rather than binary, as it automatically fetches one line at a time, splitting on newlines. Both require the while loop, as you cannot be sure the whole output will fit in BUFSIZE. – Fellini

@Darth-CodeX -- it isn't a good idea to make substantive changes to code in the answers of others. In this case your edit introduced illegal C code into the answer. It turns out that empty braces initializers are not legal in Standard C. This is often a source of confusion; they are legal in C++, and it looks like they will finally be legal in C23. – Emerald

@adabsurdum: is there a legal way to zero-initialize a buffer pre-C23 in Standard C? – Fellini

The traditional way is to use {0} in C. IIRC, gcc has supported the C++ style {} even in C code for a long time. – Emerald

Answer 3

popen is supported on Windows, see here:

http://msdn.microsoft.com/en-us/library/96ayss4b.aspx

If you want it to be cross-platform, popen is the way to go.

Cuomo

Answer 4

Well, assuming you're on a command line in a windows environment, you can use pipes or command line redirects. For instance,

commandThatOutputs.exe > someFileToStoreResults.txt

or

commandThatOutputs.exe | yourProgramToProcessInput.exe

Within your program, you could use the C standard input functions to read the other programs output (scanf, etc.): http://irc.essex.ac.uk/www.iota-six.co.uk/c/c1_standard_input_and_output.asp . You could also use the file example and use fscanf. This should also work in Unix/Linux.

This is a very generic question, you may want to include more details, like what type of output it is (just text, or a binary file?) and how you want to process it.

Edit: Hooray clarification!

Redirecting STDOUT looks to be troublesome, I've had to do it in .NET, and it gave me all sorts of headaches. It looks like the proper C way is to spawn a child process, get a file pointer, and all of a sudden my head hurts.

So heres a hack that uses temporary files. It's simple, but it should work. This will work well if speed isn't an issue (hitting the disk is slow), or if it's throw-away. If you're building an enterprise program, looking into the STDOUT redirection is probably best, using what other people recommended.

#include <stdlib.h>
#include <stdio.h>

int main(int argc, char* argv[])
{
    FILE * fptr;                    // file holder
    char c;                         // char buffer


    system("dir >> temp.txt");      // call dir and put it's contents in a temp using redirects.
    fptr = fopen("temp.txt", "r");  // open said file for reading.
                                    // oh, and check for fptr being NULL.
    while(1){
        c = fgetc(fptr);
        if(c!= EOF)
            printf("%c", c);        // do what you need to.
        else
            break;                  // exit when you hit the end of the file.
    }
    fclose(fptr);                   // don't call this is fptr is NULL.  
    remove("temp.txt");             // clean up

    getchar();                      // stop so I can see if it worked.
}

Make sure to check your file permissions: right now this will simply throw the file in the same directory as an exe. You might want to look into using /tmp in nix, or C:\Users\username\Local Settings\Temp in Vista, or C:\Documents and Settings\username\Local Settings\Temp in 2K/XP. I think the /tmp will work in OSX, but I've never used one.

Trismus

c should be an int because fgetc returns an int, not a char. – Fug

Answer 5

In Linux and OS X, popen() really is your best bet, as dmckee pointed out, since both OSs support that call. In Windows, this should help: http://msdn.microsoft.com/en-us/library/ms682499.aspx

Deserved

Answer 6

MSDN documentation says If used in a Windows program, the _popen function returns an invalid file pointer that causes the program to stop responding indefinitely. _popen works properly in a console application. To create a Windows application that redirects input and output, see Creating a Child Process with Redirected Input and Output in the Windows SDK.

Baryon

Answer 7

You can use system() as in:

system("ls song > song.txt");

where ls is the command name for listing the contents of the folder song and song is a folder in the current directory. Resulting file song.txt will be created in the current directory.

Blockade

what if I wanna redirect other programs output into a file ? not an os command – Fleeta

No, this is a very bad way to do it. Do it properly using pipes, don't be leaving temporary files all over the filesystem. – Tellus

Answer 8

While the accepted answer suggested popen, it is worth mentioning that popen on windows might have unwanted side effects, especially for GUI applications.

If used in a Windows program, the _popen function returns an invalid file pointer that causes the program to stop responding indefinitely. _popen works properly in a console application.

I was surprised that there isn't a lightweight library that does that cross-platform. So I decided to create a library since I need the functionality anyway.

Here's the library https://github.com/Neko-Box-Coder/System2

It works for posix and windows systems. (It doesn't use popen for windows so it should also work in GUI applications)

It can send input and receive output from command.

And have blocking and non-blocking version.

And it has both header only and source version as well

Scheer

Answer 9

//execute external process and read exactly binary or text output
//can read image from Zip file for example
string run(const char* cmd){
    FILE* pipe = popen(cmd, "r");
    if (!pipe) return "ERROR";
    char buffer[262144];
    string data;
    string result;
    int dist=0;
    int size;
    //TIME_START
    while(!feof(pipe)) {
        size=(int)fread(buffer,1,262144, pipe); //cout<<buffer<<" size="<<size<<endl;
        data.resize(data.size()+size);
        memcpy(&data[dist],buffer,size);
        dist+=size;
    }
    //TIME_PRINT_
    pclose(pipe);
    return data;
}

Sheri

What language is it? Surely not C (string.resize?) – Gusty

looks like c++ :( – Kinross