How can I redirect the stdout and stderr of a command to both the console and a log file while outputting in real time?

Question

The following bit of code does exactly what I want, except it only prints to the console.

cmd := exec.Command("php", "randomcommand.php")

cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr

if err := cmd.Run(); err != nil {
    log.Fatal(err)
}

randomcommand.php:

// randomcommand.php simply alternates output between stdout and stderr 20 times

$stdout = fopen('php://stdout', 'w+');
$stderr = fopen('php://stderr', 'w+');
for ($i = 1;$i <= 20; $i++) {
    fwrite($stdout, "stdout $i\n");
    fwrite($stderr, "stderr $i\n");
}

Output:

stdout 1
stderr 1
stdout 2
stderr 2
stdout 3
stderr 3
stdout 4
stderr 4
stdout 5
stderr 5
stdout 6
stderr 6
stdout 7
stderr 7
stdout 8
stderr 8
stdout 9
stderr 9
stdout 10
stderr 10
stdout 11
stderr 11
stdout 12
stderr 12
stdout 13
stderr 13
stdout 14
stderr 14
stdout 15
stderr 15
stdout 16
stderr 16
stdout 17
stderr 17
stdout 18
stderr 18
stdout 19
stderr 19
stdout 20
stderr 20

What I'm trying to achieve is:

1. Print the command's stdout and stderr to the console in real-time;
2. Log the command's stdout and stderr to a file, exactly as it appears in the console;
3. Respect the exact order of writes to stdout and stderr, and;
4. Not modify the command itself

I've tried piping the command stdout and stderr into scanners or using io.Copy, each in a goroutine, but order of the output was not maintained. Probably because alternating between goroutines would have to occur at the same time as output alternates, which is just about impossible.

I also tried the using "select" as in the examples from here. But order of output was not maintained.

Same problem with Logstreamer.

The other idea I have to try is to see if I can somehow just read from the console buffer, but this just doesn't feel right.

Does anyone know if this is even possible and worth pursuing?

Eulogist

why not just shell. php xxx.php &>file – Ninny

possible duplicate of Streaming commands output progress – Apia

You know that your four points cannot be achieved due to buffering? – Messidor

golang.org/pkg/io/#MultiWriter check out the MultiWriter. It allows to define multiple destinations (stdout and a file, for example ;)) – Hatfield

Answer 1

As I said in the comment section, this can be achieved using MultiWriter

package main

import (
    "io"
    "log"
    "os"
    "os/exec"
)

func main() {
    // Logging capability
    f, err := os.OpenFile("log.log", os.O_RDWR|os.O_CREATE|os.O_APPEND, 0666)
    if err != nil {
        log.Fatalf("Error opening file: %v", err)
    }
    defer f.Close()
    mwriter := io.MultiWriter(f, os.Stdout)
    cmd := exec.Command("ls")
    cmd.Stderr = mwriter
    cmd.Stdout = mwriter
    err = cmd.Run() //blocks until sub process is complete
    if err != nil {
        panic(err)
    }
}

When you declare your command, and before you run it, just specify that Stdout and Stderr are using the MultiWriter defined above. This MultiWriter instance contains both a log file and the standard output.

Hatfield

Depado, you're a go nut! The answer is so simple I couldn't see it right in front of my face. Thanks! – Eulogist