In Go, how do I capture stdout of a function into a string?

Question

In Python, for example, I can do the following:

realout = sys.stdout
sys.stdout = StringIO.StringIO()
some_function() # prints to stdout get captured in the StringIO object
result = sys.stdout.getvalue()
sys.stdout = realout

Can you do this in Go?

Gastronome

Answer 1

I agree you should use the fmt.Fprint functions if you can manage it. However, if you don't control the code whose output you're capturing, you may not have that option.

Mostafa's answer works, but if you want to do it without a temporary file you can use os.Pipe. Here's an example that's equivalent to Mostafa's with some code inspired by Go's testing package.

package main

import (
    "bytes"
    "fmt"
    "io"
    "os"
)

func print() {
    fmt.Println("output")
}

func main() {
    old := os.Stdout // keep backup of the real stdout
    r, w, _ := os.Pipe()
    os.Stdout = w

    print()

    outC := make(chan string)
    // copy the output in a separate goroutine so printing can't block indefinitely
    go func() {
        var buf bytes.Buffer
        io.Copy(&buf, r)
        outC <- buf.String()
    }()

    // back to normal state
    w.Close()
    os.Stdout = old // restoring the real stdout
    out := <-outC

    // reading our temp stdout
    fmt.Println("previous output:")
    fmt.Print(out)
}

Harbour

This one is more similar to the Python example. :) – Gastronome

You really want/need to start the io.Copy before anything that writes to the pipe because there is a limit to the buffering capabilities of a pipe. Moving the start the goroutine is trivial and completely bypasses the issue. E.g.: play.golang.org/p/PNqa5M8zo7 – Haiku

Also don't ignore errors from os.Pipe and, possibly use defer to restore os.Stdout so that any added code that might exit the function doesn't need to worry about it. And this is very unsafe for concurrent use. – Haiku

Why is there so much more code than the Python example? Not trying to troll, genuinely curious why Go seems to need more complexity for that seemingly simple operation. – Antilogy

This solution doesn't work for me. It ends immediately after w.Close(). Anyone else have this issue? – Belch

I put your logic into a library: github.com/PumpkinSeed/cage – Keeter

Answer 2

This answer is similar to the previous ones but looks cleaner by using io/ioutil.

https://go.dev/play/p/BmgcoE70QO5

package main

import (
  "fmt"
  "io"
  "os"
)

func main() {
  rescueStdout := os.Stdout
  r, w, _ := os.Pipe()
  os.Stdout = w

  fmt.Println("Hello, playground") // this gets captured

  w.Close()
  out, _ := io.ReadAll(r)
  os.Stdout = rescueStdout
  
  fmt.Printf("Captured: %s", out) // prints: Captured: Hello, playground
}

Vollmer

This makes the same mistake as the accepted answer. It's a bad idea to write to a pipe and then later read it back from the same goroutine; especially if you're implying that your middle "block" can be anything with any amount of output. It can't, pipes have limited buffering capabilities. – Haiku

I use this in my part of my tests as I am using a third party package that prints to stdout and I need to fetch that output. I don't control the third party code so there is no other way of doing it. If you come up with another solution, I am happy to learn! ;-) – Vollmer

mattes, simply use a goroutine as in the accepted answer but start it earlier. Otherwise if you hit a case where the third party code you don't control decides to dump a lot of output (on failure perhaps, or running an a different OS where the buffering of pipes is different) you'll get "fatal error: all goroutines are asleep - deadlock" (e.g. play.golang.org/p/NtpJkCEXDX fails). – Haiku

This is an easy solution to the small plugins I write. Thanks! – Ostia

Answer 3

I don't recommend this, but you can achieve it with altering os.Stdout. Since this variable is of type os.File, your temporary output should also be a file.

package main

import (
    "fmt"
    "io/ioutil"
    "os"
    "path/filepath"
)

func print() {
    fmt.Println("output")
}

func main() {
    // setting stdout to a file
    fname := filepath.Join(os.TempDir(), "stdout")
    fmt.Println("stdout is now set to", fname)
    old := os.Stdout // keep backup of the real stdout
    temp, _ := os.Create(fname) // create temp file
    os.Stdout = temp

    print()

    // back to normal state
    temp.Close()
    os.Stdout = old // restoring the real stdout

    // reading our temp stdout
    fmt.Println("previous output:")
    out, _ := ioutil.ReadFile(fname)
    fmt.Print(string(out))
}

I don't recommend because this is too much hacking, and not very idiomatic in Go. I suggest passing an io.Writer to the functions and writing outputs to that. This is the better way to do almost the same thing.

package main

import (
    "bytes"
    "fmt"
    "io"
    "os"
)

func print(w io.Writer) {
    fmt.Fprintln(w, "output")
}

func main() {
    fmt.Println("print with byes.Buffer:")
    var b bytes.Buffer
    print(&b)
    fmt.Print(b.String())

    fmt.Println("print with os.Stdout:")
    print(os.Stdout)
}

Proteinase

Answer 4

I think the whole idea is not advisable (race condition) at all, but I guess one can mess with os.Stdout in a way similar/analogical to your example.

Mor

What kinds of race conditions are you thinking of? – Gastronome

os.Stdout is a global variable. Unconditional/unprotected setting and restoring of it is a canonical example of a race condition. – Mor

Answer 5

Even though the options listed above works, there is a clean approach in modern Go, that makes use of io.Pipe and io.Copy.

package main

import (
    "bytes"
    "fmt"
    "io"
    "os"
)

// Your function
func some_function(w *io.PipeWriter) {
    defer w.Close()
    // Fill pipe writer
    fmt.Fprintln(w, "Hello World")
}

// main function
func main() {
    // create a pipe reader and writer
    pr, pw := io.Pipe()

    // pass writer to function
    go some_function(pw)

    // custom buffer to get standard output of function
    var b bytes.Buffer

    // create a multi writer that is a combination of
    // os.Stdout and variable byte buffer `b`
    mw := io.MultiWriter(os.Stdout, &b)

    // copies pipe reader content to standard output & custom buffer
    _, err := io.Copy(mw, pr)

    if err != nil {
        if err != io.EOF {
            panic(err)
        }
    }

    // use variable
    fmt.Println(b.String())
}

The above program works this way:

1. Create a pipe that gives a reader and writer. It means, if you write something into pipe writer, will be copied to pipe reader by go
2. Create a MultiWriter with os.Stdout and custom buffer b
3. some_function(as a go-routine) will write a string into pipe writer
4. io.Copy will then copy content from pipe reader into multi-writer
5. os.Stdout will receive the output as well as your custom buffer b
6. Use buffer b

io package comes with all batteries included to work with io.Reader and io.Writer. No need to use os package, unless files are involved.

Running snippet: https://goplay.tools/snippet/3NcLVNmbEDd

Tireless