Go memory consumption with many goroutines

Question

I was trying to check how Go will perform with 100,000 goroutines. I wrote a simple program to spawn that many routines which does nothing but print some announcements. I restricted the MaxStack size to just 512 bytes. But what I noticed was the program size doesn't decrease with that. It was consuming around 460 MB of memory and hence around 4 KB per goroutine. My question is, can we set max stack size lower than the "minimum" stack size (which may be 4 KB) for the goroutines. How can we set the minimum Stack size that Goroutine starts with ? Below is sample code I used for the test:

package main

import "fmt"
import "time"

import "runtime/debug"

func main() {
    fmt.Printf("%v\n", debug.SetMaxStack(512))
    var i int
    for i = 0; i < 100000; i++ {
        go func(x int) {
            for {
                time.Sleep(10 * time.Millisecond)
                //fmt.Printf("I am %v\n", x)
            }
        }(i)
    }
    fmt.Println("Done")
    time.Sleep(999999999999)
}

Customhouse

Answer 1

The runtime/debug.SetMaxStack function only determines a what point does go consider a program infinitely recursive, and terminate it. http://golang.org/pkg/runtime/debug/#SetMaxStack

Setting it absurdly low does nothing to the minimum size of stacks, and only limits the maximum size by virtue of your program crashing when any stack's in-use size exceeds the limit.

Technically the crash only happens when the stack must be grown, so your program will die when a stack needs more than 8KB (or 4KB prior to go 1.2).

The reason why your program uses a minimum of 4KB * nGoroutines is because stacks are page-aligned, so there can never be more than one stack on a VM page. Therefore your program will use at least nGoroutines worth of pages, and OSes usually only measure and allocate memory in page-sized increments.

The only way to change the starting (minimum) size of a stack is to modify and recompile the go runtime (and possibly the compiler too).

Go 1.3 will include contiguous stacks, which are generally faster than the split stacks in Go 1.2 and earlier, and which may also lead to smaller initial stacks in the future.

Taipan

Go 1.3 will have a minimum stack size back down at 4 kB. We hope that Go 1.4 will be able to ratchet the minimum stack size down to 1 or 2 kB. – Condorcet

Answer 2

There's currently no way to set the minimum stack size for goroutines.

Go 1.2 increased the minimum size from 4KB to 8KB

The docs say:

"In Go 1.2, the minimum size of the stack when a goroutine is created has been lifted from 4KB to 8KB. Many programs were suffering performance problems with the old size, which had a tendency to introduce expensive stack-segment switching in performance-critical sections. The new number was determined by empirical testing."

But they go on to say:

"Updating: The increased minimum stack size may cause programs with many goroutines to use more memory. There is no workaround, but plans for future releases include new stack management technology that should address the problem better."

So you may have more luck in the future.

See http://golang.org/doc/go1.2#stack_size for more info.

Calabrese

I am using Go 1.2.1 on Windows. I guess the minimum stack size is still 4KB as the process with 100,000 goroutines consuming around 460MB. – Customhouse

On most systems, the second 4KB page doesn't get paged in until the stack needs it. I'm not sure what the terminology is in windows, but you're probably using twice as much virtual memory than that. – Oversell

Answer 3

Just a note: in Go 1.4: the minimum size of the goroutine stack has decreased from 8Kb to 2Kb.

And as per Go 1.13, it's still same -https://github.com/golang/go/blob/bbd25d26c0a86660fb3968137f16e74837b7a9c6/src/runtime/stack.go#L72:

// The minimum size of stack used by Go code
_StackMin = 2048

Christalchristalle