How safe are Golang maps for concurrent Read/Write operations?

Question

According to the Go blog,

Maps are not safe for concurrent use: it's not defined what happens when you read and write to them simultaneously. If you need to read from and write to a map from concurrently executing goroutines, the accesses must be mediated by some kind of synchronization mechanism. (source: https://blog.golang.org/go-maps-in-action)

Can anyone elaborate on this? Concurrent read operations seem permissible across routines, but concurrent read/write operations may generate a race condition if one attempts to read from and write to the same key.

Can this last risk be reduced in some cases? For example:

• Function A generates k and sets m[k]=0. This is the only time A writes to map m. k is known to not be in m.
• A passes k to function B running concurrently
• A then reads m[k]. If m[k]==0, it waits, continuing only when m[k]!=0
• B looks for k in the map. If it finds it, B sets m[k] to some positive integer. If it doesn't it waits until k is in m.

This isn't code (obviously) but I think it shows the outlines of a case where even if A and B both try to access m there won't be a race condition, or if there is it won't matter because of the additional constraints.

Braille

We need a reproducible example: How to create a Minimal, Complete, and Verifiable example. – Kuth

Run your code with the race detector. – Anderton

Not safe. Go 1.6 added a best-effort detection of concurrent misuse of maps. The runtime crashes the program when misuse is detected. – Albemarle

Q: "Can this last risk be reduced in some cases?" A: "No". – Meggy

Possible duplicate of Map with concurrent access – Backrest

@Kuth Here is an example of a panic from unintentional concurrent write access called many times in a benchmark test. – Uralian

Answer 1

Before Golang 1.6, concurrent read is OK, concurrent write is not OK, but write and concurrent read is OK concurrent read and write is not OK but the compiler won't complain. Since Golang 1.6, map cannot be read when it's being written. So After Golang 1.6, concurrent access map should be like:

package main

import (
    "sync"
    "time"
)

var m = map[string]int{"a": 1}
var lock = sync.RWMutex{}

func main() {
    go Read()
    time.Sleep(1 * time.Second)
    go Write()
    time.Sleep(1 * time.Minute)
}

func Read() {
    for {
        read()
    }
}

func Write() {
    for {
        write()
    }
}

func read() {
    lock.RLock()
    defer lock.RUnlock()
    _ = m["a"]
}

func write() {
    lock.Lock()
    defer lock.Unlock()
    m["b"] = 2
}

Or you will get the error below:

ADDED:

You can detect the race by using go run -race race.go

Change the read function:

func read() {
    // lock.RLock()
    // defer lock.RUnlock()
    _ = m["a"]
}

Another choise:

As we known, map was implemented by buckets and sync.RWMutex will lock all the buckets. concurrent-map use fnv32 to shard the key and every bucket use one sync.RWMutex.

Hustings

Concurrent read and write was never ok even before 1.6 it just didn't complain. – Vapor

This is really nice solution. But is it possible to use channels instead of mutex? – Zee

@Zee I think channel can't do it. As map can't read when it's writting, so just use one channel to handle the map. – Hustings

Do I have to put lock in comparisons like this -> if m["a"] == 1{} ? – Leonor

@Zee You can use a channel talking to a control goroutine which has exclusive access to the map. So have a goroutine that's listening on a channel for an operation(read or write) along with a new channel for the return data, but this is still synchronous and the caller will still be blocked waiting for a response on the return channel. Not sure what the implications of this are, but it seems unnecessary. – Oxblood

> Before Golang 1.6, concurrent read is OK, concurrent write is not OK, but write and concurrent read is OK. This is flat out incorrect. It has always been incorrect to concurrently read and write to a map. – Thermolabile

I think accessing map from multiple routine is a frequent use case, why does Golang not provide built-in goroutine-safe map? – Muttonchops

@Muttonchops pkg.go.dev/sync#Map is not a built-in but at least part of the standard lib. – Durante

Answer 2

Concurrent read (read only) is ok. Concurrent write and/or read is not ok.

Multiple goroutines can only write and/or read the same map if access is synchronized, e.g. via the sync package, with channels or via other means.

Your example:

1. Function A generates k and sets m[k]=0. This is the only time A writes to map m. k is known to not be in m.
2. A passes k to function B running concurrently
3. A then reads m[k]. If m[k]==0, it waits, continuing only when m[k]!=0
4. B looks for k in the map. If it finds it, B sets m[k] to some positive integer. If it doesn't it waits until k is in m.

Your example has 2 goroutines: A and B, and A tries to read m (in step 3) and B tries to write it (in step 4) concurrently. There is no synchronization (you didn't mention any), so this alone is not permitted / not determined.

What does it mean? Not determined means even though B writes m, A may never observe the change. Or A may observe a change that didn't even happen. Or a panic may occur. Or the Earth may explode due to this non-synchronized concurrent access (although the chance of this latter case is extremely small, maybe even less than 1e-40).

Related questions:

Map with concurrent access

what does not being thread safe means about maps in Go?

What is the danger of neglecting goroutine/thread-safety when using a map in Go?

Uro

Answer 3

Go 1.6 Release Notes

The runtime has added lightweight, best-effort detection of concurrent misuse of maps. As always, if one goroutine is writing to a map, no other goroutine should be reading or writing the map concurrently. If the runtime detects this condition, it prints a diagnosis and crashes the program. The best way to find out more about the problem is to run the program under the race detector, which will more reliably identify the race and give more detail.

Maps are complex, self-reorganizing data structures. Concurrent read and write access is undefined.

Without code, there's not much else to say.

Kuth

Answer 4

You can use sync.Map which is safe for concurrent use. The only caveat is that you are gonna give up on type safety and change all the reads and writes to your map to use the methods defined for this type

Wiley

Note that sync.Map is only intended for specific use cases where lock contention ends up being a bottleneck. For most usual cases, a native map with a mutex is recommended: "The Map type is specialized. Most code should use a plain Go map instead, with separate locking or coordination, for better type safety and to make it easier to maintain other invariants along with the map content." golang.org/pkg/sync/#Map – Belovo

Answer 5

After long discussion it was decided that the typical use of maps did not require safe access from multiple goroutines, and in those cases where it did, the map was probably part of some larger data structure or computation that was already synchronized. Therefore requiring that all map operations grab a mutex would slow down most programs and add safety to few. This was not an easy decision, however, since it means uncontrolled map access can crash the program.

The language does not preclude atomic map updates. When required, such as when hosting an untrusted program, the implementation could interlock map access.

Map access is unsafe only when updates are occurring. As long as all goroutines are only reading—looking up elements in the map, including iterating through it using a for range loop—and not changing the map by assigning to elements or doing deletions, it is safe for them to access the map concurrently without synchronization.

As an aid to correct map use, some implementations of the language contain a special check that automatically reports at run time when a map is modified unsafely by concurrent execution.

Clove

Answer 6

You can store a pointer to an int in the map, and have multiple goroutines read the int being pointed to while another writes a new value to the int. The map is not being updated in this case.

This wouldn't be idiomatic for Go and not what you were asking.

Or instead of passing a key to a map, you could pass the index to an array, and have that updated by one goroutine while others read the location.

But you're probably just wondering why a map's value can't be updated with a new value when the key is already in the map. Presumably nothing about the map's hashing scheme is being changed - at least not given their current implementation. It would seem the Go authors don't want to make allowances for such special cases. Generally they want code to be easy to read and understand, and a rule like not allowing map writes when other goroutines could be reading keeps things simple and now in 1.6 they can even start to catch misuse during normal runtimes - saving many people many hours of debugging.

Bowline

It's been several years and I've seen a lot more discussion about race conditions. I don't like my answer from three years ago because even reading and writing an int, which is just like writing to an int being pointed to by another variable, is not considered safe by those who would know. Would it work? On some architectures, probably. But most processor hardware engineers would still say the results are undefined. The golang sync/atomic package is our friend and does what is necessary from the various architectures golang supports. – Bowline

Answer 7

As the other answers here stated, the native map type is not goroutine-safe. A couple of notes after reading the current answers:

1. Do not use defer to unlock, it has some overhead that affects performance (see this nice post). Call unlock directly.
2. You can achieve better performance by reducing time spent between locks. For example, by sharding the map.
3. There is a common package (approaching 400 stars on GitHub) used to solve this called concurrent-map here which has performance and usability in mind. You could use it to handle the concurrency issues for you.

Apotheosize

Answer 8

Map is concurrent safe for read only in Golang. Let's say, your map is written first and never be written again then you don't need any mutex type of thing to make sure that only one go routine is accessing your map. I have given an example below about map concurrent safe reading.

package main

import (
    "fmt"
    "sync"
)

var freq map[int]int

// An example of concurrent read from a map
func main()  {
    // Map is written before accessing from go routines
    freq = make(map[int]int)
    freq[1] = 1
    freq[2] = 2

    wg := sync.WaitGroup{}
    wg.Add(10)

    for i:=1;i<=10;i++ {
        // In go routine we are only reading val from map
        go func(id int, loop int) {
            defer wg.Done()
            fmt.Println("In loop ", loop)
            fmt.Println("Freq of 1: ", freq[id])
        }(1, i)
    }

    wg.Wait()
}

Aesculapian

Answer 9

I am going to share one package that internally handles all these things, so first, I will tell you what I was doing and getting issues while iterating the map and also updating, at the same time, issues in concurrent writing and reading.

I have one map and update it with every tick of the WebSocket (I was getting LTP of 20 scripts via socket), so for 20 scripts, a lot of ticks are coming continuously and in the same map; I have to do some different operations one operation in every 2 seconds in a different go routine and another operation in every 1 minute in different go routine so the program was crashing continuously after some time. Then I used concurrent-map and removed all the mutex I was using. Internally, the concurrent-map package developer handled everything on their side, and ran program smoothly.

Package Github Repository : Link
To know more about package : Link

Special Thanks to concurrent-map developers

Circumfluent