context.WithValue: how to add several key-value pairs

Question

With Go's context package it is possible to pass request-specific data to the stack of request handling functions using

func WithValue(parent Context, key, val interface{}) Context

This creates a new Context which is a copy of parent and contains the value val which can be accessed with key.

How do I proceed if I want to store several key-value pairs in a Context? Shall I call WithValue() several times, each time passing the Context received from my last call to WithValue()? This appears cumbersome.
Or shall I use a struct and put all my data there, s.t. I need to pass only one value (which is the struct), from which all others can be accessed?

Or is there a way of passing several key-value pairs to WithValue()?

Skit

Answer 1

You pretty much listed your options. The answer you're seeking for depends on how you want to use the values stored in the context.

context.Context is an immutable object, "extending" it with a key-value pair is only possible by making a copy of it and adding the new key-value to the copy (which is done under the hood, by the context package).

Do you want further handlers to be able to access all the values by key in a transparent way? Then add all in a loop, using always the context of the last operation.

One thing to note here is that the context.Context does not use a map under the hood to store the key-value pairs, which might sound surprising at first, but not if you think about it must be immutable and safe for concurrent use.

Using a map

So for example if you have a lot of key-value pairs and need to lookup values by keys fast, adding each separately will result in a Context whose Value() method will be slow. In this case it's better if you add all your key-value pairs as a single map value, which can be accessed via Context.Value(), and each value in it can be queried by the associated key in O(1) time. Know that this will not be safe for concurrent use though, as a map may be modified from concurrent goroutines.

Using a struct

If you'd use a big struct value having fields for all the key-value pairs you want to add, that may also be a viable option. Accessing this struct with Context.Value() would return you a copy of the struct, so it'd be safe for concurrent use (each goroutine could only get a different copy), but if you have many key-value pairs, this would result in unnecessary copy of a big struct each time someone needs a single field from it.

Using a hybrid solution

A hybrid solution could be to put all your key-value pairs in a map, and create a wrapper struct for this map, hiding the map (unexported field), and provide only a getter for the values stored in the map. Adding only this wrapper to the context, you keep the safe concurrent access for multiple goroutines (map is unexported), yet no big data needs to be copied (map values are small descriptors without the key-value data), and still it will be fast (as ultimately you'll index a map).

This is how it could look like:

type Values struct {
    m map[string]string
}

func (v Values) Get(key string) string {
    return v.m[key]
}

Using it:

v := Values{map[string]string{
    "1": "one",
    "2": "two",
}}

c := context.Background()
c2 := context.WithValue(c, "myvalues", v)

fmt.Println(c2.Value("myvalues").(Values).Get("2"))

Output (try it on the Go Playground):

two

If performance is not critical (or you have relatively few key-value pairs), I'd go with adding each separately.

Dogmatism

What about using sync.Map? I guess it would get the best of everything. – Griffin

@Griffin Yes, you could also use sync.Map if you need to mutate values in the map. – Dogmatism

as the Go context doc says, "The provided key must be comparable and should not be of type string or any other built-in type to avoid collisions between packages using context. Users of WithValue should define their own types for keys" – Tootle

"Accessing this struct with Context.Value() would return you a copy of the struct." Actually, you can store struct as a pointer in context, than you will have direct access to original struct not copy – Parent

@Parent Yes, but that's not safe for concurrent use. If you use a non-pointer, "all you can get" is a copy, and everyone who gets it has their own copy, so it's safe for concurrent use by nature. – Dogmatism

@Dogmatism yes, but you can apply mutex for safe concurrent use :) – Parent

Answer 2

Yes, you're correct, you'll need to call WithValue() passing in the results each time. To understand why it works this way, it's worth thinking a bit about the theory behind context's.

A context is actually a node in a tree of contexts (hence the various context constructors taking a "parent" context). When you request a value from a context, you're actually requesting the first value found that matches your key when searching up the tree, starting from the context in question. This means that if your tree has multiple branches, or you start from a higher point in a branch, you could find a different value. This is part of the power of contexts. Cancellation signals, on the other hand, propagate down the tree to all child elements of the one that's canceled, so you can cancel a single branch, or cancel the entire tree.

For an example, here's a context tree that contains various things you might store in contexts:

The black edges represent data lookups, and the grey edges represent cancelation signals. Note that they propogate in opposite directions.

If you were to use a map or some other structure to store your keys, it would rather break the point of contexts. You would no longer be able to cancel only a part of a request, or eg. change where things were logged depending on what part of the request you were in, etc.

TL;DR — Yes, call WithValue several times.

Judicial

"If you were to use a map or some other structure to store your keys, it would rather break the point of contexts." – While this is true, the asker has no intention to e.g. cancel the subtree starting from the last 3 of the pairs he wishes to add. The asker wants to add all his key-value pairs, and he doesn't even exposes the "middle way" contexts (and they are not accessible via the Context interface). So in my opinion in this case it comes with no disadvantage adding all key-value pairs to a single node. – Dogmatism

Yah, that's fair, it really does depend on the situation. Generally I'd say "if they're separate, unrelated values it doesn't make sense to group them" though. Eg. if you wouldn't store them together in a struct or map normally, don't do it in a context. This is, of course, just my personal preference and what I consider best practice. – Judicial

@SamWhited +1 for the deeper insight! – Skit

Answer 3

To create a golang context with multiple key-values you can call WithValue method multiple times. context.WithValue(basecontext, key, value)

    ctx := context.WithValue(context.Background(), "1", "one") // base context
    ctx = context.WithValue(ctx, "2", "two") //derived context

    fmt.Println(ctx.Value("1"))
    fmt.Println(ctx.Value("2"))

See it in action on the playground

Atrophied

This was simple, concise and worked perfectly! – Katharynkathe

Doing things this way will be purposely ignoring this statement from the documentation: "The provided key must be comparable and should not be of type string or any other built-in type...". this should help – Eudosia

@Eudosia Then can u give an example that u think is correct ? – Broody

@Broody You create a proxy type like so type contextKey string, and that's the type you use as a key type. Maybe: key1 := contextKey("key1") – Eudosia

Answer 4

As "icza" said you can group the values in one struct:

type vars struct {
    lock    sync.Mutex
    db      *sql.DB
}

Then you can add this struct in context:

ctx := context.WithValue(context.Background(), "values", vars{lock: mylock, db: mydb})

And you can retrieve it:

ctxVars, ok := r.Context().Value("values").(vars)
if !ok {
    log.Println(err)
    return err
}
db := ctxVars.db
lock := ctxVars.lock

Footrest

The provided key must be comparable and should not be of type string or any other built-in type to avoid collisions between packages using context. golang.org/pkg/context/#WithValue – Glossematics

If you wish to use string as a key, you can create type myKey string then context.WithValue(context.Background(), myKey("values"), vars{lock: mylock, db: mydb}) for addition & r.Context().Value(myKey("values")).(vars) in order to retrieve it. – Aureolin

Answer 5

I created a helper pkg to add multiple key-value pairs at once

package econtext

import (
    "context"
)

func WithValues(ctx context.Context, kv ...interface{}) context.Context {
    if len(kv)%2 != 0 {
        panic("odd numbers of key-value pairs")
    }
    for i := 0; i < len(kv); i = i + 2 {
        ctx = context.WithValue(ctx, kv[i], kv[i+1])
    }
    return ctx
}

usage -

ctx = econtext.WithValues(ctx,
    "k1", "v1",
    "k2", "v2",
    "k3", "v3",
)
        

Wrath

Answer 6

One (functional) way to do it is using currying and closures

r.WithContext(BuildContext(
   r.Context(),
   SchemaId(mySchemaId),
   RequestId(myRequestId),
   Logger(myLogger)
))

func RequestId(id string) partialContextFn {
   return func(ctx context.Context) context.Context {
      return context.WithValue(ctx, requestIdCtxKey, requestId)
   }
}

func BuildContext(ctx context.Context, ctxFns ...partialContextFn) context.Context {
   for f := range ctxFns {
      ctx = f(ctx)
   }

   return ctx
} 

type partialContextFn func(context.Context) context.Context

Chisel

Thank you for the concise method. I believe the BuildContext has an error. It should be func BuildContext(ctx context.Context, ctxFns ...partialContextFn) context.Context { for _, f := range ctxFns { ctx = f(ctx) } return ctx } – Nainsook

Answer 7

import (
    "google.golang.org/grpc/metadata"
    "context"
)

func main(){
    scheme := "bearer"
    token := getToken() // get token in string
    md := metadata.Pairs("authorization", fmt.Sprintf("%s %v", scheme, token))
    nCtx := metautils.NiceMD(md).ToOutgoing(context.Background())
}

Garment