Add headers for each HTTP request using client

Question

I know that I can add headers to each HTTP request manually using

cli := &http.Client{}
req, err := http.NewRequest("GET", "https://myhost", nil)
req.Header.Add("X-Test", "true")
if err != nil {
    panic(err)
}
rsp, err := cli.Do(req)

but I want to add this header automatically for each HTTP request in my app.

What is the best way to do it?

Morbific

Why not simply wrap this into a function for making requests? – Trophoplasm

Wrapping could work as long as you're the one using it directly. But once you have to hand over a client to another library that uses it in its own way, wrapping isn't sufficient anymore. – Mockup

Sure, I can wrap client.Do with custom function to append headers, but client.Get and client.Post looks more laconic than client.NewRequest, error checking, then myDo func. – Morbific

Answer 1

I'm aware of three possible solutions to this. In (my) order of preference:

1. Wrap http.NewRequest with custom code that adds desired headers:

    func MyRequest(method, path string, body io.Reader) (*http.Request, error) {
        req, err := http.NewRequest(method, path, body)
        if err != nil {
            return nil, err
        }
        req.Header.Add("X-Test", "true")
        return req, nil
    }
   

   This approach has the advantage of being straight-forward, non-magical, and portable. It will work with any third-party software, that adds its own headers, or sets custom transports.

   The only case where this won't work is if you depend on a third-party library to create your HTTP requests. I expect this is rare (I don't recall ever running into this in my own experience). And even in such a case, perhaps you can wrap that call instead.

2. Wrap calls to client.Do to add headers, and possibly any other shared logic.

    func MyDo(client *http.Client, req *http.Request) (*http.Response, error) {
        req.Header.Add("X-Test", "true")
        // Any other common handling of the request
        res, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        // Any common handling of response
        return res, nil
    }
   

   This approach is also straight-forward, and has the added advantage (over #1) of making it easy to reduce other boilerplate. This general method can also work very well in conjunction with #1. One possible draw-back is that you must always call your MyDo method directly, meaning you cannot rely on third party software which calls http.Do itself.

3. Use a custom http.Transport

    type myTransport struct{}
   
    func (t *myTransport) RoundTrip(req *http.Request) (*http.Response, error) {
        req.Header.Add("X-Test", "true")
        return http.DefaultTransport.RoundTrip(req)
    }
   

   Then use it like this:

    client := &Client{Transport: &myTransport{}}
    req := http.NewRequest("GET", "/foo", nil)
    res, err := client.Do(req)
   

   This approach has the advantage of working "behind the scenes" with just about any other software, so if you rely on a third-party library to create your http.Request objects, and to call http.Do, this may be your only option.

   However, this has the potential disadvantage of being non-obvious, and possibly breaking if you're using any third-party software which also sets a custom transport (without bothering to honor an existing custom transport).

Ultimately, which method you use will depend on what type of portability you need with third-party software. But if that's not a concern, I suggest using the most obvious solution, which, by my estimation, is the order provided above.

Inconspicuous

I've noticed that you don't raise any concerns about the manipulation of an incoming header. Is it preferable to create a new one (e.g. by newReq := req.Clone(ctx)) or would you say that a neglectable here? – Perplex

@NotX: What do you mean by "incoming header"? This is a request we're creating. – Inconspicuous

Just "incoming" in the manner of: it got passed to the MyDo function. I see my wording is a bit misleading here, I didn't mean something like an "incoming" request. What I mean: the Go api doesn't really encourage the modification of the passed req at other places (e.g. you can't change its context, but replace it with req.WithContext(...), it provides a Clone functionality etc.), so I'm wonderhing how big of an issue it not to honor that when modifying the headers of the passed request. – Perplex

@NotX: I see. Well, for options 2 and 3, where we're using a request that the function didn't directly create, you could easily check for the pre-existence of such a header if you want to. But I'm not sure that's really your question. Is it safe/allowed to modify the header in a request you receive this way? Yes, it is. Although, it does mean that the request object isn't safe for concurrent use, but that's always been the case, so we're not really changing anything. – Inconspicuous

Yeah, I just wanted to know whether its allowed/intended to modify the passed request in such a way. Thanks for the clarification! – Perplex

Another note: the RoundTripper spec states "RoundTrip should not modify the request, except for consuming and closing the Request's Body". That said, if ppl. don't really honor that in general, I won't either. I mean, it's "should", not "must". :) – Perplex

@NotX: Nice observation. I'm quite sure I hadn't noticed that before. – Inconspicuous

Answer 2

It's possible to configure http.Client to use custom transport, which can handle each request in the client (found this implementation in golang.org/x/oauth2 library). This example appends headers to each http request:

type transport struct {
    headers map[string]string
    base    http.RoundTripper
}

func (t *transport) RoundTrip(req *http.Request) (*http.Response, error) {
    for k, v := range t.headers {
        req.Header.Add(k, v)
    }
    base := t.base
    if base == nil {
        base = http.DefaultTransport
    }
    return base.RoundTrip(req)
}

func main() {
    cli := &http.Client{
        Transport: &transport{
            headers: map[string]string{
                "X-Test": "true",
            },
        },
    }
    rsp, err := cli.Get("http://localhost:8080")
    defer rsp.Body.Close()
    if err != nil {
        panic(err)
    }
}

Morbific