How to add headers info using Transport in golang net/http

Question

I am trying to control keep-alives session to reuse the tcp connection by creating a Trasport.

Here is my snippet and I am not sure how to add headers info for authentication.

url := "http://localhost:8181/api/v1/resource"
tr := &http.Transport{
    DisableKeepAlives:   false,
    MaxIdleConns:        0,
    MaxIdleConnsPerHost: 0,
    IdleConnTimeout:     time.Second * 10,
}
client := &http.Client{Transport: tr}
resp, err := client.Get(url)

Delisadelisle

You don't. The Headers are taken from the *http.Request. The default for DisableKeepAlives is false, which means connections will be reused whenever possible . – Congress

So, looks like i don't need to create Transport as its DisableKeepAlives is false by default. – Delisadelisle

Also, If you're going to override the DefaultTransport, you should still make sure you copy all the important settings, i.e. you almost always want a DialContext with a Timeout. (and your first 3 fields are the zero value, so setting those doesn't do anything) – Congress

@Congress tried using go-routine but the number of TCP connection is piling up. – Delisadelisle

What are you using a goroutine for? You should have a single Transport, probably the DefaultTransport, regardless of any goroutines. – Congress

I am trying to do parallel get request from a pool of worker. – Delisadelisle

Here is the snippet i am trying: play.golang.org/p/O2nYhbhgyI it creates lots of tcp session. – Delisadelisle

Answer 1

Don't mix the Client from the Request.
The client uses a Transport and run the request: client.Do(req)

You set header on the http.Request with (h Header) Set(key, value string):

req.Header.Set("name", "value")

Daphene

Ok let me try. So, how should i do if i want to customize the transport struct. – Delisadelisle

@James What you are already doing. The headers though are set on the request. – Daphene

Here, I am trying to use by multiple goroutine. So, thought of creating only once and reuse it. – Delisadelisle

@James: yes, you should only use 1 Transport, but that still has nothing to do with setting the headers in each Request. – Congress

@James What headers do you want to set? Depending on your protocol, you might need them only on the first write, like in grpc: github.com/grpc/grpc-go/blob/… – Daphene

Answer 2

This may not be what you want for your specific question - setting it in the request makes more sense in your case, but to answer your question directly, you should be able to add a default header to all the requests going through the transport by using a custom RoundTrip method for your Transport.

Check out https://golang.org/pkg/net/http/#RoundTripper

Something like :

type CustomTransport struct {
  http.RoundTripper
}

func (ct *CustomTransport) RoundTrip(req *http.Request) (*http.Response, error) {
    req.Header.Add("header-key", "header-value")
    return ct.RoundTripper.RoundTrip(req)
}

url := "http://localhost:8181/api/v1/resource"
tr := &CustomTransport{
    DisableKeepAlives:   false,
    MaxIdleConns:        0,
    MaxIdleConnsPerHost: 0,
    IdleConnTimeout:     time.Second * 10,
}
client := &http.Client{Transport: tr}
resp, err := client.Get(url)

I found this useful when I didn't have direct access to the http Client used by an API client library (or each request object directly), but it allowed me to pass in a transport.

Helsell

Can't modify the request in your RoundTripper "// RoundTrip should not modify the request" pkg.go.dev/net/http#RoundTripper – Clem

Answer 3

This is what I found:

    package main

    import (
        "fmt"
        "io/ioutil"
        "net/http"
    )

    var URL = "http://httpbin.org/ip"

    func main() {
        tr := &http.Transport{DisableKeepAlives: false}
        req, _ := http.NewRequest("GET", URL, nil)
        req.Header.Set("Authorization", fmt.Sprintf("Bearer %s", "Token"))
        req.Close = false

        res, err := tr.RoundTrip(req)
        if err != nil {
            fmt.Println(err)
        }
        body, _ := ioutil.ReadAll(res.Body)
        fmt.Println(string(body))
    }

And it works.

Delisadelisle

That seems in line with what I recommended in my answer. +1 – Daphene