How to program Go to use a proxy when using a custom transport?

Question

How to write Go programs that use a proxy automatically according to the proxy environment variables?

The go get itself support the standard proxy environment variables, but i'm talking about the Go program/code itself.

This blog says,

By default http.Client checks the HTTP_PROXY and HTTPS_PROXY variables before processes any http.Request.

I tried it, but it doesn't work for my following code:

tr := &http.Transport{
    TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
}
client := &http.Client{Transport: tr}
resp, err := client.Get(url)

Augmenter

How are you creating your custom transport? Note the quoted doc says "by default", but you're overriding the defaults. – Bessbessarabia

golang.org/pkg/net/http/#ProxyFromEnvironment – Klemperer

@Peter, "ProxyFromEnvironment returns the URL of the proxy", that should be the same as what I specified in the terminal. I.e., the problem still holds, "The go get itself support the standard proxy environment variables, but i'm talking about the Go program/code itself." – Augmenter

@xpt, but you are not using it when creating the Transport. You have to assign it to the Proxy field. – Klemperer

Answer 1

You can use http.ProxyFromEnvironment method

  var PTransport = & http.Transport { Proxy: http.ProxyFromEnvironment }
  client: = http.Client { Transport: PTransport }

ProxyFromEnvironment returns the URL of the proxy to use for a given request, as indicated by the environment variables HTTP_PROXY, HTTPS_PROXY and NO_PROXY (or the lowercase versions thereof). HTTPS_PROXY takes precedence over HTTP_PROXY for https requests.

I have tried below code, it works, Just add in ur proxy details in terminal.

export http_proxy='http://user:password@prox-server:3128'
export https_proxy='http://user:password@prox-server:3128'
export HTTP_PROXY='http://user:password@prox-server:3128'
export HTTPS_PROXY='http://user:password@prox-server:3128'

package main

import (
  "fmt"
  "net/http"
  "io/ioutil"
)

func main() {

  var PTransport = & http.Transport {
    Proxy: http.ProxyFromEnvironment
  }
  client: = http.Client {
    Transport: PTransport
  }
  req, err: = http.NewRequest("GET", "https://jsonplaceholder.typicode.com/todos/1", nil)
  req.Header.Add("If-None-Match", `some value`)
  resp, err: = client.Do(req)
  if err != nil {
    panic(err)
  }
  defer resp.Body.Close()

  bodyBytes, err: = ioutil.ReadAll(resp.Body)
  if err != nil {
    panic(err)
  }

  bodyString: = string(bodyBytes)
  fmt.Printf("GET Response = %s \n", string(bodyString))


}

Lexicostatistics

Thanks a lot Chinmay! I wish I could give you more point than the +25. – Augmenter

Run code snippet does not work for this -- it is setup for JavaScript, not Go. – Tullusus

Answer 2

Set the Transport Proxy field to http.ProxyFromEnvironment to use a proxy configured from environment variables:

tr := &http.Transport{
  Proxy: ProxyFromEnvironment,
  TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
}

It's usually best to clone the default transport and set your options in that transport:

tr := http.DefaultTransport.(*http.Transport).Clone()
tr.TLSClientConfig = &tls.Config{InsecureSkipVerify: true}
client := &http.Client{Transport: tr}
resp, err := client.Get(url)

The default transport includes the correct value for the Proxy field, timeouts and other useful settings.

If the application is not creating a custom transport, then none of this is required to honor the proxy related environment variables.

Vishinsky

Note that DefaultTransport already includes Proxy: ProxyFromEnvironment, so the initial statement is kind of misleading -- if you aren't constructing a non-default transport, you may not need to set the proxy at all (as demonstrated in the second example). An even simpler example that will use a proxy: client := &http.Client{} – Tullusus

I upvoted this approach because the second example looked smart. Unfortunately, it doesn't compile: invalid indirect of http.DefaultTransport (type http.RoundTripper). Related: net/http: add Transport.Clone – Tullusus