Go HTTP Post and use Cookies

Question

I'm trying to use Go to log into a website and store the cookies for later use.

Could you give example code for posting a form, storing the cookies, and accessing another page using the cookies?

I think I might need to make a Client to store the cookies, by studying http://gotour.golang.org/src/pkg/net/http/client.go

package main

import ("net/http"
        "log"
        "net/url"
        )

func Login(user, password string) string {
        postUrl := "http://www.pge.com/eum/login"

        // Set up Login
        values := make(url.Values)
        values.Set("user", user)
        values.Set("password", password)

        // Submit form
        resp, err := http.PostForm(postUrl, values)
        if err != nil {
                log.Fatal(err)
        }
        defer resp.Body.Close()

        // How do I store cookies?
        return "Hello"
}

func ViewBill(url string, cookies) string {

//What do I put here?

}

Daltondaltonism

Unfortunately a standard cookie Jar implemntation didn't make it into Go1 but it looks like it is planned for future addition: codereview.appspot.com/5544082 – Hotheaded

Look at using gorillatoolkit.org/pkg/sessions – Unmarked

Answer 1

Go 1.1 introduced a cookie jar implementation net/http/cookiejar.

import (
    "net/http"
    "net/http/cookiejar"
)

jar, err := cookiejar.New(nil)
if err != nil { 
  // error handling 
}

client := &http.Client{
    Jar: jar,
}

Pish

I am getting undefined: cookieJar using this – Usher

@SalvatoreTimpani you forgot the import. It works well with it. – Munda

Answer 2

First you'll need to implement the http.CookieJar interface. You can then pass this into the client you create and it will be used for requests made with the client. As a basic example:

package main

import (
    "fmt"
    "net/http"
    "net/url"
    "io/ioutil"
    "sync"
)

type Jar struct {
    lk      sync.Mutex
    cookies map[string][]*http.Cookie
}

func NewJar() *Jar {
    jar := new(Jar)
    jar.cookies = make(map[string][]*http.Cookie)
    return jar
}

// SetCookies handles the receipt of the cookies in a reply for the
// given URL.  It may or may not choose to save the cookies, depending
// on the jar's policy and implementation.
func (jar *Jar) SetCookies(u *url.URL, cookies []*http.Cookie) {
    jar.lk.Lock()
    jar.cookies[u.Host] = cookies
    jar.lk.Unlock()
}

// Cookies returns the cookies to send in a request for the given URL.
// It is up to the implementation to honor the standard cookie use
// restrictions such as in RFC 6265.
func (jar *Jar) Cookies(u *url.URL) []*http.Cookie {
    return jar.cookies[u.Host]
}

func main() {
    jar := NewJar()
    client := http.Client{nil, nil, jar}

    resp, _ := client.PostForm("http://www.somesite.com/login", url.Values{
        "email": {"myemail"},
        "password": {"mypass"},
    })
    resp.Body.Close()

    resp, _ = client.Get("http://www.somesite.com/protected")

    b, _ := ioutil.ReadAll(resp.Body)
    resp.Body.Close()

    fmt.Println(string(b))
}

Airmail

You should probably use a map keyed on hostname for the cookie jar. – Moultrie

for a generic solution, agreed. This would be similar to how a browser works, but in the end it depends on implementation details. – Airmail

"Implementations of CookieJar must be safe for concurrent use by multiple goroutines." This is not concurrency safe. Not to mention the fact this sends your cookies to other hosts. I am going to downvote. – Kelikeligot

strong points for something that might get a copy and paste. I updated the example with a lock/unlock on SetCookies but honestly not quite sure about Cookies. A new slice and copies of the cookies would assure thread safety but I'm unaware of how the method might be used internally. Since I can't really answer that, I'll just as well down vote my own answer so copy and pasters beware! – Airmail

You're going to get a panic: runtime error: assignment to entry in nil map unless you set jar.cookies = map[string][]*http.Cookie{} somewhere ;-) – Classicism

For anyone using this approach and having issues with loosing cookies, have a look at https://mcmap.net/q/372986/-use-golang-to-login-to-private-site-and-pull-info for a fix – Plasty

Sorry to say, but this is a nightmare from a security perspective! Not considering Same Origin Policy at all and therefore vulnerable to leaking potentially sensible cookies (like session tokens) to hosts that should never receive them... – Nunes

Answer 3

At the version 1.5 of Go, we can use http.NewRequest to make a post request with cookie.

package main                                                                                              
import "fmt"
import "net/http"
import "io/ioutil"
import "strings"

func main() {
    // Declare http client
    client := &http.Client{}

    // Declare post data
    PostData := strings.NewReader("useId=5&age=12")

    // Declare HTTP Method and Url
    req, err := http.NewRequest("POST", "http://localhost/", PostData)

    // Set cookie
    req.Header.Set("Cookie", "name=xxxx; count=x")
    resp, err := client.Do(req)
    // Read response
    data, err := ioutil.ReadAll(resp.Body)

    // error handle
    if err != nil {
        fmt.Printf("error = %s \n", err);
    }   

    // Print response
    fmt.Printf("Response = %s", string(data));
}           

Mohler

Answer 4

net/http/cookiejar is a good option, but I like to know what cookies are actually required when making my requests. You can get the response cookies like this:

package main
import "net/http"

func main() {
   res, err := http.Get("https://stackoverflow.com")
   if err != nil {
      panic(err)
   }
   for _, c := range res.Cookies() {
      println(c.Name, c.Value)
   }
}

and you can add cookies like this:

package main
import "net/http"

func main() {
   req, err := http.NewRequest("GET", "https://stackoverflow.com", nil)
   if err != nil {
      panic(err)
   }
   req.AddCookie(&http.Cookie{Name: "west", Value: "left"})
}

• https://golang.org/pkg/net/http#Request.AddCookie
• https://golang.org/pkg/net/http#Response.Cookies

Kendra

Answer 5

Another way of doing it. Works in Go 1.8.

    expiration := time.Now().Add(5 * time.Minute)
    cookie := http.Cookie{Name: "myCookie", Value: "Hello World", Expires: expiration}
    http.SetCookie(w, &cookie)

Tamica

He asked for http client, not server. – Discountenance