how to close/abort a Golang http.Client POST prematurely

Question

I'm using http.Client for the client-side implementation of a long-poll:

resp, err := client.Post(url, "application/json", bytes.NewBuffer(jsonPostBytes))
if err != nil {
    panic(err)
}
defer resp.Body.Close()

var results []*ResponseMessage
err = json.NewDecoder(resp.Body).Decode(&results)  // code blocks here on long-poll

Is there a standard way to pre-empt/cancel the request from the client-side?

I imagine that calling resp.Body.Close() would do it, but I'd have to call that from another goroutine, as the client is normally already blocked in reading the response of the long-poll.

I know that there is a way to set a timeout via http.Transport, but my app logic need to do the cancellation based on a user action, not just a timeout.

Vitamin

Answer 1

Nope, client.Post is a handy wrapper for 90% of use-cases where request cancellation is not needed.

Probably it will be enough simply to reimplement your client to get access to underlying Transport object, which has CancelRequest() function.

Just a quick example:

package main

import (
    "log"
    "net/http"
    "time"
)

func main() {
    req, _ := http.NewRequest("GET", "http://google.com", nil)
    tr := &http.Transport{} // TODO: copy defaults from http.DefaultTransport
    client := &http.Client{Transport: tr}
    c := make(chan error, 1)
    go func() {
        resp, err := client.Do(req)
        // handle response ...
        _ = resp
        c <- err
    }()

    // Simulating user cancel request channel
    user := make(chan struct{}, 0)
    go func() {
        time.Sleep(100 * time.Millisecond)
        user <- struct{}{}
    }()

    for {
        select {
        case <-user:
            log.Println("Cancelling request")
            tr.CancelRequest(req)
        case err := <-c:
            log.Println("Client finished:", err)
            return
        }
    }
}

Superhighway

not that it's pertinent to this answer, but it often preferable to use the http.DefaultTransport, which has sensible timeouts (and ProxyFromEnvironment) instead of an empty Transport. – Cymric

Of course, don't use the above exactly as written. It's a stripped down example and it discards the *http.Response returned from client.Do without calling resp.Body.Close(). Don't ever do that. – Idaidae

Ah, tr.CancelRequest() (and the related example code) is what I was looking for. – Vitamin

@CaffeineComa if you could unaccept this and accept paulo's or themihai's answer below, that would be fine. It is a better answer now. – Undercurrent

Answer 2

Using CancelRequest is now deprecated.

The current strategy is to use http.Request.WithContext passing a context with a deadline or that will be canceled otherwise. Just use it like a normal request afterwards.

req, err := http.NewRequest("GET", "http://example.com", nil)
// ...
req.Header.Add("If-None-Match", `W/"wyzzy"`)
req = req.WithContext(ctx)
resp, err := client.Do(req)
// ...

Battement

Answer 3

The standard way is to use a context of type context.Context and pass it around to all the functions that need to know when the request is cancelled.

func httpDo(ctx context.Context, req *http.Request, f func(*http.Response, error) error) error {
    // Run the HTTP request in a goroutine and pass the response to f.
    tr := &http.Transport{}
    client := &http.Client{Transport: tr}
    c := make(chan error, 1)
    go func() { c <- f(client.Do(req)) }()
    select {
    case <-ctx.Done():
        tr.CancelRequest(req)
        <-c // Wait for f to return.
        return ctx.Err()
    case err := <-c:
        return err
    }
}

golang.org/x/net/context

// A Context carries a deadline, cancelation signal, and request-scoped values
// across API boundaries. Its methods are safe for simultaneous use by multiple
// goroutines.
type Context interface {
    // Done returns a channel that is closed when this Context is canceled
    // or times out.
    Done() <-chan struct{}

    // Err indicates why this context was canceled, after the Done channel
    // is closed.
    Err() error

    // Deadline returns the time when this Context will be canceled, if any.
    Deadline() (deadline time.Time, ok bool)

    // Value returns the value associated with key or nil if none.
    Value(key interface{}) interface{}
}

Source and more on https://blog.golang.org/context

Update

As Paulo mentioned, Request.Cancel is now deprecated and the author should pass the context to the request itself(using *Request.WithContext) and use the cancellation channel of the context(to cancel the request).

package main

import (
    "context"
    "net/http"
    "time"
)

func main() {
    cx, cancel := context.WithCancel(context.Background())
    req, _ := http.NewRequest("GET", "http://google.com", nil)
    req = req.WithContext(cx)
    ch := make(chan error)

    go func() {
        _, err := http.DefaultClient.Do(req)
        select {
        case <-cx.Done():
            // Already timedout
        default:
            ch <- err
        }
    }()

    // Simulating user cancel request
    go func() {
        time.Sleep(100 * time.Millisecond)
        cancel()
    }()
    select {
    case err := <-ch:
        if err != nil {
            // HTTP error
            panic(err)
        }
        print("no error")
    case <-cx.Done():
        panic(cx.Err())
    }

}

Dreeda

You don't even need the extra goroutine or the special client. Just set req.Cancel = ctx.Done() and it will handle everything for you. – Undercurrent

The http library suffered several updates since I posted the answer. The most idiomatic solution is now provided by @Paulo. – Dreeda

@Undercurrent I've updated my answer. Feel free to write new answers if the method changes again(though I doubt it) and hopefully it will get the votes it deserves. I think the SO authors are not responsible for maintenance. The voting system should help in this regard. – Dreeda

This is regarding the update portion of your code. I am new to go but this looks wrong. Idea is to cancel the request, here the first go routine keeps running until request is actually complete. You don't cancel it, you just move ahead in the main thread based on who completes first. – Moriahmoriarty

@shadow, I've updated the answer to include the actual cancellation op (WithCancel) – Dreeda

Answer 4

To add to the other answers that attach context.Context to http requests, since 1.13 we have:

A new function NewRequestWithContext has been added and it accepts a Context that controls the entire lifetime of the created outgoing Request, suitable for use with Client.Do and Transport.RoundTrip.

https://golang.org/doc/go1.13#net/http

This function can be used instead of using NewRequest and then Request.WithContext.

req, err := http.NewRequest(...)
if err != nil {...}
req.WithContext(ctx)

becomes

req, err := http.NewRequestWithContext(ctx, ...)
if err != nil {...}

Cantwell

Answer 5

@Paulo Casaretto 's answer is right, should using http.Request.WithContext.

Here is a full demo (be aware of the time numbers: 5, 10, 30 seconds).

HTTP Server:

package main

import (
    "fmt"
    "log"
    "net/http"
    "time"
)

func handler(w http.ResponseWriter, r *http.Request) {
    fmt.Println("before sleep")
    time.Sleep(10 * time.Second)
    fmt.Println("after sleep")

    fmt.Fprintf(w, "Hi")
}

func main() {
    http.HandleFunc("/", handler)
    log.Fatal(http.ListenAndServe(":9191", nil))
}

The HTTP Server console print:

before sleep
after sleep 

HTTP Client:

package main

import (
    "context"
    "fmt"
    "net/http"
    "time"
)

func main() {
    ctx, cancel := context.WithCancel(context.Background())

    go func() {
        fmt.Println("before request")
        client := &http.Client{Timeout: 30 * time.Second}
        req, err := http.NewRequest("GET", "http://127.0.0.1:9191", nil)
        if err != nil {
            panic(err)
        }
        req = req.WithContext(ctx)
        _, err = client.Do(req)
        if err != nil {
            panic(err)
        }
        fmt.Println("will not reach here")
    }()

    time.Sleep(5 * time.Second)
    cancel()
    fmt.Println("finished")
}

The HTTP Client console print:

before request
finished

Repulsion