How to limit the connections count of an HTTP Server implemented in Go?

Question

I am trying to implement an HTTP Server in Golang.

My problem is, I have to limit the maximum active connections count at any particular time to 20.

Hassi

Using nginx as a reverse proxy is easiest. #17777084 – Footmark

Answer 1

You can use the netutil.LimitListener function to wrap around net.Listener if you don't want to implement your own wrapper:-

connectionCount := 20

l, err := net.Listen("tcp", ":8000")

if err != nil {
    log.Fatalf("Listen: %v", err)
}

defer l.Close()

l = netutil.LimitListener(l, connectionCount)

log.Fatal(http.Serve(l, nil))

Sizar

Does this eliminate the normal tcpKeepAliveListener behavior of http.ListenAndServe? – Dierdredieresis

Answer 2

The trick with this is to implement your own net.Listener. I have an example of a listener here (see waitConn and WaitListener) that tracks connections (but doesn't limit them), which you can use as the inspiration for an implementation. It will be shaped something like this:

type LimitedListener struct {
    sync.Mutex
    net.Listener
    sem chan bool
}

func NewLimitedListener(count int, l net.Listener) *net.LimitedListener {
    sem := make(chan bool, count)
    for i := 0; i < count; i++ {
        sem <- true
    }
    return &net.LimitedListener{
        Listener: l,
        sem:      sem,
    }
}

func (l *LimitedListener) Addr() net.Addr { /* ... */ }
func (l *LimitedListener) Close() error { /* ... */ }
func (l *LimitedListener) Accept() (net.Conn, err) {
    <-l.sem // acquire
    // l.Listener.Accept (on error, release before returning)
    // wrap LimitedConn
    return c, nil
}

type LimitedConn struct { /* ... */ }

func (c *LimitedConn) Close() error {
    /* ... */
    c.sem <- true // release
}

Essentially what this is doing is creating your own implementation of net.Listener that you can give to Serve that only calls the underlying Accept when it can acquire the semaphore; the semaphore so acquired is only released when the (suitably wrapped) net.Conn is Closed. Note that technically this use of the semaphore is correct with respect to the go1.2 memory model; a simpler semaphore will be legal in future versions of Go.

Ensile

Thank for Your reply. could you tell me how to monitor the server (view number of active idle connections)??? – Hassi

store the original count, and subtract len(l.sem) from that to get (instantaneously) how many connections are waiting to be closed. – Ensile

This might not be safe. I've occasionally seen multiple calls to Close() on the same Connection from net/http/Server. This is undefined behavior according to the io.Closer interface and with this implementation it results in Close() calls blocking when trying to release (read from chan). Consider using netutil.LimitListener as described in another answer instead; it has a wrapper to ensure that the channel is only read from once per connection. (It still passes the repeated Close calls to the underlying Listener though, which is still undefined behavior.) – Meehan

Answer 3

With the help of channel you can limit the count of active connections.

1.At the server start up time create a channel and put equal number of limit count(in your case 20) values into the channel.

2.Remove one value from the channel while serving one request.

One example from the web

type limitHandler struct {
    connc   chan struct{}
    handler http.Handler
}

func (h *limitHandler) ServeHTTP(w http.ResponseWriter, req *http.Request) {
    select {
    case <-connc:
        h.handler.ServeHTTP(w, req)
        connc <- struct{}{}
    default:
        http.Error(w, "503 too busy", StatusServiceUnavailable)
    }
}

func NewLimitHandler(maxConns int, handler http.Handler) http.Handler {
    h := &limitHandler{
        connc:   make(chan struct{}, maxConns),
        handler: handler,
    }
    for i := 0; i < maxConns; i++ {
        connc <- struct{}{}
    }
    return h
}

Postlude

connc <- struct{}{} should probably be defer-ed. Then, in case of a panic (that's recovered from elsewhere), the connection "slot" would still get returned to the pool. – Headstream

Technically this doesn't limit incoming connections, it only limits how many responses you send at a time. It is a useful pattern, though probably based on e.g. load rather than an absolute connection count. – Ensile