Does RPC have a timeout mechanism?

Question

If RPC does not have a timeout mechanism, how do I "kill" an RPC call if it is trying to call an RPC method of a server that is closed?

Ifill

Answer 1

You can use channels to implement a timeout pattern:

import "time"

c := make(chan error, 1)
go func() { c <- client.Call("Service", args, &result) } ()
select {
  case err := <-c:
    // use err and result
  case <-time.After(timeoutNanoseconds):
    // call timed out
}

The select will block until either client.Call returns or timeoutNanoseconds elapsed.

Liverpudlian

This is a good solution. But be aware that, when timeout happens, the goroutine will continue to run in the background, possibly holding on to OS resources. – Nessi

This would be correct if it would be an unbuffered channel. As I'm using a buffered channel with size 1 the go routine will not block. – Liverpudlian

Sebastian, the goroutine may block forever in client.Call() if RPC server won't respond. This will result in goroutines' leak. – Marcelo

At least in Go 1.5, there is a client.Go method that is preferable – Emphysema

Please be aware that in a large scale service time.After results in memory issues since it won't get collected by GC until it's triggered. Instead of time.Timer may be more desired because you can stop it manually. – Gainless

Answer 2

if you want to implement a timeout (to prevent a call from taking too long), then you'll want to change rpc.Dial for net.DialTimeout (notice they're separate packages: rpc vs net). Also be aware that the returned type isn't a client any more (as it is in the previous example); instead it is a 'connection'.

  conn, err := net.DialTimeout("tcp", "localhost:8080", time.Minute)
  if err != nil {
    log.Fatal("dialing:", err)
  }

  client := rpc.NewClient(conn)

Adrial

Answer 3

It seems the only solution for net/rpc is to close the underlying connection when you notice stuck requests. Then the client should finish pending requests with "connection broken" errors.

An alternative way is to use https://github.com/valyala/gorpc , which supports timeout RPC calls out of the box.

Marcelo

According my test, rpc.Client.Call method still not exit even I closed the underlying connection – Paleo

Answer 4

func (client *Client) Call(serviceMethod string, args interface{}, reply interface{}) error

Call method may block goroutine forever

Change use Go method:

func (client *Client) Go(serviceMethod string, args interface{}, reply interface{}, done chan *Call) *Call

Client example:

call := rpcClient.Go(method, args, reply, make(chan *rpc.Call, 1))
    select {
    case <-time.After(timeout):
        log.Printf("[WARN] rpc call timeout(%v) %v => %v", timeout, rpcClient, s.RpcServer)
        rpcClient.Close()
        return errors.New("timeout")
    case resp := <-call.Done:
        if resp != nil && resp.Error != nil {
            rpcClient.Close()
            return resp.Error
        }

Mestizo

Answer 5

Or, anno now, someone might prefer to use context instead. This also takes care of returning a proper error when timed out. (context.DeadlineExceeded)

import (
    "context"
    "log"
    "net/rpc"
)

type Client struct {
    *rpc.Client
}

// CallEx is a context aware wrapper around rpc's Client.Call()
func (c *client) CallEx(ctx context.Context, serviceMethod string, args interface{}, reply interface{}) error {
    ec := make(chan error, 1)
    go func() {
        ec <- c.Call(serviceMethod, args, reply)
    }()
    select {
    case err := <-ec:
        return err
    case <-ctx.Done():
        return ctx.Err()
    }
}

Invoke this with a Deadlined context:

type Args struct {
   A, B int
}

func main(){
    rpc, err := rpc.DialHTTP("tcp", "host")
    if err != nil {
        t.Fatal(err)
    }
    c := client{rpc}

    ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
    defer cancel()

    var i int
    if err := c.CallEx(ctx, "Calc.Multiply", Args{2, 2}, &i); err != nil {
        log.Fatal(err)
    }
}

Marilou

as upstairs @Marcelo 's comment remind, if server not response, client.call will never return, the goroutine will leak. – Pelt