Go GRPC Bidirectional Stream Performance
Asked Answered
M

0

9

We are developing a high frequency trading platform and in one of our components we have implemented the grpc with golang. And we needed to use bidirectional streaming in one of our usecases , we made a sample implementation as in below code , however when we test the performance of the code by checking the difference between timestamps of the logs in

Recv Time %v Index: %v Num: %v
Send Time %v, Index: %v, Num: %v

we found out that calling .Send method of the stream from client side and receiving the same data by calling .Recv on the server side tooks approximately 400-800 microseconds which is too low for us. We need maximum 10-50 microseconds performance , and when we read the guidelines we saw that grpc can go up to nanoseconds if both client and server is in the same computer (Which is exactly our case)

So I think we are missing some options or some performance tricks about it. Does anyone know what we can do to increase this performance problem

Client Code:

package main

import (
    "context"
    "log"
    "math/rand"

    pb "github.com/pahanini/go-grpc-bidirectional-streaming-example/src/proto"

    "time"

    "google.golang.org/grpc"
)

func main() {
    rand.Seed(time.Now().Unix())

    // dail server
    conn, err := grpc.Dial(":50005", grpc.WithInsecure())
    if err != nil {
        log.Fatalf("can not connect with server %v", err)
    }

    // create stream
    client := pb.NewMathClient(conn)
    stream, err := client.Max(context.Background())
    if err != nil {
        log.Fatalf("openn stream error %v", err)
    }

    var max int32
    ctx := stream.Context()
    done := make(chan bool)
    msgCount := 100
    fromMsg := 0

    // first goroutine sends random increasing numbers to stream
    // and closes int after 10 iterations
    go func() {
        for i := 1; i <= msgCount; i++ {
            // generate random nummber and send it to stream
            rnd := int32(i)
            req := pb.Request{Num: rnd}
            if i-1 >= fromMsg {
                sendTime := time.Now().UnixNano()
                log.Printf("Send Time %v, Index: %v, Num: %v", sendTime,i-1,req.Num)
            }

            if err := stream.Send(&req); err != nil {
                log.Fatalf("can not send %v", err)
            }
            //afterSendTime := time.Now().UnixNano()
            //log.Printf("After Send Time %v", afterSendTime)
            //log.Printf("---------------")
            //log.Printf("%d sent", req.Num)
            //time.Sleep(time.Millisecond * 200)
        }
        if err := stream.CloseSend(); err != nil {
            log.Println(err)
        }
    }()

    // third goroutine closes done channel
    // if context is done
    go func() {
        <-ctx.Done()
        if err := ctx.Err(); err != nil {
            log.Println(err)
        }
        close(done)
    }()

    <-done
    log.Printf("finished with max=%d", max)
}

Server Code:

package main

import (
    "io"
    "log"
    "net"
    "time"

    pb "github.com/pahanini/go-grpc-bidirectional-streaming-example/src/proto"

    "google.golang.org/grpc"
)

type server struct{}

func (s server) Max(srv pb.Math_MaxServer) error {

    log.Println("start new server")
    var max int32
    ctx := srv.Context()

    i := 0
    fromMsg := 0
    for {
        // exit if context is done
        // or continue
        select {
        case <-ctx.Done():
            return ctx.Err()
        default:
        }

        // receive data from stream
        req, err := srv.Recv()

        if err == io.EOF {
            // return will close stream from server side
            log.Println("exit")
            return nil
        }
        if err != nil {
            log.Printf("receive error %v", err)
            continue
        }

        if i >= fromMsg {
            recvTime := time.Now().UnixNano()
            log.Printf("Recv Time %v Index: %v Num: %v", recvTime,i,req.Num)
        }

        i++

        // continue if number reveived from stream
        // less than max
        if req.Num <= max {
            continue
        }

        // update max and send it to stream
        /*
            max = req.Num
            resp := pb.Response{Result: max}
            if err := srv.Send(&resp); err != nil {
                log.Printf("send error %v", err)
            }
        */
        //log.Printf("send new max=%d", max)
    }
}

func main() {
    // create listiner
    lis, err := net.Listen("tcp", ":50005")
    if err != nil {
        log.Fatalf("failed to listen: %v", err)
    }

    // create grpc server
    s := grpc.NewServer()
    pb.RegisterMathServer(s, server{})

    // and start...
    if err := s.Serve(lis); err != nil {
        log.Fatalf("failed to serve: %v", err)
    }
}
Marashio answered 29/4, 2020 at 11:0 Comment(5)
Schrödinger is curious if logging is throwing off your results, perhaps? There are more precise ways to run benchmarks.Corcovado
gRPC also has tools for benchmarks: github.com/grpc/grpc-go/blob/master/benchmark/benchmain/…Planish
Is it possible to send messages for ~10 seconds (or any length of time) and then divide that time by the total number of messages sent/received?Alternately
It seem there is one answer of this issue github.com/grpc/grpc-go/issues/3589Zoraidazorana
Go is an amazing language, but wouldn't be my first choice HFT. reddit.com/r/golang/comments/af721g/… gRPC is an amazing framework, but wouldn't be my first choice for performant IPC in Go: https://mcmap.net/q/299337/-how-to-implement-inter-process-communication-in-goMumps

© 2022 - 2024 — McMap. All rights reserved.