gRPC Gateway with Go

January 2, 2023

I created a simple app or back-end in Go to calculate weights. I built this app to demonstrate gRPC along with gRPC Gateway and its unit tests. gRPC Gateway is cool. You code once for gRPC services and gRPC Gateway translates those services to HTTP services for standard RESTful JSON API. There is no need to code the HTTP routes, JSON validation, and unit tests because the gRPC Gateway already covers them. It saves you time and effort. gRPC is fast and bandwidth-efficient. Feel free to read my previous article about gRPC.

The app is simple. It covers CRUD to demonstrate REST API and a simple calculation to calculate weights. I think I don't need to put its details here. Let's talk about the gRPC stuff only.

Protocol Buffers

I wrote the protocol buffers in proto3. Writing protocol buffers is the first thing we need to do for working with gRPC.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57


syntax = "proto3";

import "google/api/annotations.proto";
import "google/protobuf/empty.proto";
import "google/protobuf/timestamp.proto";

option go_package = "github.com/aristorinjuang/weight-go/tools/grpc";

service WeightService {
  rpc ListWeights(google.protobuf.Empty) returns (Weights) {
    option (google.api.http) = {
      get: "/v1/weights"
    };
  };
  rpc CreateWeight(WeightParams) returns (google.protobuf.Empty) {
    option (google.api.http) = {
      post: "/v1/weights"
      body: "*"
    };
  };
  rpc ReadWeight(WeightParams) returns (Weight) {
    option (google.api.http) = {
      get: "/v1/weights/{date}"
    };
  };
  rpc UpdateWeight(WeightParams) returns (google.protobuf.Empty) {
    option (google.api.http) = {
      put: "/v1/weights/{date}"
      body: "*"
    };
  };
  rpc DeleteWeight(WeightParams) returns (google.protobuf.Empty) {
    option (google.api.http) = {
      delete: "/v1/weights/{date}"
    };
  };
}

message WeightParams {
  string date = 1;
  double max = 2;
  double min = 3;
}

message Weight {
  google.protobuf.Timestamp date = 1;
  double max = 2;
  double min = 3;
  double difference = 4;
}

message Weights {
  repeated Weight weights = 1;
  double average_max = 2;
  double average_min = 3;
  double average_difference = 4;
}

You can notice that there are some differences with ordinary protocol buffers. We have an option from google.api.http. We specified the routes along with their HTTP methods and parameters to that option. {date} means WeightParams.date. * means all fields of WeightParams. We can use google.protobuf.Empty if there is no request and response.

Buf

Now, it's time to compile the protocol buffers. I used Buf instead of protoc. Buf is a new way of working with protocol buffers. Follow my link and install Buf, https://docs.buf.build/installation/.

buf.yaml

1
2
3
4


version: v1
name: buf.build/aristorinjuang/weight-go
deps:
  - buf.build/googleapis/googleapis

buf.gen.yaml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


version: v1
plugins:
  - name: go
    out: .
    opt: paths=source_relative
  - name: go-grpc
    out: .
    opt: paths=source_relative,require_unimplemented_servers=false
  - name: grpc-gateway
    out: .
    opt: paths=source_relative

Put those YAML files into the root of your proto files or along with them. We put buf.build/googleapis/googleapis as the dependency. It will generate three Go files. One of them is from the grpc-gateway.

Go to the folder that has those YAML files and type buf generate on the terminal. You can put some parameters on Buf if you want to compile proto files from another folder such as the root of the project. Eg; buf generate --template tools/grpc/buf.gen.yaml --output tools/grpc/ tools/grpc.

gRPC Services

Now, it's time to code the gRPC services along with their unit tests.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


func (h *handler) CreateWeight(ctx context.Context, in *pb.WeightParams) (*emptypb.Empty, error) {
  date, err := time.Parse("2006-01-02", in.GetDate())
  if err != nil {
    return &emptypb.Empty{}, err
  }

  weight, err := entity.NewWeight(date, in.GetMax(), in.GetMin())
  if err != nil {
    return &emptypb.Empty{}, err
  }

  if err := h.repository.Create(weight); err != nil {
    return &emptypb.Empty{}, err
  }

  return &emptypb.Empty{}, nil
}

This article will be too long if I explain all methods. Let me explain this method only. You can't return nil for *emptypb.Empty, use &emptypb.Empty{} instead. The date is a string in this app because the time is unnecessary.

Unit Tests

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39


func (t *weightGRPCTest) TestCreateWeight() {
  t.Run("success", func() {
    t.weightRepository.On("Create", data.Weights[0]).Return(nil).Once()

    _, err := t.client.CreateWeight(context.Background(), &pb.WeightParams{
      Date: data.Weights[0].Date().Format("2006-01-02"),
      Max:  data.Weights[0].Max(),
      Min:  data.Weights[0].Min(),
    })

    t.NoError(err)
  })

  t.Run("failed", func() {
    t.weightRepository.On("Create", data.Weights[0]).Return(errors.New("failed to create a weight")).Once()

    _, err := t.client.CreateWeight(context.Background(), &pb.WeightParams{
      Date: data.Weights[0].Date().Format("2006-01-02"),
      Max:  data.Weights[0].Max(),
      Min:  data.Weights[0].Min(),
    })

    t.Error(err)
  })

  t.Run("failed to create a weight", func() {
    _, err := t.client.CreateWeight(context.Background(), &pb.WeightParams{
      date: 2023-01-01",
    })

    t.Error(err)
  })

  t.Run("failed to parse the date", func() {
    _, err := t.client.CreateWeight(context.Background(), &pb.WeightParams{})

    t.Error(err)
  })
}

This was how I tested the CreateWeight method. We mocked the weightRepository in some sub-tests so we can test the CreateWeight method for any condition. We don't need to assert the *emptypb.Empty.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39


type weightGRPCTest struct {
  suite.Suite
  weightRepository *repository.WeightRepositoryMock
  client           pb.WeightServiceClient
}

func (t *weightGRPCTest) dialer() func(context.Context, string) (net.Conn, error) {
  listener := bufconn.Listen(1024 * 1024)
  server := grpc.NewServer()

  pb.RegisterWeightServiceServer(server, New(t.weightRepository))

  go func() {
    server.Serve(listener)
  }()

  return func(context.Context, string) (net.Conn, error) {
    return listener.Dial()
  }
}

func (t *weightGRPCTest) SetupSuite() {
  t.weightRepository = new(repository.WeightRepositoryMock)

  conn, _ := grpc.DialContext(
    context.Background(),
    "bufnet",
    grpc.WithContextDialer(t.dialer()),
    grpc.WithTransportCredentials(insecure.NewCredentials()),
  )

  t.client = pb.NewWeightServiceClient(conn)
}

...

func TestWeightGRPC(t *testing.T) {
  suite.Run(t, new(weightGRPCTest))
}

I used the suite package from Testify to test this gRPC service. The initial setup to test this is to create a dial function, dialer(), then create a connection for the gRPC service client. 1024 * 1024 is the capacity of the data in a pipe, it is a ring buffer of fixed capacity.

Conclusion

gRPC Gateway and Buf are cool. This is the best option if you want to provide services for both gRPC and HTTP standard REST APIs in a single codebase. Even this condition is not often. Because gRPC is widely used for communication between internal services. HTTP REST API is widely used for front-end or public. There is also a tool called GraphQL to provide APIs better than the HTTP REST API to the public.

Writing unit tests for gRPC is a little bit confusing at first. I hope this article can help. You can check the completed code, https://github.com/aristorinjuang/weight-go, which is 100% tested.