I've been working on a microservices platform for a client and we hit the point where the REST APIs between internal services were becoming a real headache. Versioning was messy, there was no enforced contract between services, and JSON serialization overhead was noticeable under load. We switched the internal service-to-service calls to gRPC and haven't looked back. Here's how to get started.
Why gRPC
If you don't know the first thing about gRPC, don't worry — we'll cover it. The short version: gRPC is a remote procedure call framework from Google that uses Protocol Buffers as its IDL (interface definition language) and HTTP/2 as its transport.
The killer feature is the contract. You define your service and messages in a .proto file, run a code generator, and you get type-safe client and server stubs in whatever languages you need. Go, Java, Python, Node — they all speak the same proto-defined contract. If you change the API in a backwards-incompatible way, the generated code breaks at compile time, not at 3am in production.
Compared to REST/JSON for internal services:
- Performance: binary encoding + HTTP/2 multiplexing is meaningfully faster than JSON over HTTP/1.1
- Streaming: gRPC supports bidirectional streaming natively, which REST doesn't
- Type safety: no more debugging "did the other service change that field from string to int"
- Code generation: clients write themselves
The downside is that it's not human-readable on the wire and you can't curl it easily. That's fine for internal service-to-service calls; for public APIs I'd still reach for REST.
Setup
Install the Protocol Buffer compiler:
$ brew install protobuf
$ protoc --version
libprotoc 3.5.1
Install the Go plugin for protoc:
$ go get -u github.com/golang/protobuf/protoc-gen-go
Add $GOPATH/bin to your PATH if it isn't already — protoc needs to find the plugin binary.
Install the gRPC package:
$ go get -u google.golang.org/grpc
The .proto file
Our example is a user lookup service. Simple: you send a user ID, you get back a user.
syntax = "proto3";
package users;
service UserService {
rpc GetUser (GetUserRequest) returns (UserResponse);
rpc ListUsers (ListUsersRequest) returns (ListUsersResponse);
}
message GetUserRequest {
string user_id = 1;
}
message ListUsersRequest {
int32 page = 1;
int32 per_page = 2;
}
message UserResponse {
string user_id = 1;
string username = 2;
string email = 3;
int64 created_at = 4;
}
message ListUsersResponse {
repeated UserResponse users = 1;
int32 total = 2;
}
Generating Go code
$ protoc --go_out=plugins=grpc:. users.proto
This generates users.pb.go with all the types, the server interface, and a client implementation. You implement the server interface; the client is ready to use as-is.
The server
package main
import (
"context"
"log"
"net"
"google.golang.org/grpc"
pb "github.com/myorg/users/proto"
)
type server struct{}
func (s *server) GetUser(ctx context.Context, req *pb.GetUserRequest) (*pb.UserResponse, error) {
// In real life this hits a database
return &pb.UserResponse{
UserId: req.UserId,
Username: "mharris",
Email: "marie@example.com",
CreatedAt: 1519689600,
}, nil
}
func (s *server) ListUsers(ctx context.Context, req *pb.ListUsersRequest) (*pb.ListUsersResponse, error) {
users := []*pb.UserResponse{
{UserId: "u1", Username: "mharris", Email: "marie@example.com"},
{UserId: "u2", Username: "jsmith", Email: "j@example.com"},
}
return &pb.ListUsersResponse{Users: users, Total: int32(len(users))}, nil
}
func main() {
lis, err := net.Listen("tcp", ":50051")
if err != nil {
log.Fatalf("failed to listen: %v", err)
}
s := grpc.NewServer()
pb.RegisterUserServiceServer(s, &server{})
log.Println("user service listening on :50051")
if err := s.Serve(lis); err != nil {
log.Fatalf("failed to serve: %v", err)
}
}
The client
package main
import (
"context"
"log"
"time"
"google.golang.org/grpc"
pb "github.com/myorg/users/proto"
)
func main() {
conn, err := grpc.Dial("localhost:50051", grpc.WithInsecure())
if err != nil {
log.Fatalf("did not connect: %v", err)
}
defer conn.Close()
c := pb.NewUserServiceClient(conn)
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
resp, err := c.GetUser(ctx, &pb.GetUserRequest{UserId: "u1"})
if err != nil {
log.Fatalf("could not get user: %v", err)
}
log.Printf("User: %s (%s)", resp.Username, resp.Email)
}
Running it:
$ go run server/main.go &
2018/02/27 14:22:01 user service listening on :50051
$ go run client/main.go
2018/02/27 14:22:04 User: mharris (marie@example.com)
How this compares to the REST approach
In the microservices post from 2016 we built REST services where each service defined its own HTTP handlers and the consumer had to know the URL structure, parse JSON, and hope the schema hadn't changed. With gRPC, the proto file is the contract. Both sides compile against the same generated code. If I add a required field to GetUserRequest and forget to update a client, the Go compiler tells me before anything ships.
The other thing I've come to appreciate: interceptors. gRPC has a middleware chain (called interceptors) for both client and server where you can wire in logging, metrics, authentication, and tracing in one place. Adding distributed tracing across all your gRPC services is a few lines of shared code. With REST you're wrapping HTTP handlers on every service individually.
The one thing that's still annoying
Debugging. You can't just curl a gRPC endpoint and see what comes back. I use grpc_cli for quick ad-hoc calls:
$ grpc_cli call localhost:50051 users.UserService.GetUser "user_id: 'u1'"
connecting to localhost:50051
user_id: "u1"
username: "mharris"
email: "marie@example.com"
But it's not as frictionless as curl | jq. You get used to it.