🐏 mogoz

Search

SearchSearch

grpc, protocol buffers and friends

Jun 30, 2024, 4 min read

tags : OpenAPI Ecosystem, Custom Protocols, Protocol Buffers

What’s all this madness?

Structured RPC frameworks

OpenAPI Ecosystem and grpc?

  • Single source of truth is defined in protobuf(schema).
  • grpc -> grpc-gateway -> swagger
  • Instead of grpc-gateway you could also use grpc-web or buf connect
  • Protocol Buffers provides automatic documentation, client/server stub generation etc.

grpc-web vs grpc-gateway vs grpc-go?

Both(not-together) grpc-web and grpc-gateway can be used to access a grpc server from the browser. They work differently.

NameDescriptionUse-case
grpc-webA spec patch that makes GRPC/Protobuf communication possible in the browserBrowser client
grpc-gatewayExposes a more conventional RESTful/JSON APIExpose JSON endpoints
grpc-gogo runtime that’s needed to use grpcUse grpc in Golang

grpc-web (official, a protocol and implementation)

  • Official answer from the gRPC project to the question of using gRPC in the browser.
  • This can be thought of a extension of the original grpc spec
  • Provides a JS client library that supports the same API as gRPC-Node to access a gRPC service.
  • Differences with original grpc: check this
  • Also requires a proxy (Envoy)
  • gRPC makes extensive use of HTTP trailers. Because browsers don’t support trailers, no code running in a browser can speak the gRPC protocol. To work around this, the gRPC team introduced the gRPC-Web protocol, which encodes trailers at the end of the response body.

grpc-gateway (non-official, a plugin for protoc)

  • A plugin of protoc
  • Reads a gRPC service definition
    • Generates a reverse-proxy server
    • RP server translates a RESTful JSON API into gRPC.
  • gRPC-Gateway project also includes an OpenAPIv2 generator

grpc-go (a runtime)

  • grpc-go runtime requires a particular version of the Go protobuf library
  • twirp and buf connect don’t use grpc-go

TODO grpc alternatives

These are complete different RPC protocols, all based on Protocol Buffers. But grpc has a good marketshare so it has something of a grpc-ecosystem with things like grpc-go, grpc-gateway, grpc-web, openapi spec generation etc. (All these are structured rpc frameworks)

grpc (2016)

  • Lacks interoperability with the rest of the Go ecosystem.
  • gRPC implements structured RPCs
  • Hard requirement on 2
  • Dependency on grpc-go runtime (generated code and the runtime library are tightly linked)
  • gRPC: The Bad Parts | Hacker News

twirp (2018)

  • all plain HTTP 1.1
  • Every request is a POST
  • URLs are static strings identifying the right method in the right service.
  • The body is either JSON- or binary-encoded protobuf, and so is the response.
  • A header identifies the encoding of the body.
  • Twirp is a structured RPC framework
  • Can be used with our favorite Go routers: go-chi/chi, gorrila/mux or plain old standard library.
  • Better interoperability with the Go HTTP ecosystem.
  • Does not have streaming support, makes file upload/download harder to do.
  • error metadata string only while grpc error metadata is more verbose(?)

buf connect (2021)

Can be considered a mix of twirp+grpc but taking only the good parts. Has QoL improvements. Eg. access to the request/response headers (Does not use Context for headers)

  • Schema First API development built around Protocol Buffers
    • So you want some sort of tool to produce code from a schema.
  • Tooling includes buf cli and connect itself.
  • Better interoperability with the Go HTTP ecosystem: everything is an http.Handler or http.Client. (similar to twirp)
    • compatible with net/http, but does not support grpc-go interceptor APIs
  • Supported protocols: grpc, grpc-web protocols and connect protocol
    • In case of twirp, it has less compatibaility with the grpc ecosystem
  • You don’t need grpc-gateway, server automatically supports a REST-ish HTTP API.
  • Can talk to grpc servers but needs the WithGRPC config. otherwise uses Connect’s own protocol.

  • Connect protocol vs connect implementation

    • Connect protocol is similar to gRPC-Web, but with fewer limitations.
    • Connect implementations allow you to set up servers/clients for all three protocols at once (gRPC, grpc-web and Connect).

  • Using connect with flutter apps

Others

  • drpc
  • starpc

TODO Schema compatibility

Graph View

Backlinks