tags : OpenAPI Ecosystem, Custom Protocols, Protocol Buffers
What’s all this madness?
Structured RPC frameworks
- Code generation from schema
- These use a communication protocol (usually HTTP) and a data serialization format (Eg. Protocol Buffers)
- See the section on structured RPC: Twirp: a sweet new RPC framework for Go
- Examples are grpc, twirp, buf connect etc.
OpenAPI Ecosystem and grpc?
- Single source of truth is defined in protobuf(schema).
grpc -> grpc-gateway -> swagger- See this for generate openapi spec from grpc-gateway
- Instead of
grpc-gatewayyou could also use grpc-web or buf connect - Protocol Buffers provides automatic documentation, client/server stub generation etc.
grpc-web vs grpc-gateway vs grpc-go?
Both(not-together) grpc-web and grpc-gateway can be used to access a grpc server from the browser. They work differently.
| Name | Description | Use-case |
|---|---|---|
| grpc-web | A spec patch that makes GRPC/Protobuf communication possible in the browser | Browser client |
| grpc-gateway | Exposes a more conventional RESTful/JSON API | Expose JSON endpoints |
| grpc-go | go runtime that’s needed to use grpc | Use grpc in Golang |
grpc-web (official, a protocol and implementation)
- Official answer from the gRPC project to the question of using gRPC in the browser.
- This can be thought of a extension of the original
grpcspec - Provides a JS client library that supports the same API as gRPC-Node to access a gRPC service.
- Differences with original grpc: check this
- Also requires a proxy (Envoy)
- gRPC makes extensive use of HTTP trailers. Because browsers don’t support trailers, no code running in a browser can speak the gRPC protocol. To work around this, the gRPC team introduced the gRPC-Web protocol, which encodes trailers at the end of the response body.
grpc-gateway (non-official, a plugin for protoc)
- A plugin of
protoc - Reads a gRPC service definition
- Generates a reverse-proxy server
- RP server translates a RESTful JSON API into gRPC.
- gRPC-Gateway project also includes an OpenAPIv2 generator
grpc-go (a runtime)
grpc-goruntime requires a particular version of the Go protobuf library- twirp and buf connect don’t use
grpc-go
TODO grpc alternatives
These are complete different RPC protocols, all based on Protocol Buffers. But grpc has a good marketshare so it has something of a grpc-ecosystem with things like grpc-go, grpc-gateway, grpc-web, openapi spec generation etc. (All these are structured rpc frameworks)
grpc (2016)
- Lacks interoperability with the rest of the Go ecosystem.
- gRPC implements structured RPCs
- Hard requirement on 2
- Dependency on
grpc-goruntime (generated code and the runtime library are tightly linked) - gRPC: The Bad Parts | Hacker News
twirp (2018)
- all plain HTTP 1.1
- Every request is a POST
- URLs are static strings identifying the right method in the right service.
- The body is either JSON- or binary-encoded protobuf, and so is the response.
- A header identifies the encoding of the body.
- Twirp is a structured RPC framework
- Can be used with our favorite Go routers: go-chi/chi, gorrila/mux or plain old standard library.
- Better interoperability with the Go HTTP ecosystem.
- Does not have streaming support, makes file upload/download harder to do.
- error metadata string only while grpc error metadata is more verbose(?)
buf connect (2021)
Can be considered a mix of twirp+grpc but taking only the good parts. Has QoL improvements. Eg. access to the request/response headers (Does not use Context for headers)
- Schema First API development built around Protocol Buffers
- So you want some sort of tool to produce code from a schema.
- Tooling includes
bufcli andconnectitself. - Better interoperability with the Go HTTP ecosystem: everything is an http.Handler or http.Client. (similar to twirp)
- compatible with
net/http, but does not supportgrpc-gointerceptor APIs
- compatible with
- Supported protocols:
grpc,grpc-webprotocols andconnectprotocol- In case of twirp, it has less compatibaility with the grpc ecosystem
- You don’t need
grpc-gateway, server automatically supports a REST-ish HTTP API. - Can talk to
grpcservers but needs theWithGRPCconfig. otherwise usesConnect’s own protocol.
-
Connect protocol vs connect implementation
- Connect protocol is similar to gRPC-Web, but with fewer limitations.
- Connect implementations allow you to set up servers/clients for all three protocols at once (gRPC, grpc-web and Connect).
-
Using connect with flutter apps
Others
- drpc
- starpc