Spike - Transport mechanisms

grpc

• grpc/grpc-go: The Go language implementation of gRPC. HTTP/2 based RPC
  • bidirectional streaming
  • cancellation propagation
  • interaction with flow-control at the application layer

• cascading call-cancellation
• load balancing & failover
  • Visual demo of http1 vs http2 loading

• grpc / gRPC Concepts

• clients can specify a timeout DEADLINE
• clients can provide METADATA that is opaque to gRPC itself

• grpc / Tutorial

• "Note that in gRPC-Go, RPCs operate in a blocking/synchronous mode, which means that the RPC call waits for the server to respond, and will either return a response or an error."
• "We also pass a context.Context object which lets us change our RPC’s behaviour if necessary, such as time-out/cancel an RPC in flight."

• mwitkow/go-grpc-middleware: Golang gRPC Middleware Helpers: interceptor chaining, context propagation etc.

• "gRPC Go recently acquired support for Interceptors, i.e. middleware that is executed either on the gRPC Server before the request is passed onto the user's application logic, or on the gRPC client either around the user call. It is a perfect way to implement common patters: auth, logging, message, validation, retries or monitoring. These are generic building blocks that make it easy to build multiple microservices easily. The purpose of this repository is to act as a go-to point for such reusable functionality. It contains some of them itself, but also will link to useful external repos."

• shafreeck/fperf: Framework of performance testing

JSON gateways / proxies

• grpc-ecosystem/grpc-gateway: gRPC to JSON proxy generator
• philips/grpc-gateway-example
• Language Guide (proto3) JSON

Reference

• gRPC: Internet-scale RPC framework is now 1.0 | Hacker News
• gRPC and the displacement of REST based APIs
  • "Tracing is also built in.. Because requests can be tracked using the same context through multiple services, it's possible to cancel requests on different systems and or trace them to see what is causing delays"
• Take a REST with HTTP/2, Protobufs, and Swagger
• gRPC: Google's high-performance, open-source RPC framework by Sameer Ajmani - YouTube
  • gRPC Go - slides

rpc - The Go Programming Language

• Package rpc provides access to the exported methods of an object across a network or other I/O connection.
• gob - The Go Programming Language
  • Package gob manages streams of gobs - binary values exchanged between an Encoder (transmitter) and a Decoder (receiver). A typical use is transporting arguments and results of remote procedure calls (RPCs) such as those provided by package "net/rpc".
  • Gobs of data - The Go Blog

Protocol buffers

Overview

• Content (data) separated from Structure (design)
• Go Generated Code | Protocol Buffers | Google Developers
• proto - GoDoc
• Language Guide (proto3) | Protocol Buffers | Google Developers
  • "Note that for scalar message fields, once a message is parsed there's no way of telling whether a field was explicitly set to the default value (for example whether a boolean was set to false) or just not set at all: you should bear this in mind when defining your message types."
  • "Also note that if a scalar message field is set to its default, the value will not be serialized on the wire."

protobuf/third_party.md at master · google/protobuf

• protobuf-wireshark
• estan/protoc-gen-doc: Documentation generator plugin for Google Protocol Buffers

Cap'n Proto: Introduction

• zombiezen/go-capnproto2: Cap'n Proto library and code generator for Go
  • Getting Started · zombiezen/go-capnproto2 Wiki
  • Cap'n Proto: RPC Protocol
    • supports promises and pipelining requests

Protobuff vs Thrift vs JSON

• 5 Reasons to Use Protocol Buffers Instead of JSON For Your Next Service

  • Less Boilerplate Code

• Thrift vs Protocol Bufffers vs JSON | MirthLab

  • JSON
    • no built-in way to allow versioning of objects
    • handwritten adhoc code to encoding/decoding, error handling and type conversions
  • Both Protobuf and Thrift allow some type of data versioning so that both clients and servers can continue to work without being upgraded, even if the protocol has changed
  • (Apache) Thrift has richer data structures than (Google) Protobuf (e.g.: Map and Set)
  • Protobuf is faster and smaller than Thrift; has better documentation and arguably cleaner API

• Thrift vs Protocol Buffers vs Avro - Biased Comparison

• 5 Lessons Learnt From Choosing Zeromq And Protobuf

• FlatBuffers: Benchmarks

• Home · eishay/jvm-serializers Wiki- Benchmarks

TODO

• RPC vs REST
• Benchmark HTTP2 streaming for large results