Building High Performance APIs In Go Using gRPC And Protocol Buffers

Building High Performance APIs in Go with gRPC and

Protocol Buffers

Shiju Varghese medium.com/@shijuvar

GopherCon India 2017

Agenda

• gRPC

• Protocol Buffers

• Building APIs with gRPC

Monolithic Architecture

Catalog Management

Customer Accounts

Orders Management

Payment

Monolithic Database

Monolithic E-Commerce App

Shipment

eCom Store Web

Moving to MicroserviceseCom Store Web

Catalog Management

Customer Accounts

Order Management

Payment

Shipment

Store DB

Catalog DB

Customer DB

Order DB

Payment DB

Shipment DB

Communications on Microservices

• Inter-process communication between Microservices

• Serialisation and Deserialisation of Messages

• Scaling APIs into billions of APIs calls

• Building streaming APIs

• RESTful with JSON or RPC with Binary Encoding?

Why Not REST• Uses HTTP/1.x; Separate TCP Connection per

request

• Text on the wire; Not performance efficient

• Harder API Evolution

• Not Domain-Specific, Not Strongly-Typed

• Lack of Streaming capabilities

• High performance, open-source RPC framework

• Open source version of Google’s internal framework Stubby

• Uses Protocol Buffers as the IDL

• HTTP/2 for transport

• Bi-Directional streaming

• RPC is Efficient, Domain-Specific and Strongly-Typed

• Works across languages and platforms

• Google's language-neutral, platform-neutral, extensible mechanism for serialising structured data

• IDL - Describe once and generate interfaces for multiple languages

• Structure of the Request and Response

• Binary format for network transmission

• Supports multiple languages

Protocol Buffers

Communication between gRPC Server and Client app

Types of RPC Methods

• Simple RPC

• Server-side streaming RPC

• Client-side streaming RPC

• Bi-directional streaming RPC

gRPC Workflow

ProtoBuf Definitions

protoc Compiler Go

Ruby

Java

gRPC Server

gRPC Client Define 1 Compile 2

Implement 3

Generate Code

protoc --go_out=plugins=grpc

syntax="proto3";packageorder;

serviceOrderService{//AsimpleRPCrpcCreateOrder(Order)returns(OrderResponse){}}messageOrder{stringid=1;stringstatus=2;int64created_on=3;messageOrderItem{stringcode=1;stringname=2;floatunit_price=3;int32quantity=4;}

repeatedOrderItemorder_items=4;}messageOrderResponse{stringorder_id=1;stringerror=2;}

typeserverstruct{}

func(s*server)CreateOrder(ctxcontext.Context,in*pb.Order)(*pb.OrderResponse,error){//Implementationgoeshere}funcmain(){lis,err:=net.Listen("tcp",port)iferr!=nil{log.Fatalf("failedtolisten:%v",err)}//CreatesanewgRPCservers:=grpc.NewServer()pb.RegisterOrderServiceServer(s,&server{})s.Serve(lis)}

gRPC Server

gRPC Clientconn,err:=grpc.Dial(address,grpc.WithInsecure())iferr!=nil{log.Fatalf("Unabletoconnect:%v",err)}deferconn.Close()client:=pb.NewOrderServiceClient(conn)createOrders(client)

funccreateOrders(clientpb.OrderServiceClient){order:=&pb.Order{Id:uuid.NewV4().String(),Status:"Created",CreatedOn:time.Now().Unix(),OrderItems:[]*pb.Order_OrderItem{&pb.Order_OrderItem{Code:"knd100",Name:"KindleVoyage",UnitPrice:220,Quantity:1,},&pb.Order_OrderItem{

Code:"kc101",Name:"KindleVoyageSmartShellCase",UnitPrice:10,Quantity:2,},},}resp,err:=client.CreateOrder(context.Background(),order)}

Installation # Install Protocol Buffers compiler

$ brew install protobuf (macOS)

(Or download from https://github.com/google/protobuf/releases/tag/v3.0.0)

# Install Go Plugin

$ go get github.com/golang/protobuf/protoc

$ go get github.com/golang/protobuf/protoc-gen-go

# Install Go RPC Framework

$ go get google.golang.org/grpc

medium.com/@shijuvar linkedin.com/in/shijuvar email: gophermonk@gmail.com

THANK YOU

Articles:Building High Performance APIs In Go Using gRPC And Protocol Buffers

Benchmarking Protocol Buffers, JSON and XML in Go