Building Consistent RESTful APIs in a high-performance environment

Building Consistent RESTful APIs in a High-Performance Environment

Yegor Borovikov, Software Architect

Brandon Duncan, Director of Engineering

LinkedIn Corporationhttp://blog.linkedin.com/

2

Topics We’ll Cover

> Examples of RESTful APIs What’s missing? Variety versus Uniformity

> Domain Model as Foundation Provides Uniformity Allows Flexibility

> Examples of LinkedIn APIs> Using Incentives to Scale

Some LinkedIn production APIs metrics> Q & A

3

Examples of RESTful APIsWhat if you want to get a person’s profile?

> Use one of these…http://social.yahooapis.com/v1/user/{guid}/profilehttp://api.linkedin.com/v1/people/{guid}http://www.orkut.com/social/rest/people/{guid}/@self<?xml version="1.0" encoding="UTF-8"?><person> <id>111222</id> <first-name>Gertrude</first-name> <last-name>Stein</last-name> <headline>Author of Tender Buttons</headline> <connections total="76"> …</person>

4

Examples of RESTful APIsWhat’s Missing?

> Ability to get exactly what you need (variety) If you need more, may require multiple

API calls (if they exist) If you need less, resources are wasted

> Consistency of responses (uniformity) “Same” object returned by different APIs

may have different structure Once in production, hard to get consistent later

5

Examples of RESTful APIsMultiple Calls to Get What You Need

> Want to get user’s friend’s profile? Do this… http://social.yahooapis.com/v1/user/123/connections

<connections yahoo:start="0" yahoo:count="1" yahoo:total="1"> <connection yahoo:uri="http://social.yahooapis.com/v1/user/123/connection/456?view=usercard"> <guid>456</guid> <contactId>4</contactId> </connection></connections>

6

Examples of RESTful APIs Multiple Calls to Get What You Need

> … then make second call to get friend’s profile:http://social.yahooapis.com/v1/user/456/profile

<profile yahoo:uri="http://social.yahooapis.com/v1/user/456/profile"> <guid>456</guid> <birthdate>3/3</birthdate> <created>2008-08-4T17:13:56Z</created> ...</profile>

Latent, redundant data Optimization requires stickiness

7

Typical SolutionVariety versus Uniformity

> Solution: introduce another call

> Desire for variety of responses undermines uniformity of requests

> Leads to RPC-like REST APIs

> Many APIs + Great Documentation =Lots of Reading + Lack of Automation

8

Domain Model as Foundation Sample Domain Model /people : Person[] // collection of Person resources /id : string // primary key /name : string /email : string // unique key /photo : url /best-friend : Person /friends : Person[] /jobs : Job[] // collection of Job resources /company : Company /title : string /start-date : date /end-date : date … /companies : Company[] /name : string /ceo : Person …

9

Domain Model as Foundation Follow request URL to navigate through your model

> To get a person’s profile:http://api.linkedin.com/v2/people/123

<person uri=“urn:linkedin:v2:people/123” key=“123”> <id>123</id> <name>Reid Hoffman</name> <email>reid@linkedin.com</email> <best-friend uri=“urn:linkedin:v2:people/456”/> …</person>

Conventional URL in request Default representation in response

/people[/id=123] /id /name /email /photo /best-friend /friends /jobs /company /title /start-date /end-date … /companies /name /ceo …

10

Domain Model as Foundation Fine-grained Request

> What if you only need certain fields(e.g., name and photo)?http://api.linkedin.com/v2/people/123:(name,photo)

<person> <name>Reid Hoffman</name> <photo>http://media.linkedin.com/photos/123.jpeg</photo></person>

/people[/id=123] /id /name /email /photo /best-friend /friends /jobs /company /title /start-date /end-date …

11

Domain Model as Foundation Fine-grained Request

> To get names and photos of one’s friends and their best friends:…/v2/people/456/friends:(name,photo,best-friend:

(name,photo))<friends total=“66” start=“0”> <friend uri=“urn:linkedin:v2:people/123” key=“123”> <name>Reid Hoffman</name> <photo>http://media.linkedin.com/photos/123.jpeg</photo> <best-friend uri=“urn:linkedin:v2:people/456” key=“456”> <name>Brandon Duncan</name> <photo>http://media.linkedin.com/photos/456.jpeg</photo> </best-friend> </friend> <friend>…</friend></friends>

/people[/id=456] /id /name /email /photo /best-friend /friends /123 /id /name /email /photo /best-friend /name /photo /jobs …

12

Domain Model as Foundation Fine-grained Request

> Allows client to construct custom calls

> Better than digging for the closest matching API:http://social...com/v1/user/123/profilehttp://social...com/v1/user/123/profile/usercardhttp://social...com/v1/user/123/profile/tinyusercard

> Allows optimization on the backend

13

Domain Model as Foundation Benefits

> Provides a frame for both request and response semantics

> Still allows for flexible syntax Requests – path, query params, matrix params… Responses – JSON, XML, POJOs, protobuff…

> Helps to unify and automate many development tasks on both ends Request / response creation, parsing, marshalling Code (and documentation) generation Discovery services

14

Examples of LinkedIn APIs HTTP GET - Read

…/people/email=brandon@gmail.com/friends?sort=name

…/people/123/friends;sort=name:(name,jobs;sort=start-date)

…/people:(id,name,photo)?name=page&company=google

…/people::(123,456)…/people::(123,456):(name,photo)

15

Examples of LinkedIn APIs HTTP PUT - Update

> Set the user’s name:PUT http://api.linkedin.com/v2/people/123/name<name>Reid Hoffmann</name>

> Update the user’s profile - change name and best-friend and remove photo:

PUT http://api.linkedin.com/v2/people/123<person> <name>Reid Hoffman</name> <best-friend uri=“urn:linkedin:v2:people/999”/> <photo xsi:nil=“true”/></person>

/people[/id=123] /id /name /email /photo /best-friend …

/people[/id=123] /id /name /email /photo /best-friend …

16

Examples of LinkedIn APIs HTTP POST - Create

> Add a friendPOST http://api.linkedin.com/v2/people/123/friends<friend uri=“urn:linkedin:v2:people/888”/>

201 CreatedLocation: http://api.linkedin.com/v2/people/123/friends/888

/people[/id=123] /id /name /email /photo /best-friend /friends /456 /888 …

17

Examples of LinkedIn APIs HTTP DELETE - Remove

> Remove a friendDELETE http://api.linkedin.com/v2/people/123/friends/456

> Delete a companyDELETE

http://api.linkedin.com/v2/companies/exchange=NYSE&ticker=GM

> Delete two companiesDELETE http://api.linkedin.com/v2/companies::(664,665)

18

Use Standard Headers

> Content-Type> Last-Modified> Accept> Vary> Authorization> Cache-Control> Content-MD5> Location> Warning

19

Incentive System

> Multiple ways to get at the same data

> Partner can ask for exactly what they need

> Associate cost with resources, system of accounting creates incentives for partners

> Throttling by resource rather than API

20

Real-World Example Xobni Toolbar

> Xobni makes ~20 million profile API calls per week> Default representation is ~2k on average> Using in-line filter brings average to ~1.5k

25% reduction in response size ~11000 Mbits savings per day

11k Mbits out of LinkedIn datacenter 11k Mbits into Xobni datacenter Saves both companies money

21

Yegor Borovikovyborovik@linkedin.com

Brandon Duncanbduncan@linkedin.com

blog.linkedin.com