HYPERMEDIA: THE MISSING ELEMENT

to Building Adaptable Web APIs in Rails

Toru Kawamura @tkawa

!RubyKaigi 2014

ハイパーメディア: RailsでWeb APIをつくるには、これが足りない

@tkawaToru Kawamura

• Freelance Ruby/Rails programmer

• Technology Assistance Partner at SonicGarden Inc.

• RESTafarian inspired by Yohei Yamamoto (@yohei)

• Co-organizer of Sendagaya.rb

• Organizer of the reading group of “RESTful Web APIs”

Web API

“Web”

http://www.opte.org/the-internet/

http://pixabay.com/en/spider-web-net-grid-silk-drops-13516/

https://www.flickr.com/photos/tamaki/260594564/

• Private

• For internal use

• For SPA or dedicated clients only

• Almost expected, almost controllable

• Public

• For external use

• For general-purpose clients

• Less expected, less controllable

http://www.slideshare.net/yohei/webapi-36871915

“Whether an API should be RESTful or not depends on the requirement”

– 「WebAPIのこれまでとこれから」by @yohei

http://pixabay.com/en/spider-web-net-grid-silk-drops-13516/

Change

Change is inevitable !

Web APIs must adapt to changes

変化は避けられない Web APIは変化に適応しなければならない

Two types of Change

With versioning Without versioning

Incompatible Compatible

Breaks clients Does not break clients

Breaking Change Non-Breaking Change

Breaking Changes are Harmful

• Terrible user experience

• Forces client developers to rewrite/redeploy code

• What if on …

壊す変更は有害

ひどいユーザ体験

クライアント開発者にコードの書き直し・再デプロイを強いる

With versioning Without versioning

Incompatible Compatible

Breaks clients Does not break clients

Breaking Change Non-Breaking Change

Because of what?なぜ起こるの？

Many clients are built from human-readable documentation

GET /v1/statuses?id=#{id} GET /v1/statuses?id=#{id}

人間が読める説明書から作られるクライアントがたくさんある

GET /v2/statuses/#{id} GET /v1/statuses?id=#{id}×Need to rewrite code

Some clients are built from machine-readable documentation

{ "apiVersion": "1.0.0", "basePath": "http://petstore.swagger.wordnik.com/api", "resourcePath": "/store", "produces": [ "application/json" ], "apis": [ { "path": "/store/order/{orderId}", "operations": [ { "method": "GET", "summary": "Find purchase order by ID", "notes": "For valid response try integer IDs with value <= 5. Anything above 5 or nonintegers will generate API errors", "type": "Order", "nickname": "getOrderById", "authorizations": {}, "parameters": [

GET /v1/statuses?id=#{id} GET /v1/statuses?id=#{id}

機械が読める説明書から作られるクライアントもある

{ "apiVersion": "2.0.0", "basePath": "http://petstore.swagger.wordnik.com/api", "resourcePath": "/store", "produces": [ "application/json" ], "apis": [ { "path": "/store/order/{orderId}", "operations": [ { "method": "GET", "summary": "Find purchase order by ID", "notes": "For valid response try integer IDs with value <= 5. Anything above 5 or nonintegers will generate API errors", "type": "Order", "nickname": "getOrderById", "authorizations": {}, "parameters": [

GET /v2/statuses/#{id} GET /v1/statuses?id=#{id}×Need to regenerate code

{ uber: { version: "1.0", data: [{ url: "http://www.ishuran.dev/notes/1", name: "Article", data: [ { name: "articleBody", value: "First note's text" }, { name: "datePublished", value: null }, { name: "dateCreated", value: "2014-09-11T12:00:31+09:00" }, { name: "dateModified", value: "2014-09-11T12:00:31+09:00" }, { name: "isPartOf", rel: "collection", url: "/notes"

• API changes should be reflected in clients

• It is good to split up explanations of the API and embed them into each API response

• A lot of assumptions about the API make a tight coupling

Because of Coupling密結合のせい

APIの変更がクライアントに反映されるべき

APIの説明を分割して各レスポンスに埋め込むのが良い

APIについての多大な仮定は密結合を生む

With versioning Without versioning

Incompatible Compatible

Breaks clients Does not break clients

Breaking Change Non-Breaking Change

because of the Coupling because of the Decoupling

Decoupling in a example: FizzBuzzaaS

• by Stephen Mizell http://fizzbuzzaas.herokuapp.com/http://smizell.com/weblog/2014/solving-fizzbuzz-with-hypermedia

• Server knows how to calculate FizzBuzz for given number (<= 100)

• Server knows what the next FizzBuzz will be

• Client wants all FizzBuzz from one to the last in order

例で見る疎結合

サーバは100までの数のFizzBuzzを計算できる

サーバは次のFizzBuzzが何になるか知っている

クライアントは1から最後まで順番にすべてのFizzBuzzが欲しい

http://sef.kloninger.com/posts/201205fizzbuzz-for-

managers.html

Coupled client

• Every URL and parameter is hardcoded

• Duplicates the server logic such as counting up

"/v2/fizzbuzz/#{i}"

(1..1000)

(1..100).each do |i| answer = HTTP.get("/v1/fizzbuzz?number=#{i}") puts answer end

密結合なクライアント

すべてのURLとパラメータがハードコードされている

カウントアップのようなサーバロジックと同じことをやっている

Decoupled client

• No hardcoded URLs

• Client doesn’t break when changing URLs / the restriction

root = HTTP.get_root answer = root.link('first').follow puts answer while answer.link('next').present? answer = answer.link('next').follow puts answer end Link ‘next’ is the key

疎結合なクライアント

ハードコードされたURLなし

URLや条件を変えてもクライアントは壊れない

The “API Call” metaphor is dangerous

• We need to move away from the paradigm where a client arranges a URL and parameters in advance and calls API (like RPC…)

• What a client does next should be to choose from links in the response == HYPERMEDIA

「APIコール」のメタファーは危険

URLとパラメータを用意してAPIを呼ぶというRPCのようなパラダイムから離れよう

クライアントが次にすることはリンクから選ぶことこれがハイパーメディア

This is not imaginary but already present in HTML

これは想像上のものではなく、すでにHTMLにある

The HTML Web• Web apps and websites

have been changing constantly without breaking browsers

• Why don’t browsers break on the HTML Web?

There are links in HTML

WebアプリやWebサイトはずっと変わり続けているけどブラウザは壊れていないのはなぜ？

http://www.youtypeitwepostit.com/messages

Workflow in HTML• Web app includes a

(suggested) workflow

• Workflow is represented by a sequence of screen transitions — Links and Forms

Webアプリはワークフローを含む ワークフローは一連の画面遷移で表現される

それはリンクとフォーム”RESTful Web APIs” p.11 Figure 1-7

Hypermedia show the workflow

• Each screen includes what a browser can do next through links and forms like a “menu”

• A browser chooses from the “menu” to go to the next step

• This is HYPERMEDIA and exactly what FizzBuzzaaS does

3

4

各画面は次に何ができるかのリンクやフォームの「メニュー」を含み、ブラウザはその中から選ぶ

これがハイパーメディア

ハイパーメディアはワークフローを示す

One more hint in a Crawler

• Crawlers follow links and can submit some forms • Crawlers understand the data in an HTML document

and their “meaning” • How can they do that?

クローラーにはもう1つヒントが

クローラはHTMLの中のデータと意味を理解しているどうやって？https://support.google.com/webmasters/answer/99170

• Mechanism that embeds structured data within an HTML document

• Document structure can change without changing data • Connects data with a URL that roughly represents

the “meaning of data” (this is also a kind of link)

Microdata<div itemscope itemtype="http://schema.org/Person"> My name is <span itemprop="name">Bob Smith</span> but people call me <span itemprop="nickname">Smithy</span>. Here is my home page: <a href="http://www.example.com" itemprop="url">www.example.com</a> I live in Albuquerque, NM and work as an <span itemprop="title">engineer</span> at <span itemprop="affiliation">ACME Corp</span>. </div>

URLに結びつけることで大まかな「データの意味」も表す

Microdataは構造化データをHTMLに埋め込むしくみ

• Mechanism that embeds structured data within an HTML document

• Document structure can change without changing data • Connects data with a URL that roughly represents

the “meaning of data” (this is also a kind of link)

<div itemscope itemtype="http://schema.org/Person"> My name is <span itemprop="name">Bob Smith</span> but people call me <span itemprop="nickname">Smithy</span>. Here is my home page: <a href="http://www.example.com" itemprop="url">www.example.com</a> I live in Albuquerque, NM and work as an <span itemprop="title">engineer</span> at <span itemprop="affiliation">ACME Corp</span>. </div>

schema.org is the standard vocabulary promoted by

Bing, Google, Yahoo! and Yandex

Microdata

http://getschema.org/index.php/Main_Page

You could build a Web API in HTML

• “Microdata DOM API” allows clients to extract data from HTMLhttp://www.w3.org/TR/microdata/#using-the-microdata-dom-api

• Available in JavaScript: https://github.com/termi/Microdata-JS

• There are also some specs for translating Microdata into JSON

• HTML’s great advantage is that it has links and forms built-in

var user = document.getItems('http://schema.org/Person')[0]; var name = user.properties['name'][0].itemValue; alert('Hello ' + name + '!');

HTMLでWeb APIを作ることもできる

Microdata DOM APIでHTMLからデータを抽出できる

MicrodataからJSONに変換もできる

HTMLはリンクとフォームを持っているのが大きなアドバンテージ

But you probably want aJSON Web API…

• You have to fill in links and forms (also the meanings of data, if possible)

data link form

HTML+Microdata ✓✓ ✓ ✓

JSON ✓ - -

✓✓: including “meaning of data”

でもたぶんJSON Web APIが欲しいよね

リンクとフォームを埋めればいい（できればデータの意味も）

Links and Forms in JSON

• Use a JSON-based format that can represent links and forms

• There are other formats Siren, Collection+JSON, Mason, Verbose, etc

data link form

JSON ✓ - -

JSON +Link header ✓ ✓ -

HAL ✓ ✓ -

JSON-LD ✓✓ ✓ -

JSON-LD+Hydra ✓✓ ✓ ✓

UBER ✓ ✓ ✓✓✓: including “meaning of data”

リンクとフォームを表現できるJSONベースのフォーマットがある

A Solution

Hypermicrodata gem

• Translate HTML into JSON on Server-side

• Extract not only Microdata but also links and forms from HTML

• Generate a JSON-based format that naturally fits with an explanation of meaning of data

https://github.com/tkawa/hypermicrodata

サーバサイドでHTMLをJSONに変換

MicrodataだけではなくリンクとフォームもHTMLから抽出

データの意味も表しやすい形でJSONベースのフォーマットを生成

Design procedure in Rails with Hypermicrodata gem

1. Design resources

2. Draw a state diagram

3. Connect names of data with corresponding URLs

4. Write HTML templates (Haml, Slim, etc) with Microdata markup(Then, write profiles and explanations that are not defined in schema.org, if necessary)

Hypermicrodata gemを使ったRailsによる設計手順

1. リソース設計

2. 状態遷移図を描く

3. データの名前を対応するURLに結びつける

4. HTMLテンプレートを書きMicrodataでマークアップ

1. Design resources

column name short description type

text content text of note text

published_at published time of note datetime

(id, created_at, updated_at) (auto-generated)

$ rails g model Note text:text published_at:datetime

model: Notecontroller : NotesController

routing: resources :notes

2. Draw a state diagram

Collection Memberitem

collection

create*†

update*, delete*

* unsafe † non-idempotent

Begin with Collection & Member Resource pattern of Rails (API ver.)

Collection of Note

Note (text, published_at,

created_at, updated_at, id)

item

collection

create*†

update*, delete*,

* unsafe † non-idempotent

publish*

next, prev

Home

notes home

3. Connect names of data with corresponding URLs

Collection of Note http://schema.org/ItemList

Note http://schema.org/Article

text http://schema.org/articleBody

published_at http://schema.org/datePublished

created_at http://schema.org/dateCreated

updated_at http://schema.org/dateModified

id (No need because each note has its own URL)

Home http://schema.org/SiteNavigationElement

4. Write HTML templates with Microdata

/app/views/notes/index.html.haml

GET /notes HTTP/1.1 Host: www.example.com Accept: application/vnd.amundsen-uber+json

%div{itemscope: true, itemtype: 'http://schema.org/ItemList', itemid: notes_url, data: {main_item: true}} - @notes.each do |note| = link_to note.text.truncate(20), note, rel: 'item', itemprop: 'hasPart' = form_for Note.new do |f| = f.text_field :text = f.submit rel: 'create'

{ "uber": { "version": "1.0", "data": [{ "url": "http://www.example.com/notes", "name": "ItemList", "data": [ { "name": "hasPart", "rel": "item", "url": "/notes/1" }, { "name": "hasPart", "rel": "item", "url": "/notes/2" }, { "rel": "create", "url": "/notes", "action": "append", "model": "note%5Btext%5D={text}" }, { "rel": "profile", "url": "/assets/note.alps"} ] }] } }

Collection of Note

Link

Form

%div{itemscope: true, itemtype: 'http://schema.org/Article', itemid: note_url(@note), data: {main_item: true}} %span{itemprop: 'articleBody'}= @note.text %span{itemprop: 'datePublished'}= @note.published_at %span{itemprop: 'dateCreated'}= @note.created_at %span{itemprop: 'dateModified'}= @note.updated_at = form_for @note, method: :put do |f| = f.text_field :text = f.submit rel: 'update' = button_to 'Destroy', @note, method: :delete, rel: 'delete' = button_to 'Publish', publish_note_path(@note), rel: 'publish' unless @note.published? = link_to 'Next note', note_path(@note.next), rel: 'next' if @note.next = link_to 'Prev note', note_path(@note.prev), rel: 'prev' if @note.prev = link_to 'Collection of Note', notes_path, rel: 'collection', itemprop: 'isPartOf'

/app/views/notes/show.html.haml

GET /notes/1 HTTP/1.1 Host: www.example.com Accept: application/vnd.amundsen-uber+json

Note

{ "uber": { "version": "1.0", "data": [{ "url": "http://www.example.com/notes/1", "name": "Article", "data": [ { "name": "articleBody", "value": "First note's text" }, { "name": "datePublished", "value": null }, { "name": "dateCreated", "value": "2014-09-11T12:00:31+09:00" }, { "name": "dateModified", "value": "2014-09-11T12:00:31+09:00" }, { "name": "isPartOf", "rel": "collection", "url": "/notes" }, { "rel": "update", "url": "/notes/1", "action": "replace", "model": "note%5Btext%5D={text}" }, { "rel": "delete", "url": "/notes/1", "action": "remove" }, { "rel": "publish", "url": "/notes/1/publish", "action": "append" }, { "rel": "next", "url": "/notes/2" }, { "rel": "profile", "url": "/assets/note.alps" } ] }] } }

%div{itemscope: true, itemtype: 'http://schema.org/Article', itemid: note_url(@note), data: {main_item: true}} %span{itemprop: 'articleBody'}= @note.text %span{itemprop: 'datePublished'}= @note.published_at %span{itemprop: 'dateCreated'}= @note.created_at %span{itemprop: 'dateModified'}= @note.updated_at = form_for @note, method: :put do |f| = f.text_field :text = f.submit rel: 'update' = button_to 'Destroy', @note, method: :delete, rel: 'delete' = button_to 'Publish', publish_note_path(@note), rel: 'publish' unless @note.published? = link_to 'Next note', note_path(@note.next), rel: 'next' if @note.next = link_to 'Prev note', note_path(@note.prev), rel: 'prev' if @note.prev = link_to 'Collection of Note', notes_path, rel: 'collection', itemprop: 'isPartOf'

Note

{ "uber": { "version": "1.0", "data": [{ "url": "http://www.example.com/notes/1", "name": "Article", "data": [ { "name": "articleBody", "value": "First note's text" }, { "name": "datePublished", "value": null }, { "name": "dateCreated", "value": "2014-09-11T12:00:31+09:00" }, { "name": "dateModified", "value": "2014-09-11T12:00:31+09:00" }, { "name": "isPartOf", "rel": "collection", "url": "/notes" }, { "rel": "update", "url": "/notes/1", "action": "replace", "model": "note%5Btext%5D={text}" }, { "rel": "delete", "url": "/notes/1", "action": "remove" }, { "rel": "publish", "url": "/notes/1/publish", "action": "append" }, { "rel": "next", "url": "/notes/2" }, { "rel": "profile", "url": "/assets/note.alps" } ] }] } }

= button_to 'Publish', publish_note_path(@note), rel: 'publish' unless @note.published? = link_to 'Next note', note_path(@note.next), rel: 'next' if @note.next = link_to 'Prev note', note_path(@note.prev), rel: 'prev' if @note.prev

explanation of restriction

{ "rel": "publish", "url": "/notes/1/publish", "action": "append" }, { "rel": "next", "url": "/notes/2" },

Now you can publish, but cannot go prev

3 Pros of this design procedure• DRY

• When providing both HTML and JSON

• Awareness of links and forms

• Framing the API as an HTML Web app gets you focused on these state transition

• Constraints

• “Constraints are liberating”

この設計手順の3つのメリット

HTMLとJSON両方提供するならDRY

APIをWebアプリと同じように考えることで状態遷移に着目しリンクとフォームを意識できる

「制約は自由をもたらす」

If you want to write only JSON, you should keep in mind

• To stay focused on the link/form pattern:

• Draw a state diagram

• To keep your API decoupled:

• Use view templates or representers such as Jbuilder/RABL instead of model.to_json

• Use a JSON-based format with links and forms

• In addition, it is better to use standard names such as schema.org

もしJSONだけを書くときは注意すること

リンク・フォームを意識するために状態遷移図を描きましょう

疎結合のために、model.to_jsonはやめてビューテンプレートを使いましょう

リンクとフォームを持ったJSONベースのフォーマットを使いましょう

schema.orgのような標準名を使うとさらに良いです

– 「Webを支える技術」@yohei

“WebアプリとWeb APIを分けて考えない”

“Don’t consider Web app and Web API separately”

Conclusion: Design Your Web API the same way

as an HTML Web App

• A Web API is nothing special, It just has a different representation format

• Awareness of state transitions by drawing a diagram will remind you of links and forms

結論: Web APIはHTML Webアプリと同じように設計しよう

Web APIは特別なものではなく、ただ表現フォーマットが違うだけ

状態遷移図を描いて状態遷移を意識することで、リンクやフォームを忘れずにすむ

Finally• Unfortunately, no de-facto standard JSON format,

client implementations, libraries, etc

• We can do better by focusing on the principles and constraints of REST

• Hypermedia is one of the most important elements of REST, and a key step toward building Web APIs adaptable to change

残念ながら、デファクトスタンダードがない

RESTの制約・原則を意識するともっとうまくできる

ハイパーメディアはRESTの最も重要な要素で変化に適応できるWeb APIへの重要なステップ

Build a Better & Adaptable Web API. Thank you for your attention.

• L. Richardson & M. Amundsen “RESTful Web APIs” (O’Reilly)

• 山本陽平 “Webを支える技術” (技術評論社)

• Designing for Reuse: Creating APIs for the Future http://www.oscon.com/oscon2014/public/schedule/detail/34922

• API Design Workshop 配布資料 http://events.layer7tech.com/tokyo-wrk

• https://speakerdeck.com/zdne/robust-mobile-clients-v2

• http://www.slideshare.net/yohei/webapi-36871915

• http://smizell.com/weblog/2014/solving-fizzbuzz-with-hypermedia

• 山口 徹 “Web API デザインの鉄則” WEB+DB PRESS Vol.82

References