Detection of REST Patterns and Antipatterns: A Heuristics-based Approach

Detection of REST Patterns and Antipatterns: A Heuristics-based Approach

Francis Palma, Johann Dubois, Naouel Moha, and Yann-Gaël Guéhéneuc

ICSOC 2014, Paris, France

Detection of REST Patterns and Antipatterns 2

{

"Movie": {

"director": "Roberto",

"movieID": "MV00004",

"title": "Life is Wonderful"

}

}

HTTP/1.1 200 OK

Content-Type: application/json


{

"Movie": {




}

}

HTTP/1.1 200 OK


HTTP Method


{

"Movie": {




}

}

HTTP/1.1 200 OK


HTTP Method

Resource Location

Resource Representation


{

"Movie": {




}

}

HTTP/1.1 200 OK


HTTP Method

Resource Location

Problem Context (1/2)




1. SOAP-style operation name




2. Traditional Method/Parameter style





3. Wrong HTTP Status Code





3. Wrong HTTP Status Code

4. Requested resource format unavailable

DropBox Server Response 1: Header: { x-frame-options=[SAMEORIGIN], x-dropbox-request-id=[b9a25269beb2c75fa7d7e21e1638bb9d], Connection=[keep-alive], Server=[nginx], pragma=[no-cache], cache-control=[no-cache], x-server-response-time=[64], x-dropbox-http-protocol=[None], set-cookie=[gvc=MjExODUyMTE….. expires=Tue, 26 Mar 2019 18:34:14 GMT], Transfer-Encoding=[chunked], Date=[Thu, 27 Mar 2014 18:34:14 GMT], Content-Type=[application/json], X-RequestId=[c64da98881e565a90a5dd9aecea9f049] } Body: { "hash": "f9d780e7655fe43261b4de9ec9a926eb", "revision": 2, "rev": "21e8a5a19", "thumb_exists": false, "bytes": 0, "modified": "Tue, 28 Jan 2014 21:45:31 +0000", "path": "/test", "is_dir": true, "icon": "folder", "root": "dropbox", "contents": [ { "revision": 3, "rev": "31e8a5a19", "thumb_exists": false, "bytes": 4, "modified": "Tue, 28 Jan 2014 21:46:30 +0000", "client_mtime": "Tue, 28 Jan 2014 21:46:30", "path": "/test/test.txt", "is_dir": false, "icon": "page_white_text", "root": "dropbox", "mime_type": "text/plain", "size": "4 bytes“ } ], "size": "0 bytes“ }


4

Problem Context (2/2) DropBox Server Response 1: Header: { x-frame-options=[SAMEORIGIN], x-dropbox-request-id=[b9a25269beb2c75fa7d7e21e1638bb9d], Connection=[keep-alive], Server=[nginx], pragma=[no-cache], cache-control=[no-cache], x-server-response-time=[64], x-dropbox-http-protocol=[None], set-cookie=[gvc=MjExODUyMTE….. expires=Tue, 26 Mar 2019 18:34:14 GMT], Transfer-Encoding=[chunked], Date=[Thu, 27 Mar 2014 18:34:14 GMT], Content-Type=[application/json], X-RequestId=[c64da98881e565a90a5dd9aecea9f049] } Body: { "hash": "f9d780e7655fe43261b4de9ec9a926eb", "revision": 2, "rev": "21e8a5a19", "thumb_exists": false, "bytes": 0, "modified": "Tue, 28 Jan 2014 21:45:31 +0000", "path": "/test", "is_dir": true, "icon": "folder", "root": "dropbox", "contents": [ { "revision": 3, "rev": "31e8a5a19", "thumb_exists": false, "bytes": 4, "modified": "Tue, 28 Jan 2014 21:46:30 +0000", "client_mtime": "Tue, 28 Jan 2014 21:46:30", "path": "/test/test.txt", "is_dir": false, "icon": "page_white_text", "root": "dropbox", "mime_type": "text/plain", "size": "4 bytes“ } ], "size": "0 bytes“ }

No hyperlinks to follow!


4

Problem Context (2/2) DropBox Server Response 1: Header: { x-frame-options=[SAMEORIGIN], x-dropbox-request-id=[b9a25269beb2c75fa7d7e21e1638bb9d], Connection=[keep-alive], Server=[nginx], pragma=[no-cache], cache-control=[no-cache], x-server-response-time=[64], x-dropbox-http-protocol=[None], set-cookie=[gvc=MjExODUyMTE….. expires=Tue, 26 Mar 2019 18:34:14 GMT], Transfer-Encoding=[chunked], Date=[Thu, 27 Mar 2014 18:34:14 GMT], Content-Type=[application/json], X-RequestId=[c64da98881e565a90a5dd9aecea9f049] } Body: { "hash": "f9d780e7655fe43261b4de9ec9a926eb", "revision": 2, "rev": "21e8a5a19", "thumb_exists": false, "bytes": 0, "modified": "Tue, 28 Jan 2014 21:45:31 +0000", "path": "/test", "is_dir": true, "icon": "folder", "root": "dropbox", "contents": [ { "revision": 3, "rev": "31e8a5a19", "thumb_exists": false, "bytes": 4, "modified": "Tue, 28 Jan 2014 21:46:30 +0000", "client_mtime": "Tue, 28 Jan 2014 21:46:30", "path": "/test/test.txt", "is_dir": false, "icon": "page_white_text", "root": "dropbox", "mime_type": "text/plain", "size": "4 bytes“ } ], "size": "0 bytes“ }

4




DropBox Server Response 2: Header: { x-frame-options=[SAMEORIGIN], x-dropbox-request-id=[cd12e1e844327464485842b11b530071], Connection=[keep-alive], Server=[nginx], pragma=[no-cache], cache-control=[no-cache], x-server-response-time=[110], x-dropbox-http-protocol=[None], set-cookie=[gvc=MzIwNTkxODQzNjQy.....; expires=Sat, 06 Apr 2019 22:11:47 GMT; Transfer-Encoding=[chunked], Date=[Mon, 07 Apr 2014 22:11:47 GMT], Content-Type=[application/json], X-RequestId=[d509463440ada422459335fd3c71d309] } Body: { "referral_link": "https://db.tt/AaWjP9HP", "display_name": "Francis Palma", "uid": 118690394, "country": "CA", "quota_info": { "datastores": 0, "shared": 293074019, "quota": 2147483648, "normal": 1661304356 }, "team": null, "email": "[email protected]" }


4




DropBox Server Response 2: Header: { x-frame-options=[SAMEORIGIN], x-dropbox-request-id=[cd12e1e844327464485842b11b530071], Connection=[keep-alive], Server=[nginx], pragma=[no-cache], cache-control=[no-cache], x-server-response-time=[110], x-dropbox-http-protocol=[None], set-cookie=[gvc=MzIwNTkxODQzNjQy.....; expires=Sat, 06 Apr 2019 22:11:47 GMT; Transfer-Encoding=[chunked], Date=[Mon, 07 Apr 2014 22:11:47 GMT], Content-Type=[application/json], X-RequestId=[d509463440ada422459335fd3c71d309] } Body: { "referral_link": "https://db.tt/AaWjP9HP", "display_name": "Francis Palma", "uid": 118690394, "country": "CA", "quota_info": { "datastores": 0, "shared": 293074019, "quota": 2147483648, "normal": 1661304356 }, "team": null, "email": "[email protected]" }


4



Alchemy Client Request: Header: { cache-control=[no-cache], content-type=[application/xml], connection=[keep-alive], host=[access.alchemyapi.com], accept=[application/xml], path=/calls/url/URLGetRankedNamedEntities?... get /calls/url/urlgetrankednamedentities?... user-agent=[Apache CXF 2.7.5], pragma=[no-cache] } Alchemy Server Response: Header: { content-type=[application/xml; charset=utf-8], cache-control=[no-cache], connection=[keep-alive], access-control-allow-origin=[*], content-length=[506], server=[nginx], date=[Fri, 15 Aug 2014 19:30:28 GMT] } Body: <?xml version="1.0" encoding="UTF-8"?> <results> <status>OK</status> <usage>By accessing AlchemyAPI or using information generated by AlchemyAPI, you are agreeing to be bound by the AlchemyAPI Terms of Use:http://www.alchemyapi.com/company/terms.html </usage> <url>http://www.cnn.com/2011/09/28/us/...</url> <language>english</language> <microformats> <a href="/cnn" rel="tag">cnn</a> </microformats> </results>

4



4



4


YouTube Client Request: Header: { cache-control=[no-cache],

content-type=[application/xml], connection=[keep-alive], host=[www.googleapis.com], accept=[application/xml], get/youtube/v3/subscriptions?mine=true&part=snippet&access_token=.. http/1.1=[null], user-agent=[Apache CXF 2.7.11], pragma=[no-cache]

} YouTube Server Response: Header: { x-frame-options=[SAMEORIGIN], content-type=[application/json; charset=UTF-8], cache-control=[private, max-age=300,must-revalidate,no-transform], x-xss-protection=[1; mode=block], x-content-type-options=[nosniff], expires=[Tue, 14 Oct 2014 17:57:26 GMT], etag=["PSjn-HSKiX6orvNhGZvglLI2lvk/PZz4CABe3efkukxgHuo_yc_qoJs"], content-length=[324], server=[GSE], alternate-protocol=[443:quic,p=0.01], date=[Tue, 14 Oct 2014 17:57:26 GMT], vary=[X-Origin, Referer, Origin] } Body: { "kind": "youtube#videoListResponse", "etag": "\"PSjn-HSKiX6orvNhGZvglLI2lvk/PZz4CABe3efkukxgHuo_yc_qo", "pageInfo": { "totalResults": 1, "resultsPerPage": 1 }, “ items": [ { "kind": "youtube#video", "etag": "\"PSjn- HSKiX6orvNhGZvglLI2lvk/9hUt36nrZXNpfqDh...", "id": "SRQtW-sjDGw" } ] }


4


YouTube Client Request: Header: { cache-control=[no-cache],

content-type=[application/xml], connection=[keep-alive], host=[www.googleapis.com], accept=[application/xml], get/youtube/v3/subscriptions?mine=true&part=snippet&access_token=.. http/1.1=[null], user-agent=[Apache CXF 2.7.11], pragma=[no-cache]

} YouTube Server Response: Header: { x-frame-options=[SAMEORIGIN], content-type=[application/json; charset=UTF-8], cache-control=[private, max-age=300,must-revalidate,no-transform], x-xss-protection=[1; mode=block], x-content-type-options=[nosniff], expires=[Tue, 14 Oct 2014 17:57:26 GMT], etag=["PSjn-HSKiX6orvNhGZvglLI2lvk/PZz4CABe3efkukxgHuo_yc_qoJs"], content-length=[324], server=[GSE], alternate-protocol=[443:quic,p=0.01], date=[Tue, 14 Oct 2014 17:57:26 GMT], vary=[X-Origin, Referer, Origin] } Body: { "kind": "youtube#videoListResponse", "etag": "\"PSjn-HSKiX6orvNhGZvglLI2lvk/PZz4CABe3efkukxgHuo_yc_qo", "pageInfo": { "totalResults": 1, "resultsPerPage": 1 }, “ items": [ { "kind": "youtube#video", "etag": "\"PSjn- HSKiX6orvNhGZvglLI2lvk/9hUt36nrZXNpfqDh...", "id": "SRQtW-sjDGw" } ] }


4


Why Detection is Important?

5

• From theoretical point of view

Antipatterns might strictly violate one of six REST principles

- Uniform interface, Client–server, Stateless, Cacheable, etc.

• Antipatterns in APIs might also

- hinder the understandability for clients

- limit the HATEOAS constraint

- reduce scalability and reusability

- cause security issues

- cause misapplication of HTTP methods

Detection of REST Patterns and Antipatterns

Outline

- Contribution on the detection of REST (anti)patterns

- State of the art contributions

- Our proposed SODA-R approach

- Experiments and some initial results

Outline

- Contribution on the detection of REST (anti)patterns

- State Of The Art Contributions

- Our proposed SODA-R Approach

- Experiments and Some Initial Results

Contributions

Goal: To assess the design of REST APIs

• SODA-R (Service Oriented Detection for Antipatterns in REST)

- a heuristics-based approach

• SOFA (Service Oriented Framework for Antipatterns), a framework

to support the detection of SOA antipatterns

• Empirical evidence of the presence of REST (anti)patterns


Outline

- Contribution on the detection of REST antipatterns

- State of the art contributions



Related Work

Books on REST patterns:

7 Detection of REST Patterns and Antipatterns

Related Work

Books on REST patterns:

REST antipatterns mostly online resources:

7 Detection of REST Patterns and Antipatterns

Related Work

Not applicable to REST

• resources-centric vs. operations-centric

• JSON/XML over HTTP vs. JMS or SOAP over HTTP

• human-readable documentations vs. WSDL/SCDL

• resources as nouns vs. operations as verbs

• standard HTTP methods vs. customised client-stubs


Several technology-specific approaches

• Patterns detection in SOAP Web services (Di Penta et al. 2007)

• SODOP (Demange et al. 2013)

• SOMAD (Nayrolles et al. 2013)

• SODA, SODA-W (Palma et al. 2013), EasySOC (Rodriguez et al. 2012)

Outline



- Our proposed SODA-R approach


Approach (1/5): SODA-R

SODA-R: Service Oriented Detection for Antipatterns in REST


Step 1

Analysis

Description of REST patterns and antipatterns

Heuristics

Step 2

Algorithms

Detection

Detected REST patterns and antipatterns

REST APIs Implementation

Application of Algorithms

Approach (2/5): Analysis

• Identify static properties

• Identify dynamic properties



1: FORGET-HYPER-MEDIA(response-header, response-body, http-method) 2: body-links[] EXTRACT-ENTITY-LINKS(response-body) 3: header-link response-header.getValue(“Link”) 4: if (http-method = GET and (LENGTH(body-links[]) = 0 or header-link = NIL)) or 5: (http-method = POST and (“Location:” response-header.getKeys() and 6: LENGTH(body-links[]) = 0))) then 7: print “Forgetting Hypermedia detected” 8: end if

Heuristic of Forgetting Hypermedia antipattern

Approach (3/5): Detection Heuristics

Approach (3/5): Detection Heuristics

1: ENTITY-LINKING(response-header, response-body, http-method) 2: body-links[] EXTRACT-ENTITY-LINKS(response-body) 3: header-link response-header.getValue(“Link”)

4: if (http-method = GET and (LENGTH(body-links[]) = 1 or header-link NIL)) or 5: (http-method = POST and (“Location:” response-header.getKeys() or 6: LENGTH(body-links[]) = 1))) then 7: print “Entity Linking detected” 8: end if

Heuristic of Entity Linking pattern


1: FORGET-HYPER-MEDIA(response-header, response-body, http-method) 2: body-links[] EXTRACT-ENTITY-LINKS(response-body) 3: header-link response-header.getValue(“Link”) 4: if (http-method = GET and (LENGTH(body-links[]) = 0 or header-link = NIL)) or 5: (http-method = POST and (“Location:” response-header.getKeys() and 6: LENGTH(body-links[]) = 0))) then 7: print “Forgetting Hypermedia detected” 8: end if

Heuristic of Forgetting Hypermedia antipattern

Approach (4/5): Detection


Heuristics

Detection Algorithms

Service Interfaces

Step 2.2

Dynamic Invocation

REST requests and responses

Client Authentication

REST APIs

Detected REST patterns

and antipatterns

Step 2.3

Application

Implementation

Algorithms

Interfaces

Step 2.1

Approach (5/5): SOFA Framework

Service Oriented Framework for Antipatterns

• REST Handler

- provides a wrapper layer

- automatically applies the algorithms wrapped REST APIs


Outline





Experiments (1/5): Setup

Purpose is to show

- the generalisability of SODA-R approach

- accuracy of our detection heuristics, and

- performance of the implemented algorithms

Subjects

- 8 common REST antipatterns

- 5 common REST patterns

Objects


and 8 more…

Experiments (2/5): Hypotheses

H1. Generalisability

The SODA-R approach is generalisable

H2. Accuracy

The detection heuristics have an average precision of more than 75% and a

recall of 100%, i.e., more than three-quarters of detected (anti)patterns are

true positive and we do not miss any existing (anti)patterns

H3. Performance

The implemented algorithms perform with considerably a low detection times,

i.e., on an average in the order of seconds


Experiments (3/5): Subjects

8 REST antipatterns Breaking Self-descriptiveness1

Forgetting Hypermedia1

Ignoring Caching1

Ignoring MIME Types1

Ignoring Status Code1

Misusing Cookies1

Tunnelling Through GET1

Tunnelling Through POST1

5 REST patterns - Content Negotiation2

- End-point Redirection2

- Entity Linking2

- Entity Endpoint3

- Response Caching2

[1] Tilkov, S.: REST Anti-Patterns, Available Online: www.infoq.com/articles/rest-anti-patterns (July 2008) [2] Erl, T., Carlyle, B., Pautasso, C., Balasubramanian, R.: SOA with REST: Principles, Patterns & Constraints for Building Enterprise Solutions with REST. The Prentice Hall Service Technology Series from Thomas Erl. (2012) [3] Pautasso, C.: Some REST Design Patterns (and Anti-Patterns), Available Online: http://www.jopera.org/node/442. (October 2009)


Experiments (4/5): Objects

REST APIs Online Documentations

alchemyapi.com/api/

bbyopen.com/developer/

dev.bitly.com/api.html

charlieharvey.org.uk/about/api/

dropbox.com/developers/core/docs/

developers.facebook.com/docs/graph-api/

developer.musicgraph.com/api-docs/overview/

github.com/blackducksw/ohloh_api/

integrate.teamviewer.com/en/develop/documentation/

dev.twitter.com/docs/api/

developers.google.com/youtube/v3/

developer.zappos.com/docs/api-documentation/


Experiments (5/5): Process

• Define heuristics for 8 REST antipatterns and 5 REST patterns

• Implement clients and invoke a total set of 115 methods from APIs

• Apply detection algorithms on REST requests and responses

• Manually validate detection results to identify true positives and

false negatives

• Use precision and recall to measure our detection accuracy


Outline




- Experiments and some initial results

Results (1/8): Detection (Antipatterns)

-

0.50

1.00

1.50

2.00

2.50

3.00

3.50

average_antipatterns_instance


Results (1/8): Detection (Antipatterns)


-

0.50

1.00

1.50

2.00

2.50

3.00

3.50

average_antipatterns_instance

Results (2/8): Detection (Patterns)

-

0.50

1.00

1.50

2.00

2.50

3.00

3.50

average_patterns_instances


Results (2/8): Detection (Patterns)

-

0.50

1.00

1.50

2.00

2.50

3.00

3.50

average_patterns_instances



Results (3/8): Pattern vs. Antipattern

-

0.50

1.00

1.50

2.00

2.50

3.00

3.50

Forgetting_Hypermedia Entity_Linking


Results (3/8): Pattern vs. Antipattern

-

0.50

1.00

1.50

2.00

2.50

3.00

3.50

Results (4/8): Detection


(7) A

lchem

y -

(12) B

estBuy -

(3) B

itly -

(4) C

harlieH

arvey -

(15) D

ropB

ox -

(29) F

acebook -

(8) M

usicg

raph -

(3) O

hlo

h -

(8) T

eamV

iewer -

(10) T

witter -

(9) Y

ouT

ube -

(7) Z

appos -

(115) T

otal -

Averag

e Precisio

n -

Recall -

Detectio

n T

ime -

Forgetting Hypermedia

1/1 0/0 2/2 0/0 9/10 8/8 7/7 0/0 3/3 4/4 2/3 0/0 36/38 94.58%

19.54s

1/1 0/0 2/2 0/0 9/9 8/8 7/7 0/0 3/3 4/4 2/2 0/0 36/36 100%

Results (4/8): Detection

(7) A

lchem

y -

(12) B

estBuy -

(3) B

itly -

(4) C

harlieH

arvey -

(15) D

ropB

ox -

(29) F

acebook -

(8) M

usicg

raph -

(3) O

hlo

h -

(8) T

eamV

iewer -

(10) T

witter -

(9) Y

ouT

ube -

(7) Z

appos -

(115) T

otal -

Averag

e Precisio

n -

Recall -

Detectio

n T

ime -

Forgetting Hypermedia

1/1 0/0 2/2 0/0 9/10 8/8 7/7 0/0 3/3 4/4 2/3 0/0 36/38 94.58%

19.54s

1/1 0/0 2/2 0/0 9/9 8/8 7/7 0/0 3/3 4/4 2/2 0/0 36/36 100%

Entity Linking

6/6 11/11 1/1 4/4 3/3 21/21 1/1 2/2 1/1 5/5 6/6 4/4 65/65 100% 19.90s

6/6 11/11 1/1 4/4 3/3 21/21 1/1 2/2 1/1 5/5 6/7 4/4 65/66 98.81%


Results (5/8): Hypothesis H1


• performed experiments on 12 REST APIs including well-known APIs like • analysed 115 methods in the form of HTTP requests-responses, along with their headers and bodies • 8 common REST antipatterns and 5 REST patterns





• performed experiments on 12 REST APIs including well-known APIs like • analysed 115 methods in the form of HTTP requests-responses, along with their headers and bodies • 8 common REST antipatterns and 5 REST patterns


We have a high accuracy in terms of precision and recall


Average Precision

Average Recall

Antipatterns 82.81% 90.4%

Patterns 100% 99.76%

Average 89.42% 94%


We have a high accuracy in terms of precision and recall


Average Precision

Average Recall

Antipatterns 82.81% 90.4%

Patterns 100% 99.76%

Average 89.42% 94%


The implemented algorithms perform with

considerably a low detection times

Average

Detection Times 1.124s-

Antip

atte

rns

Execution Times 20.933s-

Total (Detection + Execution)

22.056s-

Detection Times 0.022s- Patte

rn



20.436s-

Average 21.246s-




The implemented algorithms perform with

considerably a low detection times

Average

Detection Times 1.124s-

Antip

atte

rns



22.056s-

Detection Times 0.022s- Patte

rn



20.436s-

Average 21.246s-

Results (8/8): Validation Summary

• Define heuristics and implement detection algorithms for 13 REST

patterns and antipatterns

• Perform detection on 12 well-known REST APIs, including

• Average precision of detection algorithms is 89.42% and recall is 94%

• Detailed results and more materials on sofa.uqam.ca/soda-r/


Conclusion


Conclusion


Conclusion


Conclusion


Future Work

- Additional experiments with more REST APIs and methods

- Dictionary-based (WordNet, Stanford Core NLP) lexical analysis of

REST APIs to detect linguistic antipatterns

- Enrich the antipatterns catalog


Questions ?