RAPID – Resource API Design LanguagePushing the limits of language usability with XText
TED EPSTEIN, FOUNDER AND CEO
TANYA FESENKO, LEAD DEVELOPER
Ecl ipseCon NA 2015, Xtext Day
1COPYRIGHT © 2015, MODELSOLV, INC. | ALL RIGHTS RESERVED.
™
™
Overview• RepreZen: solving API interoperability• Challenge: a better API description language• RAPID Language Features
◦ Optional Fluency: accommodating coding styles
◦ Keyword Sequence: encourage, don’t Enforce
◦ Indent-Based Block Scoping: life without curly braces
• Implementation Overview• Summary• Q&A
2COPYRIGHT © 2015, MODELSOLV, INC. | ALL RIGHTS RESERVED.
COPYRIGHT © 2015, MODELSOLV, INC. | ALL RIGHTS RESERVED. 3
/ Unify Data Models
/ Federate APIs
/ Integrate Faster
The Elevator Pitch
The Elevator Pitch
COPYRIGHT © 2015, MODELSOLV, INC. | ALL RIGHTS RESERVED. 4
Make Canonical Data Models
Practical
Enable Highly Interoperable
APIs
Significantly Lower Long-
Term Integration Cost
• Frequently recommended, hard to implement• Current reality: much easier to build from
scratch than to reuse existing data models• We want to change that!
COPYRIGHT © 2015, MODELSOLV, INC. | ALL RIGHTS RESERVED. 5
Make Canonical Data Models
Practical
Enable Highly Interoperable
APIs
Significantly Lower Long-
Term Integration Cost
• Frequently recommended, hard to implement• Current reality: much easier to build from
scratch than to reuse existing data models• We want to change that!
• Reusable data models: business-oriented,technology-independent, semantically rich
• Allow variability through realization• Message payloads still conform to canonical model
Property Subsets Perspective
<party dataSource=“MSDB”>
<partyId>123</partyId>
<partyName xsi:nil=“true” nullValue=“Not Available” />
<accounts>
<account dataSource=“A2” transType=“insert”>
<accountId>XYZ</accountId>
<balance xsi:nil=“true” isRestricted=“true” />
</account>
</accounts>
</party>
Metadata Contextual Constraints
The Elevator Pitch
COPYRIGHT © 2015, MODELSOLV, INC. | ALL RIGHTS RESERVED. 6
Make Canonical Data Models
Practical
Enable Highly Interoperable
APIs
Significantly Lower Long-
Term Integration Cost
• Frequently recommended, hard to implement• Current reality: much easier to build from
scratch than to reuse existing data models• We want to change that!
• Reusable data models: business-oriented, technology-independent, semantically rich
• Allow variability through realization• Message payloads still conform to canonical model
• Service APIs speak the same language• Clients can work with multiple APIs, without multiple DTOs,
SDKs, etc.• Stop runaway spending on remedial integration technology
Changing the way we build APIs
Message Schema
COPYRIGHT © 2015, MODELSOLV, INC. | ALL RIGHTS RESERVED. 7
Business DomainModel
Realization
• Subset• Perspective• Constraints• Metadata
Resource Data
Model
Representation
• Media Type• Format• Conventions• Hyperlinks
This needs to be built into the API modeling language.
Yet Another API Description Language?YAADL? YAAPIDL?There are at least four of these...
“The nice thing about standards is that you have so many to chose from.”- Andrew Tanenbaum
COPYRIGHT © 2015, MODELSOLV, INC. | ALL RIGHTS RESERVED. 8
… and they’re pretty good.◦ Web-based editors◦ API Documentation with integrated
sandbox◦ Syntax based on YAML or Markdown◦ Interesting language-level reuse◦ REST patterns built in
If we have to build an API
modeling language, it has to be good.
… any ideas?
Who’s going to model these APIs?
COPYRIGHT © 2015, MODELSOLV, INC. | ALL RIGHTS RESERVED. 9
…andDevelopers!Developers!
Developers!
Business Analysts
Subject Matter Experts
Data Professionals
Enterprise Architects
Make it accessible, readable, versatile.
What she said.
Demo – RAPID Usability Features• Indent Syntax:
◦ Python-like block syntax, no curly braces◦ Advanced highlighting to visualize structure
• Order Assist:◦ Flexible sequence for declarations ◦ Reset to standard sequence using QuickFix or auto-format
• Optional Fluency◦ Support terse and fluent syntax with optional keywords◦ Semantic highlighting to distinguish optional vs. required keywords
COPYRIGHT © 2015, MODELSOLV, INC. | ALL RIGHTS RESERVED. 10
11
Indent-Based Syntax
Structural Highlighting -editbox.sourceforge.net
New Syntax –indents define block scope
Previous Syntax –braces for block scoping
Order Assist
12
Optional Fluency
13
Fluent
Terse
Implementation Walkthrough
COPYRIGHT © 2015, MODELSOLV, INC. | ALL RIGHTS RESERVED. 14
15
Work with Xtext
XtextGrammar
(*.xtext file)
16
Work with Xtext
XtextGrammar
(*.xtext file)Generated code (Parser and UI)
17
Work with Xtext
The rest (customized from generated code)
Lexer Formatter Content assistCode
TemplatesPrefs
Cross-reference
Scope
Code highlighter
Validation/Quickfix
Outline …
XtextGrammar
(*.xtext file)Generated code (Parser and UI)
Description GrammarCode
Customization
for (each feature):
18
Three features
Feature 1:Automate canonical ordering
19
Order Assist
20
methods
21
Implementation: Grammar
The rest (customized from generated code)
XtextGrammar
(*.xtext file)Generated code
22
Allow arbitrary order in grammar
23
Allow arbitrary order in grammar
24
Allow arbitrary order in grammarDisable code-assist and add
validator for duplicate single-valued properties to prevent
overriding
25
Sort List< INode>
Compare INodes:• Using sorting rules• Preserving:
• List delimiters• Whitespaces• Attached symbols (*, +)• Order of elements of the same
kind
Grammar Tree
Sorting rule: Sequence<GrammarElement>
26
Implementation: Grammar
The rest (customized from generated code)
XtextGrammar
(*.xtext file)Generated code
Validation
Validation
27
MyJavaValidator # checkMyElement(SemanticElement)
Validation
28
Semantic Element
Syntax Tree Node
getIncorrectlyOrderedChildren()
Incorrectly ordered Nodes
NodeModelUtils::getNode()
MyJavaValidator # checkMyElement(SemanticElement)
29
Implementation: Grammar
The rest (customized from generated code)
XtextGrammar
(*.xtext file)Generated code
Formatter
30
FormatterDefaultNodeModelFormatter#format():
31
FormatterDefaultNodeModelFormatter#format():
32
FormatterDefaultNodeModelFormatter#format():
INode:• offset• text• length• grammarElement• semanticElement
33
FormatterDefaultNodeModelFormatter#format():
Writes tokens to character stream and
then flushes it
The write methods create
formatted LineEntries with use of formatter
configuration (ElementLocators)
34
FormatterDefaultNodeModelFormatter#format():
TextRegion –offset and
length in the char stream populated in
the write methods
35
FormatterDefaultNodeModelFormatter#format():
Place for customization
36
Implementation: Grammar
The rest (customized from generated code)
XtextGrammar
(*.xtext file)Generated code
Preferences page
Usability: Preferences
37
38
Testing
1
2
3
4
5
How many permutations for five
elements ?
5!=120
39
TestingXtend Active Annotations
120 green tests!
40
Changes: Automate Canonical Ordering
The rest (customized from generated code)
XtextGrammar
(*.xtext file)Generated code
Preferences pageFormatterValidation
Enable arbitrary order
41
The rest (customized from generated code)
XtextGrammar
(*.xtext file)Generated code
Preferences pageFormatterValidation
• Errors for single-value property override
• Warnings for "incorrect" order
Changes: Automate Canonical Ordering
42
The rest (customized from generated code)
XtextGrammar
(*.xtext file)Generated code
Preferences pageFormatterValidation
Restore canonical order
Changes: Automate Canonical Ordering
43
The rest (customized from generated code)
XtextGrammar
(*.xtext file)Generated code
Preferences pageFormatterValidation
enable/disable validation,
reordering in formatter
Changes: Automate Canonical Ordering
Feature 2: Optional fluency
44
What is Optional Fluency?
45
Fluent
Terse
46
Implementation: Grammar
The rest (customized from generated code)
XtextGrammar
(*.xtext file)Generated code
Implementation
47
48
Implementation
49
Changes: Optional Fluency
The rest (customized from generated code)
XtextGrammar
(*.xtext file)Generated code
Optional fluency
Feature 3: Indent-based block scoping in Xtext
50
51
Indent-based block scoping in Xtext
Python-like notation
With curly braces
52
Indent-based block scoping in Xtext
Highlighting of boxes-editbox.sourceforge.net
53
Implementation: Grammar
The rest (customized from generated code)
XtextGrammar
(*.xtext file)Generated code
54
Grammar
NL - newlineWS – whitespaceBEGIN – indentEND - outdent
55
Grammar NL: not hidden
NL - newlineWS – whitespaceBEGIN – indentEND - outdent
56
Grammar NL: not hidden
NL - newlineWS – whitespaceBEGIN – indentEND - outdent
57
Implementation: Grammar
The rest (customized from generated code)
XtextGrammar
(*.xtext file)Generated code
Lexer
58
Custom Lexer
Custom code
Generated / Xtext code
Indent-based Syntax in Xtext -tinyurl.com/pgcowct
Insert BEGIN and END token
59
Content assist
60
Implementation: Grammar
The rest (customized from generated code)
XtextGrammar
(*.xtext file)Generated code
Content Assist
61
Custom LexersPlural now
Custom code
Generated / Xtext code
Content-assist needs its own lexer
Custom Lexers
62
Custom code
Generated / Xtext code
Content Assist
63
Also see “Enhancing Content Assist” from
XtextCON 2015 -tinyurl.com/onou2hu
Content Assist
64
Also see “Enhancing Content Assist” from
XtextCON 2015 -tinyurl.com/onou2hu
Content Assist
65
Also see “Enhancing Content Assist” from
XtextCON 2015 -tinyurl.com/onou2hu
Content Assist
66
Place for customization
Content-Assist Context Factory
67
Initialize dataAnalyze data to create content assist contexts
Content Assist
Contexts
Content-Assist Context Factory
68
Initialize dataAnalyze data to create content assist contexts
Content Assist
Contexts
Syntax tree nodes –lastCompleteNode, lastVisibleNode, currentNode, datatypeNodeSemantic model - current model
Initialize dataAnalyze data to create content assist contexts
Content Assist
Contexts
Content-Assist Context Factory
69
Start a new token, complete current one, collect available alternatives, etc…
Initialize dataAnalyze data to create content assist contexts
Content Assist
Contexts
Content-Assist Context Factory
70
Customize to respect indents
71
Implementation: Grammar
The rest (customized from generated code)
XtextGrammar
(*.xtext file)Generated code
Formatter
Formatter
72
Assign correct value to FormattingConfigBasedStream#indentationLevel
No spaces around “invisible” rules – synthetic (BEGIN and END) and NL (newline)
No line wraps should be introduced by formatter – line wraps change structure of Python-like models
73
Changes: Indent-based Syntax
The rest (customized from generated code)
XtextGrammar
(*.xtext file)Generated code
FormatterContentAssistLexer
Add BEGIN and END terminal
rules
74
The rest (customized from generated code)
XtextGrammar
(*.xtext file)Generated code
FormatterContentAssistLexer
Synthesize BEGIN and END
tokens
Changes: Indent-based Syntax
75
The rest (customized from generated code)
XtextGrammar
(*.xtext file)Generated code
FormatterContentAssistLexer
Show proposals according to indent level
Changes: Indent-based Syntax
76
The rest (customized from generated code)
XtextGrammar
(*.xtext file)Generated code
FormatterContentAssistLexer
Preserve NLs and indents
Changes: Indent-based Syntax
Summary• Still an ongoing experiment
◦ Early feedback from users has been very positive.◦ We welcome your feedback!
• API Studio is a commercial product, still pre-launch
• Offering a free license with 1-year maintenance to Conference attendees (1 per organization)
• Register at get.reprezen.com with invitation code eclipsecon2015
• Contacts:◦ [email protected]◦ [email protected]
COPYRIGHT © 2015, MODELSOLV, INC. | ALL RIGHTS RESERVED. 77
QuestionsTHANK YOU!
COPYRIGHT © 2015, MODELSOLV, INC. | ALL RIGHTS RESERVED. 78