29
25+ pages of knowledge for API enthusiasts. Learn about exciting topics such as API Development, Management, Design and Platforms & Software as a Service API Whitepaper Selection @api_conference #APICON API Conference @apiconf apiconference.net The API revolution continues!

The API revolution continues! API Whitepaper Selection...Looking to 2020 and beyond, the proportion of data produced and consumed in realtime is growing exponentially . 4 WITPAPR API

  • Upload
    others

  • View
    9

  • Download
    0

Embed Size (px)

Citation preview

Page 1: The API revolution continues! API Whitepaper Selection...Looking to 2020 and beyond, the proportion of data produced and consumed in realtime is growing exponentially . 4 WITPAPR API

25+ pages of knowledge for API enthusiasts.

Learn about exciting topics such as API Development, Management, Design

and Platforms & Software as a Service

API Whitepaper Selection

@api_conference #APICON

API Conference @apiconf apiconference.net

The API revolution continues!

Page 2: The API revolution continues! API Whitepaper Selection...Looking to 2020 and beyond, the proportion of data produced and consumed in realtime is growing exponentially . 4 WITPAPR API

API DesignThe demand for realtime APIs 3Realtime API Family helps us to reduce complexity by Kieran Kilbride-Singh

API DevelopmentWhat APIs Can and Cannot Do 6How to avoid common API misconceptions and the resulting disillusionment by Erik Wilde

Go for gRPC? 11Develop gRPC services with Go by Rainer Stropek

API Platforms & Software-as-a-ServiceX-Road – The Free and Open Source Data Exchange Layer 17A centrally managed distributed Data Exchange Layer between information systems by Petteri Kivimäki

Regulation as a Service 21Rethinking APIs by Khallai Taylor

API ManagementShopware 6 and the new API-First Approach 23Putting the API on the Acid Testby Thomas Eiling

APIs have Properties – find and test them! 26Taking executable specifications seriouslyby Dr. Michael Sperber

content

Page 3: The API revolution continues! API Whitepaper Selection...Looking to 2020 and beyond, the proportion of data produced and consumed in realtime is growing exponentially . 4 WITPAPR API

3

WHITEPAPER API Design

www.apiconference.net

by Kieran Kilbride-Singh

IDC predict that by 2025 1/3 of all data produced glob-ally will be realtime. The engineers and organizations that make up the realtime ecosystem, Ably included, have yet to agree on how we describe the APIs we’re creating and consuming that are powering this growth. The problem is that there are various ways to describe APIs that provide realtime functionality.

API evangelist Kin Lane, and many others, have been writing and talking about these new types of APIs for a while. Event-driven seems to be the most common descriptor. Gartner has adopted the term, stating that by 2020 50 % of all managed APIs will be event-driven. But still today there is no consistency or consensus between what terms like realtime API, event-driven API, or streaming API refer to. Often they’re used interchangeably.

As an engineering team and API provid-er working on a global pub/sub messaging platform, we work with these ideas every-day. Talking to our users we see the lan-guage they use to describe both what we do and what they achieve using our APIs. And we’ve watched discussions around event-driven architecture, webhooks, and streaming data proliferate. Over the years we’ve thought extensively about the best terminology to use and arrived at what we call the Realtime API Family

Defining realtimeWhen it comes to realtime in the context of user interaction, it can be defined as a function or interaction perceived as imme-diate. As a concept in this context then, re-altime is easy enough to understand. Ably

defines it as anything that happens in under 100ms as this is typically the threshold for perceiving something as happening immediately.

Any API designed such that data flowing from pro-ducers to consumers happens in the shortest amount of time possible can therefore be described as realtime.

Ably has settled on realtime API as an umbrella for event-driven, streaming, pub/sub, push, and other APIs. We’ve seen many more than four ways of describing this, but these are the most popular. It’s why we call it the Realtime API Family (Image 1).

Image 1: Realtime API Family

Realtime API Family helps us to reduce complexity

The demand for realtime APIs Looking to 2020 and beyond, the proportion of data produced and consumed in realtime is growing exponentially

Page 4: The API revolution continues! API Whitepaper Selection...Looking to 2020 and beyond, the proportion of data produced and consumed in realtime is growing exponentially . 4 WITPAPR API

4

WHITEPAPER API Design

www.apiconference.net

Streaming APIsA stream is a means of transporting data. Streaming is a consumer pattern that describes how consumers receive events through a stream.A streaming API will commonly address issues of data integrity with:

• Message ordering - ensure messages are delivered in the order they were published

• Stream continuity / resume - upon disconnection, resume from the disconnected point within a set pe-riod of time

• Contiguous serial numbers - a simple series of `ACK` or `NACK` responses, each addressing a contiguous sequence of messages

Kafka offers a streaming API following this model for internal systems.

Pub/Sub APIsPub/Sub (Publisher/Subscriber) is an infamous messag-ing design pattern.

When it debuted in 1987 it solved loose decoupling of servers. The earliest distributed systems used to send events between servers using Pub/Sub, but all within reliable networks. This pattern operates by publishing messages on a topic (or channel) within a message bus, and subscribers can listen for events based on those top-ics.

Pub/Sub is over 30 years old now. The design makes no attempt to define semantics around ordering or continuity (loss of connection). While it’s still a pat-tern used widely today, these are issues that must be considered and addressed in our world of unreliable connections.

Ably provides a pub/sub (Image 2) API following this model but with guarantees around ordering, continuity, idempotency, and more.

Push APIsPush is another umbrella term for every API that is real time.

But it’s just a producer messaging pattern. It means that data is pushed upstream over a connection, versus a pull mechanism used by the request/ response pat-tern.

An example is helpful. A push notifications API (Im-age 3) is an example of a push API. Or a sporting event might send a single score update that traverses multiple systems via push APIs, resulting in millions of messages to global fans within a few hundred milliseconds.

Push can also refer to triggering a new request as with a Webhook. Or it can mean ‘push subscription’ – i.e. a pro-ducer needs to reach out to a consumer. WebSub is an ex-ample of this. But that’s beyond the scope of this article.

Event-driven APIsEvent-driven is an architectural design pattern that de-fines how a system processes data.

Image 2: Ably’s Pub/Sub Channels API acts as a serverless message bus

API 365 Workshop: Building APIs and managing them with serverless, containers, API gateways and AzureDr. Roberto Freato (Freelance)

The main focus of this workshop is to have a look into the API ecosystem to approach the problem with good insights and to organize things to be reliable, available and to reduce the overall governance effort.

Page 5: The API revolution continues! API Whitepaper Selection...Looking to 2020 and beyond, the proportion of data produced and consumed in realtime is growing exponentially . 4 WITPAPR API

5

WHITEPAPER API Design

www.apiconference.net

event in an event-driven world triggers a chain of events which must be processed down-stream, extending through the entire data supply chain. All components in this sup-ply chain are reactive, rapidly responding to events and per-forming onwards processing.

The time-bound, reactive nature of this data supply chain causes increased engi-neering and infrastructure complexity. And when com-pared to REST APIs following the request/response model, the complexity is inverted and put onto API producers rather than API consumers.

However, that’s a conver-sation beyond the scope of this article. But it’s something we’ve spoken about before at API

And so we arrive at Realtime APIs

Streaming, Pub/Sub, and Push are all patterns that can be delivered through event-driven architecture. But the outcomes of streaming, push, pub/sub, and event-driven are all realtime functionality. They’re all means of get-ting data from producer to subscriber in the shortest possible time. So we can call them realtime APIs.

But you can’t have a realtime API unless it’s delivered as an event-driven, pub/ sub, streaming, or push API. Hence how we’ve settled on the umbrella term of real-time API to encompass them all.

Navigating complexity and fragmentation As you can see, it’s easy to use these terms interchangea-bly despite them all being different. The realtime ecosys-tem is still maturing and changes all the time. Eventually we believe developers and organizations will naturally come to a shared and standardized terminology for APIs designed to deliver real time functionality.

Until then, the Realtime API Family helps us to reduce complexity when we explain our cloud infrastructure and APIs to the developer community, our users, and our potential customers.

Image 3: Ably’s unified push notifications API

Do we have a Packaging Problem?Maxim Zaks (Self Employed)

Designing an API is a complex ende-avour. It contains of many details and considerations. One of such conside-rations is: How do we package data we are sending around? In this talk, we

will discuss different packaging techniques like CSV, XML, JSON, Protocol Buffers and FlatBuffers. We will evaluate the benefits and pitfalls those techniques bring to the table.

Kieran Kilbride-Singh (Product Marketing Manager) is writer and marketer with enough technical know-how to be dangerous in a GitHub repo. He first flexed his fingers writing about topics like interoperability in embedded IoT devices. Nowadays he thinks and

writes about realtime infrastructure, event-driven APIs, and next-generation connected experiences for Ably.io.

It simply states a system should be responding/re-acting to events as they happen. Streaming, Pub/Sub, and Push are all messaging patterns that can delivered through an event-driven architecture. As such they can all fall under the umbrella of event-driven APIs.

Most of us know what an event-driven API is but an example never hurts. Unlike traditional request/re-sponse APIs where data is requested, event-driven APIs push data from a producer to a consumer. They can be quite simple or very complex.

In the above example a retailer needs an event-driven architecture to respond to things as they happen. An

Page 6: The API revolution continues! API Whitepaper Selection...Looking to 2020 and beyond, the proportion of data produced and consumed in realtime is growing exponentially . 4 WITPAPR API

6www.apiconference.net

WHITEPAPER API Development

by Erik Wilde

Working with many global organizations in the API space, most recently as part of the Axway Catalyst Team, our role very simply is helping customers make good decisions. This of course also involves helping cus-tomers avoid bad decisions. One such bad decision is to have an incomplete understanding of the complexity of digital transformation and APIs, and to expect too much from just "doing APIs" without making sure that all necessary factors for success are in place.

I'll explain 5 common misconceptions about the powers of APIs. We expect these good things to hap-pen when using APIs, but we also have to understand that they are not quite as simple, and that success will depend on additional complexities also being addressed. Better understanding these complexities from the very beginning will help to avoid disillusionment, and will also help to make sure that all necessary ingredients for a successful API journey are considered when starting the journey.

#1: Untangling Monoliths! Of Course!Of course APIs are essential for improving modularity, allowing entangled IT systems to be decomposed into loosely coupled ecosystems and thus being easier to maintain and extend. Of course! (Image 1)

One common motivation to use APIs is to get rid of monolithic solutions that often have a variety of prob-lems associated with them, most importantly being hard to change, hard to scale, and hard to integrate with oth-

er solutions. By definition, APIs (in their modern sense of "networked interfaces") are essential in this picture, because untangling means that the resulting modular structure needs communications to function, and these will be using APIs.

The hard work in this picture, though, is to come up with a good modularization where the resulting struc-ture follows structured design principles. There are many theories and practices how to come up with a "good" structured design that represents a complex sys-tem. The hard work of doing it will lead to components that then communicate through APIs, but these APIs are an effect and not the cause of the untangling.

Sam Newman's "Monolith to Microservices" pro-vides a good guide how to systematically decompose a monolithic system into one that is composed out of many communicating components. It is telling that the book has a lot to say about decomposition patterns and

Image 1: Untangling Monoliths

How to avoid common API misconceptions and the resulting disillusionment

What APIs Can and Cannot Do APIs sometimes are perceived as having almost magical properties: Since they are so essential to digital transformation, and all of our digital experiences to-day are powered by APIs, the conclusion is that once you have and use APIs, all the wonders of digital transformation are simply going to happen. Of course that's not necessarily the case, and the purpose of this article is to explain some of the useful things where APIs play an essential role, but where APIs are necessary but not sufficient.

Page 7: The API revolution continues! API Whitepaper Selection...Looking to 2020 and beyond, the proportion of data produced and consumed in realtime is growing exponentially . 4 WITPAPR API

7www.apiconference.net

WHITEPAPER API Development

strategies, but focuses little on how to design commu-nications between the resulting components. The suc-cess of a good decomposition is defined by identifying the right components, and then carving out parts in ways that allow the overall system to remain functional throughout the transition. APIs will be created as a side-effect of doing this, but success is a function of decom-position patterns and strategies.

The worst-case scenario is that APIs will actually make things worse, because decomposition was not done properly. In this case, the newly created API-based architecture suffers from chatty APIs, and the APIs themselves are not all that useful for unlocking new value. This result often is referred to as a "distributed monolith", but this result cannot be attributed to "APIs negatively impacting performance"; it is just bad system design. In addition, bad API design choices may further negatively impact the resulting system, but the root cause in this case most often is a design where compo-nents are not loosely coupled.

#2: Scaling Up! Of Course!Of course APIs allow you to more easily scale IT re-sources by only scaling up those parts that actually need to be scaled. Of course! (Image 2)

Scaling things better is one of the reasons why APIs may be considered. When you have decoupled compo-nents, the reasoning goes, it

becomes easier to scale up those parts that need scal-ing, while other parts of the system can be left alone. In many cases when people consider microservices or serverless, one of the main values they might be interest-ed in is scaling. For microservices, you then might have to invest in how to implement scaling with something like Kubernetes. For serverless, scaling works easier, where given a sufficiently capable platform, scaling can be done transparently and is simply a question of how often serverless functions get invoked.

Since APIs are essential for both microservices and serverless (they are part of the definition of what these things even are), once again one might come to the con-clusion that with APIs in place, scaling becomes much easier. Once again, this is not necessarily the case. When it comes to scaling, and in particular the selective scal-ing that API-based components can deliver (only scal-ing those components that need to be scaled), then once again architecture is much more important than APIs.

A little while ago, when Representational State Trans-fer (REST) was a much-discussed topic, one often over-looked aspect of it was that one of the REST constraints is for services to be stateless. That way, each invocation of a service strictly is just a function of the invocation (and not of some shared conversation state). This de-sign allows scaling to be simpler, because services can be scaled without having to keep track of conversation state. This is a useful design, but it is one that depends on service designers specifically creating stateless de-signs. For services not being stateless, scaling gets con-siderably harder because requests cannot as easily be used for scaling up as required.

As before, there also is a worst-case scenario. If the system design does not take shared state into consid-eration, scaling may have unintended side-effects. It still may be possible to scale the component on demand, but scaling up the context that it needs (because it is not stateless) may have an unfortunate ripple-effect. This

possibly can be mitigated by sophis-ticated state-sharing approaches, but the underlying problem often is that of "implicit shared state".

#3: Monetization! Of Course!Of course APIs can be new sources of revenue, allowing cost-effective ex-perimentation with digital offerings and creating new revenue streams beyond the traditional sources of in-come. Of course! (Image 3)

There is an abundance of pres-entations that highlight how some companies make a lot of money with their APIs. The most prominent ex-amples may be Stripe and Twilio.

Image 2: APIS pneumatic tubes

Image 3: Monetization

Page 8: The API revolution continues! API Whitepaper Selection...Looking to 2020 and beyond, the proportion of data produced and consumed in realtime is growing exponentially . 4 WITPAPR API

8www.apiconference.net

WHITEPAPER API Development

Some of these presentations talk about the "API Econ-omy" and then jump to the conclusion that this means that money is being made by charging for API access. For organizations considering their API journeys, this looks like a tempting route, opening up new sources of revenue just by exposing APIs and charging for them. Some API vendors also rather aggressively advertise this sort of direct API-based income, hoping that potential customers will factor this into their ROI analyses.

However, the reality is that it is a small minority of cases where API value manifests itself directly in paid API access. In the majority of cases, API value mani-fests itself in enabling more value chains, and a culture in which these value chains can be built rapidly and with little coordination. For this reason, the popular "API-as-a-Product" moniker is a rather poor naming choice, as it seem to imply that the API itself is a product that can be monetized. Talking about "Product as API", on the other hand, makes it clear that value is created by prod-ucts, and that APIs simply are the connective tissue that allows this value to be unlocked and used more easily.

Once again, it is undoubtedly true that APIs are nec-essary for value being created in today's digital world, and that being better with APIs increases your chances of realizing more value. But the value is in what you do and how you allow these things to be composed in digitalized value chains. If what you do has little inher-ent value, then your efforts of digitalizing your scenario may not create a lot of additional value. And even if what you do has a lot of inherent value waiting to be realized, it is unlikely that exposing just the right API is what keeps you from realizing that value. What we see much more often are companies struggling to do the right business and organizational changes, so that the latent value in their products can be realized by building great and valuable experiences around them.

Of course, we also have a worst-case scenario here. If the perception of API monetization is such that API

value manifests itself in directly API-gener-ated revenue, this introduces an incomplete value analysis for the API journey. Nobody has yet figured out the perfect formula how to precisely evaluate the value of an API (so that investments could be targeted towards API-enabling those capabilities first that have the biggest potential), but it is cer-tainly not as simple as looking at the rev-enue generated through API access. Fully understanding value chains and evaluating and predicting their success never has been a hard science for as long as organizations have started planning and building them. This is no different for digital value chains, so make sure that you take a holistic view.

#4: Communications! Of Course!Of course APIs are the way how to com-municate in digital environments, and via

APIs all capabilities can easily interact throughout the entire organization and beyond. Of course! (Image 4)

APIs are a language: They allow peers to communi-cate in order to achieve some goal. As such, APIs sup-posedly should allow teams to become more decoupled, thereby reducing the communication and coordination overhead. In an ideal scenario, inter-team communica-tions entirely happen through APIs or associated chan-nels (such as documentation and support channels). In reality, things often are a bit trickier, and that's in part

Image 4: Difficult communication

API for the masses: Challenges in exposing a public car APIHolger Dietrich, Katja Weisheit (pentacor GmbH)

The combination of business functions from different companies offers im-mense potential for innovation and new business models. Entire, cross-industry ecosystems are emerging. If you want to be part of these ecosystems, you have to open up your business and think about publishing APIs. They offer well-defined endpoints and provide public access to business functions

and objects. However, it’s important to consider that data is extremely valuable and highly sensitive for many organizations. Our field report shows how an automotive manufacturer’s car API is exposed using an API management platform. We discuss both, technical and organizational challenges and our best practices to solve them. Moreover, we want to talk about the increased demands for security, availability, and stability.

Page 9: The API revolution continues! API Whitepaper Selection...Looking to 2020 and beyond, the proportion of data produced and consumed in realtime is growing exponentially . 4 WITPAPR API

9www.apiconference.net

WHITEPAPER API Development

because the language of the API needs to be at a useful level for everybody involved.

For example, one scenario we see often is that API efforts start on the IT side. One initial initiative may be to create "System APIs", which often are relatively low-level APIs representing capabilities provided by certain systems. This often translates to a poor experience for those teams a bit further away from system details: They are looking for solutions to their problems, and not for access to implementation details. It often takes quite a number of these "System APIs" and their orchestration in specific ways to accomplish tasks that from a business point of view should be relatively easy to do. There is a "language mismatch" of the APIs being built from the system point of view, and the API consumers looking for APIs that address their domain problems.

This problem is best solved by looking at APIs as business-level capabilities: They must have a value

Image 5: Organization

proposition why others might use them, and solving that problem in the easiest way possible then becomes the guiding principle for identifying and designing APIs. While this may sound easy, it often is a hard problem to solve, because it requires a functioning dialog between business and IT. This may be a natural thing for com-panies born digital, but it can be a real challenge for companies where business and IT are still two relatively disconnected parts of the organization.

The worst-case scenario is that technical APIs deepen the rift between business and IT. IT thinks they are de-livering value because they make capabilities available. Business thinks "this is just another tech thing" because they don't see clear business value being exposed in the technical APIs. The remedy for this is for both sides to better understand each other, and to work towards es-tablishing and using a shared language. IT needs to fo-cus their efforts on exposing APIs that have a clear value

Page 10: The API revolution continues! API Whitepaper Selection...Looking to 2020 and beyond, the proportion of data produced and consumed in realtime is growing exponentially . 4 WITPAPR API

10www.apiconference.net

WHITEPAPER API Development

proposition and accomplish a business purpose. Busi-ness needs to start "talking API", not to the point where they design JSON that they expect as responses, but to the level where they think in terms of designing and de-livering products as digitalized value chains, based on existing APIs, and exposing their own APIs. That is in essence what a true "API Culture" is all about.

#5: Organizational Issues! Of Course!Of course APIs can help with improving the way how teams communicate and how the organization can in-novate. Of course! (Image 5)

APIs are popular these days because, among other reasons, large organizations are slow at adapting to changes, and more modern organizations that are show-ing how to be faster are doing so by re-organizing them-selves, re-organizing the way they build products, and this reflects itself in API-centric organizations and cul-tures. There is no way escaping Conway's Law, meaning that in order to build something that is non-monolithic and can be changed more easily, you need an organiza-tion that has the same structural properties.

APIs play an essential role in this picture, because they greatly reduce coordination overhead. Instead of doing one-off "integration" projects, capabilities are exposed through APIs, and anybody needing them can simply access them, ideally without ever talking to the team that owns that capability. This approach scales better, and furthermore creates an abundance of options when it comes to teams designing new experiences and look-ing for existing capabilities that they can leverage. In a perfectly functioning organization, teams are decoupled and can work at their own speed, delivering and evolv-ing products as needed.

In reality, changing a large organization to this ideal picture of decoupled and empowered teams is a much

harder challenge than designing the APIs that they will produce and need. And of course, even if an organiza-tion has been changed successfully, it still takes careful design and execution of an API strategy and an API pro-gram. But in our experience, once it comes to these API-specific issues being the remaining problem, the hardest part of transformation has been solved.

The worst-case scenario for this is that organization leaders see digital transformation first and foremost as a technical challenge, and focus on technology indicators. Once APIs are being created, the assumption then is that good things will follow. We see many organizations losing months or years of precious time because they don't fully appreciate the implications for the business and organizational pillars. In our work with large com-panies, a substantial share of our time is to make sure that all the pillars necessary for digital transformation (business, organizational, technology) are addressed, and from that perspective APIs simply are a sideeffect of digital transformation, and not a cause.

ConclusionsThis article presents 5 common misconception about what APIs can do, and makes the point that they actu-ally can do these things, but not by themselves. APIs are necessary but not sufficient for many important chal-lenges and opportunities in today's increasingly digital organizations. The most important takeaway of this article is that it always takes a holistic view: APIs are part of many solutions in the digital space, but they are never the solution themselves. Luckily, organizations nowadays can benefit from a lot of experience, they can choose between many available products, and they can rely on trusted advisors for their journey. All these is-sues discussed above are very real and relevant challeng-es for many organizations today. Make sure that your API strategy and your API program take a holistic view. Many reports about "digital transformation initiatives" show high failure rates, and by looking beyond just the technical aspects, you drastically improve your chances to make meaningful changes that truly transform your organization.

This article is based on a presentation given at the API Conference in Berlin in October 2019.

The slides are available online at http://dret.net/lec-tures/apicon-2019/api-limits

API Workshop: Security with OAuth 2.0 and OpenID Connect – Get started as an API Security ExpertMatthias Biehl (API-University.com)

This workshop offers an introduction to API security with OAuth 2.0 and Ope-nID Connect. After taking this workshop you will understand the foundations and current best practice of API security.

You will get to know the actors in OAuth and OpenID Connect, the endpoints and tokens involved and how these elements interact in so-called flows. A picture says more than 1000 words, so we visualize these flows as easy-to-understand sequence diagrams.

Erik Wilde works for Axway as part of their Catalyst team. The goal of this team is to help customers make good decisions by helping them on their API journey. A core aspect of this role is to make sure that custom-ers have an API strategy, establish an API program,

and use API guidelines to cultivate and evolve their API land-scape. Part of this journey also is understanding that APIs them-selves are no silver bullet, and that APIs can only realize their full potential when they are managed in the greater context of digital transformation.

Page 11: The API revolution continues! API Whitepaper Selection...Looking to 2020 and beyond, the proportion of data produced and consumed in realtime is growing exponentially . 4 WITPAPR API

11www.apiconference.net

WHITEPAPER API Development

by Rainer Stropek

In practice, however, REST does not fit every problem. gRPC is a more general protocol that can be used to call arbitrary functions across process or network bounda-ries. In this article I describe how to use the Go pro-gramming language to develop efficient gRPC services that are easily accessible from other platforms such as .NET and C#.

Why gRPC and not REST?Many microservices don't care about storing data in da-tabases, but offer different functions. Accordingly, it is difficult to force them into the corset of REST. Is the ac-tivation of a heating function on an IoT device an insert or update? How does a service for restarting a VM fit into the concept of REST? How do you model a service for sending SMS messages so that the API can be easily understood by others? The concept of RPC is ancient and more general than REST. All operations are traced back to a collection of commands that receive param-eters and return the result.

The advantage of this generic approach is that it can be applied to a larger number of problems without hav-ing to squeeze them into a poorly fitting architectural principle. The disadvantage is that each RPC interface can be modeled differently, while RESTful APIs - where they conceptually fit - are more consistent and therefore easier to understand and use. In practice, I recommend avoiding discussions about one or the other. There is nothing to be said against offering a cloud service with a RESTful API and a gRPC API in case of doubt. The logic

in the background should be the same, only the commu-nication protocol can be chosen by the user depending on the technology and use case.

Of course, a modern RPC system like gRPC is very different from its ancestors from the 70s. It has become much more powerful and supports, for example, the ef-ficient transmission of complex data structures using protocol buffers (ProtoBuf [1]) and streaming in both communication directions. Frameworks for developing gRPC servers and generating gRPC clients are avail-able for all widely used programming languages [2]. In my experience the tools for code generation are better and more stable than many of the code generators for Swagger aka Open API. In addition, gRPC is based on HTTP/2, which gives a comparatively better perfor-mance for communication between microservices.

After this short introduction, I will not go into the basics of gRPC and ProtoBuf in this article, as they have already been described elsewhere. Let's now take a look at the development of gRPC services with Go.

Why Go?Go is primarily used for the development of web ap-plications [3]. The focus is on API and RPC services, dynamic HTML generation follows with a slight gap. Go is one of the leading programming languages in terms of container and cloud technology (e.g. Docker, Kubernetes, Terraform). A platform such as .NET is much more generally usable compared to Go, where other application areas such as UI clients, mobile apps, games etc. play an important role besides web pro-gramming.

Develop gRPC services with Go

Go for gRPC? When most people think of APIs for microservices, they think of HTTP-based, RESTful Web Services first. The REST paradigm is the predominant one when it comes to services publicly available on the Internet that many develop-ment teams deal with. It also fits quite well if the problem can be traced back to CRUD (Create, Read, Update, Delete) services in the broadest sense.

Page 12: The API revolution continues! API Whitepaper Selection...Looking to 2020 and beyond, the proportion of data produced and consumed in realtime is growing exponentially . 4 WITPAPR API

12www.apiconference.net

WHITEPAPER API Development

Specialization is a double-edged sword for program-ming languages. On the one hand, it is a disadvantage because a company cannot consistently use one lan-guage for all application areas. On the other hand, it is an advantage because the language and the libraries and tools available for it can focus on a specific application area and are correspondingly good.

In my opinion, Go is very well suited for the develop-ment of microservices that offer RESTful or gRPC APIs. Go is a simply structured language that emphasizes consistency. Stability is especially important in Go, and APIs evolve more slowly in Go than in the fast moving UI world.

By generating native code instead of Intermediate Language, good start times and good runtime perfor-mance can be achieved. Go generates a compact exe-cutable file for a microservice, which - if you want it to - even runs without any external dependencies. The compiled gRPC client shown later in Listing 4 has with-out debug information only about 8MB and you don't need to install any frameworks or other system require-ments to run it. This compactness and performance are advantages when using microservices in cloud comput-ing environments.

So there are good reasons to consider Go when devel-oping gRPC Services.

ProtoBufThe development of a gRPC API starts in Go as in all other languages with ProtoBuf files. ProtoBuf is a mechanism for language and platform independent se-

rialization of data for its transmission and/or storage. The technology is therefore used similarly to JSON or XML, but it is much more optimized for performance through more compact, binary data structures. In addi-tion to data structures, ProtoBuf also allows the defini-tion of services (=APIs) that use the data structures for parameters and return values. These services become the gRPC API.

In this article I use the current version 3 of ProtoBuf (aka proto3). In practice you still find proto2 in exist-ing applications, but new solutions should be based on version 3.

Installing the compilerEnough with the theory, let's look at an example. Be-fore we can get started, we need to download and install the latest version of the ProtoBuf compiler (aka protoc) from GitHub [4]. Don't forget to add the compiler's bin directory to the PATH environment variable so that protoc can be found on the command line. Then, we install the Go plugin for protoc with go get -u github.com/golang/protobuf/protoc-gen-go, which is responsi-ble for generating go code. The Go plugin can be found in the directory $GOPATH/bin (or %GOPATH%\bin on Windows). Also make sure that this directory is ref-erenced in the PATH environment variable.

You have already experimented with gRPC for .NET and wonder why the preliminary work at Go is so com-plicated? With .NET you had to do exactly the same thing, only Visual Studio did all the steps without you noticing. I use Visual Studio Code (VSCode for short) for Go development and therefore the above mentioned steps are necessary to set up the development environ-ment. A tip for all those who use VSCode like me: There are some plugins for VSCode that add proto3 file sup-port to the editor. Just search for ext:proto3 in the Ex-tensions Manager and choose the one you want from the available VSCode extensions.

Simple proto3 fileProto3 files always follow the same naming conventions regardless of the programming language used:

• File names follow the snake_casing_name scheme. The same applies to field names and enum values, whereby capital letters are used for the latter.

• CamelCasing is used for message names, enums and service names.

• We recommend that you use the keyword package to divide messages and services into namespaces so that identifiers are not used twice.

These naming conventions apply regardless of the pro-gramming language you use to program gRPC. The code generators will convert the ProtoBuf naming conventions into the appropriate naming conventions for the target language (for example conversion from camelCase to PascalCase).

Listing 1: greet.proto

syntax = "proto3";

// Language-specific option used go code generator.option go_package = "greet";

package demo;

// The greeting service definition.service Greeter { // Sends a greeting rpc SayHello (HelloRequest) returns (HelloReply);}

// The request message containing the user's name.message HelloRequest { string name = 1;}

// The response message containing the greetings.message HelloReply { string message = 1;}

Page 13: The API revolution continues! API Whitepaper Selection...Looking to 2020 and beyond, the proportion of data produced and consumed in realtime is growing exponentially . 4 WITPAPR API

13www.apiconference.net

WHITEPAPER API Development

Listing 1 shows a Hello World proto3 file that de-fines a service with one message for each parameter and return value. You compile it with protoc ./greet.proto --go_out=plugins=grpc:. The parameter value plugins=grpc tells the ProtoBuf Go plugin to generate gRPC compatible code. The result is a file greet.pb.go with generated Go code, which should not be modified, because it is generated every time the plugin is compiled.

Worth mentioning in Listing 1 is the option go_pack-age. The concept of ProtoBuf Packages cannot be directly assigned to Go Packages. Therefore, it is rec-ommended to explicitly specify the go package for the generated code in proto files.

The Greeter.SayHello method shown in Listing 1 is a so-called Unary RPC Call. The client sends a single request and the server responds with a single response. gRPC can also handle client-side, server-side and bidi-rectional streaming. This will be discussed later in this article.

Generated Message StructuresThe Go-code generated from the proto3 file is easily readable. Everything that starts with XXX_ can be ig-nored. These elements (e.g. fields in ProtoBuf messages) are necessary implementation details, but are not part of the public API.

The proto3 messages become Go structures (struct). The properties of the individual fields are mapped in Go as tags for the protobuf and json packages. Here is an example of the code generated from HelloRequest.Name: Name string ̀ protobuf: "bytes,1,opt,name=name,proto3" json: "name,omitempty"`.

gRPC ServiceA Go Interface is generated for each service of the proto file. In the case of the example from Listing 1, this is the GreeterServer interface. As a developer of the gRPC ser-vice, you have to implement the methods of the interface and do the actual work that the service is supposed to provide. Listing 2 shows a simple Hello World imple-mentation of the gRPC service from Listing 1.

Listing 3 contains a simple variant of the code that can be used to start the gRPC server. In practice, this

Listing 2: Implementation of the gRPC service

package main

import ( "context" "fmt"

"github.com/Samples/BeyondREST/GoGrpcServer/greet")

type service struct {}

func (s service) SayHello(ctx context.Context, req *greet.HelloRequest) (*greet.HelloReply, error) { // If you are new to Go, you can learn more about context. // Context at https://blog.golang.org/context and // https://golang.org/pkg/context/.

return &greet.HelloReply{ Message: fmt.Sprintf("Hello %s", req.Name), }, nil}

Listing 3: Starting the gRPC serverpackage main

import ( "context" "fmt" "io" "log" "net" "time"

"github.com/Samples/BeyondREST/GoGrpcServer/greet" "google.golang.org/grpc")

[...] // Add code from Listing 2 (without import and package)

func main() { listen, err := net.Listen("tcp", ":8080") if err != nil { panic(err) }

// register service server := grpc.NewServer() greet.RegisterGreeterServer(server, service{}) greet.RegisterMathGuruServer(server, mathGuruService{})

// start gRPC server log.Println("starting gRPC server...") server.Serve(listen)}

The rocky Road of the OpenAPI ASTDr. Vincenzo Chianese (Stoplight)

Let‘s go through my last 5 years in the API industry as a tooling company de-veloper seeing how the OpenAPI 2 and 3 standards have their own quirks and made my life a mess.

Page 14: The API revolution continues! API Whitepaper Selection...Looking to 2020 and beyond, the proportion of data produced and consumed in realtime is growing exponentially . 4 WITPAPR API

14www.apiconference.net

WHITEPAPER API Development

code will be more complex as you will add, for example, TLS encryption (see e.g. [5]), code for controlled shut-down of the gRPC server [6], or code for authentication (see e.g. [7]). The code in this article is intentionally kept simple so that we can concentrate on the core functions of gRPC.

gRPC clientThe client for our gRPC service was generated by pro-toc. We can use it without having to implement any in-terfaces. Listing 4 calls the gRPC service defined above. When calling the gRPC method SayHello, the context.Context [8] must be considered. It has already been in-cluded in Listing 2. This type allows us to influence the behavior of the gRPC connection and to specify things like timeouts or to cancel the gRPC call.

StreamingWhile the examples shown so far demonstrate the basic principle of gRPC in Go quite well, they only scratch the surface of what gRPC can do. In this article I would like to discuss an important, advanced feature of gRPC: Streaming. gRPC can do three variations of this:

• Server-side streaming: The server does not respond with a single response, but with a stream of respons-es.

• Client-side streaming: The client does not send a single request, but a stream of requests. The server responds with a response.

• Bidirectional streaming: The client initiates com-munication by calling the gRPC function and then sends a stream of requests. The server responds with

Listing 5: Proto-File with streaming

syntax = "proto3";

option go_package = "greet";

package demo;

[...] // Greeter Service shown in previous code samples

// Note streaming resultservice MathGuru { rpc GetFibonacci (FromTo) returns (stream NumericResult);}

message FromTo { int32 from = 1; int32 to = 2;}

message NumericResult { int32 result = 1;}

Listing 4: gRPC Clientpackage main

import ( "context" "log"

"github.com/Samples/BeyondREST/GoGrpcClient/greet" "google.golang.org/grpc")

func main() { conn, err := grpc.Dial("localhost:8080", grpc.WithInsecure()) if err != nil { panic(err) } defer conn.Close()

client := greet.NewGreeterClient(conn) response, err := client.SayHello(context.Background(), &greet.HelloRequest{Name: "FooBar"}) if err != nil { panic(err) }

log.Println(response.Message)}

The 3 Axes of Scaling APIsMilecia McGregor (Flipped Coding)

It’s great when your app starts to get more users. That means your product is a success and people like it. Sometimes this rush of users leaves companies struggling to keep their applications up

because they weren’t built to handle this kind of us-age. Scale wasn’t a consideration when the app was initially built. This happens all the time and API de-velopers need to know how to manage scaling up an application. Whether you’re starting with a monolith or cloud-based application, there are 3 major tech-niques you need to know how to use to scale an API. These 3 techniques will be able to take your app from handling a few thousand users to millions. By the time you leave this talk, you will be know everything you need to scale any API and you’ll have the 3 core techniques to make it happen.

Page 15: The API revolution continues! API Whitepaper Selection...Looking to 2020 and beyond, the proportion of data produced and consumed in realtime is growing exponentially . 4 WITPAPR API

15www.apiconference.net

WHITEPAPER API Development

a stream of responses. The two streams are independ-ent of each other and can work interlocked. This means that the client does not have to have sent all the requests before the server begins to send respons-es.

Let's have a look at an example. In this example, we re-turn the numbers of a Fibonacci sequence from a gRPC server. To make things more interesting, let's pretend that it takes longer to calculate the individual numbers. This allows us to use streaming. After all, the client can already process the first numbers, even though the server is still in the process of calculating the next results. In this example, the server is streaming responses to the client, so it is server-side streaming.

For space reasons, the rather simple source code for the iterator that calculates the Fibonacci sequence is not printed here. If you are interested, you can read it in-cluding unit tests on GitHub [9].

Listing 5 shows the extended proto3 file with the streaming service. Note in particular the keyword stream when defining the method GetFibonacci. It indi-cates that the method does not return a single response, but a stream of responses.

Listing 6 shows the implementation of the streaming method. As you can see, even with streaming, the code

is not complicated. The degree of complexity of the code is not increased by gRPC or Go, it results from the ap-plication logic to be provided by the gRPC API.

To demonstrate the platform independence of gRPC, in Listing 7 I show what a client for our streaming API in C# looks like. The gRPC client MathGuru.MathGu-ruClient was generated by Visual Studio 2019 from the same proto3 file we used for Go before (Listing 5). In the C# implementation it is noticeable that to consume the streaming API the new C# 8 function await foreach

Listing 7: Consuming the Streaming API in C#

using Grpc.Core;using Grpc.Net.Client;using GrpcDemo;using System;using System.Threading.Tasks;

namespace GrpcClient{ class Program { static async Task Main() { AppContext.SetSwitch( "System.Net.Http.SocketsHttpHandler.Http2UnencryptedSupport", true);

// Create channel. // Represents long-lived connection to gRPC service. // The port number(5001) must match the port of the gRPC server. // Tip: In ASP.NET Core apps, use gRPC client factory var channel = GrpcChannel.ForAddress("http://localhost:8080");

var mathClient = new MathGuru.MathGuruClient(channel); await ServerStreaming(mathClient); }

private static async Task ServerStreaming( MathGuru.MathGuruClient mathClient) { // Note usage of new using statement (C# 8) using var fibResult = mathClient.GetFibonacci( new FromTo() { From = 10, To = 50 });

// Note usage of new streaming enumerable await foreach (var fib in fibResult.ResponseStream.ReadAllAsync()) { Console.WriteLine(fib); } } }}

Listing 6: Implementation with streaming

func (mgs mathGuruService) GetFibonacci(fromTo *greet.FromTo,stream greet.MathGuru_GetFibonacciServer) error { if fromTo.From > fromTo.To { return status.Error(codes.InvalidArgument, "FromTo.From must be <= FromTo.To") }

fib := mathalgorithms.NewFibonacciIterator() for fib.Next() { if fib.Value() < fromTo.From { continue }

if fib.Value() <= fromTo.To { stream.Send(&greet.NumericResult{ Result: int32(fib.Value()), }) time.Sleep(250 * time.Millisecond) continue }

break }

return nil}

Page 16: The API revolution continues! API Whitepaper Selection...Looking to 2020 and beyond, the proportion of data produced and consumed in realtime is growing exponentially . 4 WITPAPR API

16www.apiconference.net

WHITEPAPER API Development

can be used. This makes the code very easy to read and write.

If you're interested in what bidirectional streaming looks like, you can take a look at a more comprehen-sive version of the Fibonacci example discussed above on GitHub [10]. There you can find a variant where the client sends requests for Fibonacci subsequences to the server via stream and the server streams the results back.

Standard Interface DefinitionsIf you try to implement a real API with gRPC, you will notice that you have to solve many problems that can be implemented with ProtoBuf, but where you would have to invent basic interface definitions yourself. Ex-amples in this area are error codes and error messages, date values (ProtoBuf does not have a datatype built in for date values), monetary amounts with currencies, geocoordinates etc. Fortunately, Google provides the proto3 files on GitHub, which the company uses for its own cloud APIs [11]. Since they are Open Source, they can be imported into your own projects. This way you save the time it would take to reinvent the wheel. In ad-dition, consumers of the gRPC API find it easier to find their way around because they find structures they may already know from the Google APIs.

ConclusionGo is a good programming language for implementing microservices. The code is consistent, stable over a long time and easy to maintain. The ecosystem of open source libraries for cloud native microservices is excellent, as Go is at home in this area. The resulting executables are performant, compact, have no dependencies and are therefore particularly convenient to use in container and serverless environments.

Microservices need a platform-independent inter-face to the outside world so that users are not forced into a certain technological direction. Here gRPC is a good choice. The code generators that generate code from proto3 files are available for practically all com-

FinTech + API First = GraphQLSebastian Galonska (Kontist GmbH)

Three years ago we have started to build a bank. We made some mistakes, one of them was our API. This talk describes our journey from an impro-vised anarchistic REST API to a well-

documented, typed GraphQL interface. I will touch on technical and cultural aspects which you should consider if you plan to step into our footsteps.

mon programming languages and are of high quality. As hopefully you have seen from the examples shown above, developing gRPC code with Go is easy to learn. At runtime, gRPC offers comparatively good perfor-mance compared to RESTful Web APIs.

So there are many reasons to consider the combination of Go and gRPC especially in a cloud backend. gRPC is not a "REST killer" though. Outwardly to the web or mobile frontends one could use gRPC Web [12], but HTTP and JSON based web APIs (e.g. REST, GraphQL, JSON:API, OData) are often the better options in this area. In my opinion, today's software development world no longer consists of monocultures in which one solution approach, one communication protocol, one programming language or one framework must be the solution for everything. The right combination of tech-nologies and tools is what counts. In my view, Go and gRPC definitely have a place here.

Rainer Stropek is an IT entrepreneur, software devel-oper, trainer, author and lecturer in the Microsoft en-vironment. He is MVP for Microsoft Azure since 2010 and develops with his team time recording for service professional

Links & Literatur

[1] https://developers.google.com/protocol-buffers/docs/proto3

[2] https://grpc.io/docs/tutorials/

[3] https://blog.golang.org/survey2018-results

[4] https://github.com/protocolbuffers/protobuf/releases

[5] https://github.com/grpc/grpc-go/blob/master/examples/features/encryption/TLS/server/main.go

[6] https://godoc.org/google.golang.org/grpc#Server.GracefulStop

[7] https://github.com/grpc/grpc-go/blob/master/examples/features/authentication/server/main.go

[8] https://godoc.org/context

[9] http://bit.ly/fibonacci-go

[10] https://github.com/rstropek/Samples/tree/master/BeyondREST/BeyondREST/GoGrpcServer

[11] https://github.com/googleapis/googleapis/tree/master/google

[12] https://github.com/grpc/grpc-web

Page 17: The API revolution continues! API Whitepaper Selection...Looking to 2020 and beyond, the proportion of data produced and consumed in realtime is growing exponentially . 4 WITPAPR API

17www.apiconference.net

WHITEPAPER API Platforms & Software-as-a-Service

by Petteri Kivimäki

X-Road is used nationwide in the Estonian public ad-ministration and in the Suomi.fi Data Exchange Layer service. In addition, Iceland is currently implementing its own X-Road data exchange layer environment called Straumurinn. X-Road is released under the MIT license and is available free of charge for any individual or or-ganization. Nordic Institute for Interoperability Solu-tions (NIIS) is responsible for the development of the X-Road core and providing support and insights to the X-Road community (Image 1).

X-Road implements a set of common features to sup-port and facilitate data exchange. X-Road provides the following features out of the box:

• address management• message routing• access rights management• organization level authentication• machine level authentication• transportation layer encryption• time-stamping

• digital signature of messages• logging• error handling.

Trusted NetworkThe identity of each organization and technical entry point (Security Server) is verified using certificates that are issued by a trusted Certification Authority (CA) when an organization joins an X-Road ecosystem. The identities are maintained centrally, but all the data is exchanged directly between a consumer and provider. Message routing is based on organization and service level identifiers that are mapped to physical network locations of the services by X-Road. All the evidence regarding the data exchange is stored locally by the data exchange parties, and no third parties have access to the data. Time-stamping and digital signature together guarantee non-repudiation of the data sent via X-Road (Image 2).

An X-Road ecosystem is a community of organiza-tions using the same instance of the X-Road software for producing and consuming services. The owner of the ecosystem, the governing authority, controls who’s

A centrally managed distributed Data Exchange Layer between information systems

X-Road – The Free and Open Source Data Exchange Layer X-Road is an open source data exchange layer solution that enables organiza-tions to exchange information over the Internet. X-Road is a centrally managed distributed data exchange layer between information systems that provides a standardized and secure way to produce and consume services. X-Road ensures confidentiality, integrity and interoperability between data exchange parties.

Page 18: The API revolution continues! API Whitepaper Selection...Looking to 2020 and beyond, the proportion of data produced and consumed in realtime is growing exponentially . 4 WITPAPR API

18www.apiconference.net

WHITEPAPER API Platforms & Software-as-a-Service

allowed to join the community, and the owner defines regulations and practices that the ecosystem must fol-low. The ecosystem may be nationwide, like in Esto-nia and Finland, or it may be limited to organizations matching certain criteria, e.g. clients of a commercial service provider. Technically, the X-Road software does not set any limitations to the size of the ecosystem or to the member organizations.

Two X-Road ecosystems can be joined together, fed-erated. Federation is a one to one relationship between two ecosystems. Members of the federated ecosystems can publish and consume services with each other as if they were members of the same ecosystem. For exam-ple, Finland's and Estonia's data exchange layers are connected to one another which enables cross-border data exchange between the countries. Federation pro-vides members of the federated ecosystems a single ac-

cess point to both local and cross-border data sources. However, it must be noted that federation is not only about technology as both legal and administrative agreements are needed between the X-Road operators and member organizations that exchange data (Image 3).

Producing and Consuming ServicesThe use of X-Road requires that all the message ex-change parties are members of an X-Road ecosystem and they have access to an X-Road entry point, Secu-rity Server, which is required for both producing and consuming services. The Security Server mediates ser-vice calls and service responses between information systems, and it encapsulates the security aspects of the X-Road infrastructure: managing keys for signing and authentication, sending messages over secure channel,

Image 1: X-Road components, roles and responsibilities.

Image 2: X-Road architecture

Page 19: The API revolution continues! API Whitepaper Selection...Looking to 2020 and beyond, the proportion of data produced and consumed in realtime is growing exponentially . 4 WITPAPR API

19www.apiconference.net

WHITEPAPER API Platforms & Software-as-a-Service

creating the proof value for messages with digital signa-tures, time-stamping and logging.

Image 3: X-Road federation – connection between two X-Road ecosystems.

X-Road logs contain all the messages processed by the Security Server. Each message is time-stamped and signed which makes it possible to verify the message content afterwards. By default, the logs are stored lo-cally by the Security Server — external parties do not have access to them.

Both data using services and data sources are iden-tified using X-Road’s globally unique identifiers. The identifiers contain information about the X-Road eco-system, member organization and information system that is consuming or producing data via X-Road. The identifiers are used internally by X-Road for routing messages between data using services and data sources. A data using service does not need to know the net-work address of a data source — X-Road automatically maps the service identifier to the correct network ad-dress.

Own Your DataBuilt-in features provided by X-Road can also be used for managing access rights to data sources. Access rights management is based on the X-Road service identifiers. The key idea of X-Road is that each service provider owns its data and is responsible for managing access rights of its services. In other words, publishing a service to X-Road does not mean that the service is automati-cally accessible to all X-Road member organizations. Usually, access rights are granted on information sys-tem level — a service provider grants a specific informa-tion system access to a service. This approach provides a tight control, but it may also generate much administra-

Mistakes to avoid while building API ProductsRahul Dighe (PayPal)

I have had the unique opportunity over the past few years to work on multiple API initiatives and observe and learn from several seasoned API practition-ers. In the past we have heard leading

practitioners talk about:• How we should follow an API First approach• Think of API-as-a-Product• Developers as your customers• Design First vs. Code First• API GovernanceHowever, I continue to notice time and again that the overall Developer Experience when it comes to using APIs continue to have gaps that are unaddressed. Even after religiously following all of the best prac-tices, one still ends up with a poor API design and strategy. This talk aims to showcase learnings that I have gathered either building APIs or witnessing oth-ers and what API practitioners need to do take their APIs to the next level.

Page 20: The API revolution continues! API Whitepaper Selection...Looking to 2020 and beyond, the proportion of data produced and consumed in realtime is growing exponentially . 4 WITPAPR API

20www.apiconference.net

WHITEPAPER API Platforms & Software-as-a-Service

tive work if the number of service consumers is changing frequently.

X-Road is an open source technology, and anyone has access to it for free of charge. An X-Road ecosystem is

Image 4: X-Road compared to point-to-point integrations

managed by a governing authority that controls who is allowed to join the ecosystem. In Estonia and Finland the ecosystems are open for all kind of organizations (public, private, non-profit etc.) and joining them is free. Joining an X-Road ecosystem does not require an or-ganization specific Security Server as a Security Server can be shared between multiple organizations or provid-ed as a service by a third party, e.g. a commercial service provider. The process that is required to join an ecosys-tem is more time consuming compared to implementing a direct point-to-point integration, but it is necessary for creating trust between the members of the ecosystem. In addition, the process is required only when a new or-ganization joins the ecosystem or a new Security Server is registered (Image 4).

X-Road DevelopmentNIIS is responsible for developing X-Road core tech-nology, and welcomes everyone to submit source code contributions, new ideas and enhancement requests re-garding X-Road. The backlog is public — anyone can access it, leave comments and submit enhancement requests through the X-Road Service Desk portal. Ac-cessing the backlog and service desk requires creating an account which can be done in few seconds using the signup form.

APIs and Cross-Border Data Ex-change in E-Government ContextPetteri Kivimäki (Nordic Institute for Interoperability Solutions (NIIS))

X-Road is an open source, distributed data exchange layer solution that ena-bles organizations to exchange informa-tion over the Internet. X-Road ensures confidentiality, integrity and interoper-

ability between data exchange parties. X-Road is re-leased under the MIT open source license. X-Road is used nationwide in the Estonian data exchange layer X-tee and in the Suomi.fi Data Exchange Layer ser-vice in Finland. However, X-Road is not just for public sector organisations – all kinds of organisations re-gardless of size or type can use it. Two X-Road eco-systems can be joined together, federated. Federation enables easy and secure cross-border data exchange between countries using X-Road. Nordic Institute for Interoperability Solutions (NIIS) is an association founded jointly by Finland and Estonia. Its mission is to develop e-governance solutions, kicking off with X-Road. X-Road provides a solution to API manage-ment and its approach is quite different from other solutions on the market. In addition, the development model – two countries developing an open source solution that‘s used for implementing nationwide services in multiple countries – is quite unique even in global scale. The aim of the presentation is to discuss X-Road as a platform, its cross-border data exchange capabilities and its unique development model.

Petteri Kivimäki is the CTO of the Nordic Institute for Interoperability Solutions (NIIS). The NIIS is an asso-ciation founded jointly by Finland and Estonia which mission is to develop e-governance solutions, kicking off with the X-Road technology.

Page 21: The API revolution continues! API Whitepaper Selection...Looking to 2020 and beyond, the proportion of data produced and consumed in realtime is growing exponentially . 4 WITPAPR API

21www.apiconference.net

WHITEPAPER API Platforms & Software-as-a-Service

by Khallai Taylor

Financial Institutions are increasingly vigilant of per-ceived external security risk yet frequently disregard how there APIs are designed for security. With the con-stant change of regulation and the heavy fines involved with non compliance. Organizations not only need to ensure that they have appropriate licensure but that there APIs at least meet secure programming criteria that are in line with global cybersecurity standards for secure APIs set forth by ISO.

Globally it is estimated that regulatory spending ex-ceeds $ 80 billion per year and will surge to $120 billion within the next 5 years. This is due to the increasing demand to secure customer information, data, and the flow of money across international borders; Thereby, necessitating the need for enhanced and ongoing cyber-security protection. Regulatory bodies in Europe recog-nized this need and the increasing capabilities of bad actors to penetrate and manipulate data by using simple measures to entice employees to make mistakes and/or circumvent company policies resulting in data breaches. Not to mention constant cyber attacks on financial or-ganizations and banking accounts of everyday people.

The evolution of Regulation-as-a-ServiceThe european union under the direction of the european banking authority found it necessary to implement regu-lation that would identify and track the flow of money within Europe, while enabling new forms of payment services to be offered. Resulting in the birth of the Pay-

ment Service Directive 2 (PSD2) in 2015. The purpose of the directive not only monitors and secures the flow of payments through europe but it requires banks to open up there data to fintechs who provide innovative solu-tions using this data and collaborating with banks. In this article we will explore the evolution of Regulation-as-a-Service using PSD2 as the example that can be applied to any financial regulatory directive that requires specialized licensing and/or oversight handed down by the regions financial regulatory body. Including international regula-tory laws that apply to privacy like GDPR, investment banking such as MiFiD and Anti-Money Laundering.

Organizations participating in the business of pay-ment or remittances are subject to the consequences of failing to meet financial regulatory requirements. In Germany BaFin ( Bundesanstalt für Finanzdienstleis-tungsaufsicht) the countries’ federal financial supervi-sory authority regulates Banks, financial institutions and fintechs. As such they work in alignment with the European Banking Authority, selective financial servic-es working groups and lobbyist to create regulations, provide standards and enforce the compliance of such regulation. With this comes a wide set of problems and ambiguity. Primary example being the implementation deadline of PSD2 that was initially set to take affect Sep. 14, 2019 within the EU. Due to the lack of a universal set of standards for PSD2 compliant APIs most banks were unable to meet the looming deadline and requirements; and in turn, blocked fintechs from accessing needed data to sell their products to bank customers. In an effort to ensure that regulatory bodies like BaFin were aware of

Rethinking APIs

Regulation as a Service The development of digital fintech, rapid technological progress and the constant flow of money across borders have made it essential for companies to offer their products and services internationally while complying with the strict data protection laws of the EU. In this article Khallai Taylor will explore the evolution of Regulation-as-a-Service using PSD2 as the example that can be applied to any financial regulatory directive that requires specialized licensing and/or oversight handed down by the regions financial regulatory body.

Page 22: The API revolution continues! API Whitepaper Selection...Looking to 2020 and beyond, the proportion of data produced and consumed in realtime is growing exponentially . 4 WITPAPR API

22www.apiconference.net

WHITEPAPER API Platforms & Software-as-a-Service

the conflict between the big banks and fintechs lobbyist across the EU advocated for an extension to the imple-mentation deadline, clarity on suggested requirements to ensure that payment APi’s adhere to PSD2 strong Customer Authentication criteria.

In Germany some financial institutions are also third party providers (fintechs) that have special licensure to offer payment initiation services and account infor-mation services issued by BaFin. This licensure can be passported to other companies allowing them to meet the regulatory requirements to provide payment ser-vices and handle customer account information. This coverage is extended to companies under the auspice of Regulation-as-a-Service. Enabling companies to remain innovative, agile and the ability to offer new products and services to customers while renting the required li-cense to meet compliance standards. Saving the organi-zation time and money while the fintech scales.

In order to combat unintended cyber attacks regula-tors have worked to ensure security measures are inter-twined into financial regulatory components that adhere to international cybersecurity standards. Such as requir-ing strong customer authentication, the use of tokens and use of electronic certificates that verify domains and identities amongst other requirements set forth in the PSD2 directive. By understanding the value of data pri-vacy many organizations are starting to realize that the APIs they create must be done with a secure API mind-set. Ensuring that technologies used for Open Banking and PSD2 are compliant thus safeguarding services of-fered to EU citizens assuring not only product innova-tion but secure APIs.

As mentioned earlier BaFin in partnership with the European Banking authority extended the PSD2 dead-line in Germany while allowing for fintechs to continue to operate as normal while still remaining complaint. Unfortunately, some banks across the EU have found ways to circumvent PSD2’s primary initiative to share data with fintechs and have blocked fintechs from using Bank data. This is an ongoing fight in which the Eu-ropean Banking Authority and regulatory bodies are working to resolve within the EU.

Culminating in the rethinking of APIs.The evolution of fintechs, rapid technical advancement and the constant flow of money across borders has made it essential for organizations to provide their products and services internationally while adhering to the strict privacy and data protection laws set forth by the EU. Yet, many fintechs do not have the money nor time to apply for specialized licensure from regulators. Thereby, these organizations are seeking alternative means to op-erate while remaining compliant. Regulation-as-a-Ser-vice allows fintechs the ability to innovate rapidly and partner with banks to offer new products to the market. I predict that over the next three years Regulation -as-a-Service will become common place within the industry allowing companies to rapidly enter new markets. Due to Regulation-as-a-Service providers who incorporate risk analysis, AI based compliance training and API standardization and testing. Facilitating the expansion of services and offerings in the financial sector. In turn organizations will become diligent in securing APIs to safeguard data and privacy of customers. Culminating in the rethinking of APIs.

API Platform: a full-stack framework to build and consume hypermedia and GraphQL APIs in minutesKévin Dunglas (Les-Tilleuls.coop)

API Platform ( api-platform.com ) is a framework for building API-driven projects that is becoming quite popular. The server component enables users to create modern, extensible and secure

web APIs very easily: design the public data model you want to expose as a set of classes, and that’s all – you get a fully-featured API! Out of the box, the API will support most hypermedia formats (JSON-LD, Hydra, JSON:API, HAL…), will be documented using OpenAPI (v2 and v3) and can expose a GraphQL endpoint. It also comes with a bunch of built-in filters, pagination, authentication, authorization, HTTP/2 Server Push of the relations, nested docu-ments, deprecation and much more! It is able to use all popular DBMS, MongoDB, ElasticSearch, or your custom system as a persistence layer.And because API Platform is built on top of the popular and battle-tested Symfony web framework, you can extend and override everything and benefit from thousands of high-quality plugins! But there’s more: API Platform also provides awesome client-side components (writ-ten in JavaScript) that work with the server compo-nent, or with any API supporting JSON-LD+Hydra! These components include a smart „admin“ builder (built on top of React Admin) and client generators supporting React, Vue, React Native, Next, Quasar and Vuetify! Last but not least, API Platform comes with the Docker images and Kubernetes chart you’ll need to deploy in production, in seconds.

Khallai Taylor (RegTheory) is a Fintech Product Evan-gelist in Berlin and a consultant of secure API’s and Blockchain in Africa, America, and Europe. Khallai works directly with engineering, product, compliance and sales teams ensuring streamlined communication and

product development between the teams. Khallai has 17+ years as a technologist, academic and an advocate for women in tech.

Page 23: The API revolution continues! API Whitepaper Selection...Looking to 2020 and beyond, the proportion of data produced and consumed in realtime is growing exponentially . 4 WITPAPR API

23www.apiconference.net

WHITEPAPER API Management

by Thomas Eiling

The software basis for Shopware 6 is completely new and follows a rigorous API-first approach. The advan-tage of this compared to the previous version is that all functionalities of the new administration can also be managed via the API. By strictly following the API-first approach, three different API endpoints have emerged: Sync, SalesChannel and Admin. The Sync endpoint is designed for importing and exporting large amounts of data, whereby 300 products per second can currently be imported. Lengthy processes, such as creating thumb-nails, are processed via a new messaging queue in the background.

Asynchronous processing accelerates the import pro-cess significantly, of course, and allows us to achieve a twenty-fold increase compared to Shopware 5. The Ad-min API is systematically used by the new administra-tion, which is based on Vue.js. You can now use all the settings and functionalities that the admin offers via the API as well. This enables you to create custom interfaces for administrative tasks, such as a view for pending or-ders that need to be shipped. It doesn’t matter which technology is ultimately used for this purpose. Last but not least, there’s the SalesChannel API. This endpoint is responsible for all requirements for connecting your own shop frontend, POS or any other sales channels, enabling you to realise completely new sales concepts on the Internet. We have published various experiments on GitHub to show how simple it is to use the new API. These consist of a one-page shop, a shop snippet and an Alexa skill. The one-page shop is an experiment that shows how easy it is to build your own storefront for

Shopware 6. The single page includes a functioning checkout with payment processing.

The shop snippet is of interest to shop operators who blog a lot, work with influencers, etc. It allows you to in-tegrate products from a Shopware 6 shop in a blog post with a simple snippet, for example. There’s also a link to the shop so you can buy there directly, of course. The third experiment, the Alexa skill, connects the Shopware

Putting the API on the Acid Test

Shopware 6 and the new API-First Approach In this article on the Shopware 6 e-commerce platform, we look at the techni-cal changes in the new version and the new API First approach.

Lessons Learned from Implementing API Management in the Real WorldEldert Grootenboer (Motion10)

Azure API Management is the gate-keeper to our data and processes, and as such it’s really important that we set it up securely. But how do we know which features of API Management

will allow us to reach our goal? And how should we implement them in such a way that we stay secure, while still providing the best experience for our devel-opers? This session will show you how to set up an API Management environment using real-life use cases. You will learn how to expose and protect your services, which policies help make our life easier, and how to handle your application lifecycle manage-ment.

Page 24: The API revolution continues! API Whitepaper Selection...Looking to 2020 and beyond, the proportion of data produced and consumed in realtime is growing exponentially . 4 WITPAPR API

24www.apiconference.net

WHITEPAPER API Management

6 shop to Amazon’s Alexa. Once enabled, you can use Alexa voice commands to search for products, order them and have the product descriptions read aloud. Furthermore, the status of an order can be tracked and new products suggested. These experiments are a good introduction to using and understanding the SalesChan-nel API.

Now let’s have a look at the Admin API. In the follow-ing sections, we will set up a demo environment; create, update, delete a category; and see how to search for spe-cific categories.

Setting up a Demo EnvironmentWe use the Docker environment running on a Linux cli-ent. First, we clone the GitHub development template using git clone [email protected]:shopware/development.git. Now we have the development template for Shop-ware 6 in the development directory. We can now go into the directory using cd development and clone the actual Shopware platform repository into the standard directory using git clone [email protected]:shopware/plat-form.git. Caution: Please do not specify any other direc-tory when cloning, as this is important for autoloading. We now have all the source code we need to get start-ed on the computer. To build and start the necessary Docker containers, we enter ./psh.phar docker:start. We then connect to the application container via ./psh.phar docker:ssh and start the installation with ./psh.phar in-stall.

This may take quite a while the first time, because the initial execution requires several caches to be cre-ated. To check whether the installation was successful, you can simply open your preferred browser and access http://localhost:8000. Mac users can also perform the set-up locally. You can find an example virtual host con-figuration in the Installation Guide under “Setting up your webserver”. All you have to do is execute bin/setup and you will be guided through an interactive installa-

Listing 1$client = new GuzzleHttp\Client([ 'base_uri' => 'http://localhost:8000', 'timeout' => 2.0, 'headers' => ['Content-Type' => 'application/json'],]);

$token = $client->post( '/api/oauth/token', [ 'body' => json_encode([ 'client_id' => $clientId, 'client_secret' => $clientSecret, 'grant_type' => 'client_credentials', ]), ]);

Listing 2$categories = $client->get( '/api/v1/category/', [ 'headers' => [ 'Authorization' => $token['token_type'] . ' ' . $token['access_token'], 'Accept' => 'application/json', ], ]);

Listing 3$result = $client->post( '/api/v1/category?_response=true', [ 'headers' => [...], 'body' => json_encode(['name' => 'new Category']), ]);$result = json_decode((string) $result->getBody(), true);$categoryIdentifier = $result['data']['_uniqueIdentifier'];

OWASP Top 10 for APIInon Shkedy (OWASP)

Application security threats are evolv-ing.Fortune 100 companies with large budgets and talented security engineers get breached through their APIs on a weekly basis. Uber full ac-

count takeover, the famous Facebook breach, & the Verizon customer bills leakage are just a few recent examples of API based breaches. Traditional vulner-abilities such as SQL injection, CSRF & XSS are less prevalent thanks to modern technologies and security education. Attackers leverage the predictable and oversharing nature of REST APIs to exploit new types of vulnerabilities that are focused on business logic abuse and authorization. OWASP (the gold standard in AppSec) has acknowledged this shift in threats and has announced the OWASP API Security Project. The project addresses modern API threats and pro-vides mitigation techniques.Come learn from the leader of the project about:• OWASP Top 10 For APIs and how they are differ-

ent from traditional top 10 lists.• Examples for complex API exploits, which involve

many steps• How to exploit an API as a pentester, and how to

protect it as a developer.

Page 25: The API revolution continues! API Whitepaper Selection...Looking to 2020 and beyond, the proportion of data produced and consumed in realtime is growing exponentially . 4 WITPAPR API

25www.apiconference.net

WHITEPAPER API Management

tion process. If something doesn’t work during installa-tion, check if there’s a file called .psh.yaml.override. If not, start the set-up script again with ./psh.phar install.

Using the Admin APIIn the following examples, I use Guzzle 6. Other than that, I have no libraries in use. You can create the re-quired client ID and client secret in the Administration by going to SETTINGS | SYSTEM | INTEGRATIONS and clicking ADD INTEGRATION. We can now use the client ID and client secret to generate a bearer to-ken, which we will need for further API requests (Listing 1). Apart from OAuth, all other routes are versioned to ensure backward compatibility with future changes to the API. You can tell by the v1 in the route name. At the moment, only Version 1 exists. We now have all the necessary information in $token to authorise ourselves for the next requests. If we want to output all existing categories, we can do so as in Listing 2. Further exam-ples on how to define filters, searches, sorting or limits for this query can be found in the online documentation for Shopware 6.

So how do we set up a new category? Ultimately, much of what we do is identical to the request in which we retrieve all existing categories. The headers are iden-tical and the endpoint in question is only extended by ?_response=true to receive a return from the API including the unique identifier generated. Furthermore, a POST request is issued instead of a GET request. In order to create a category named new category, we need the fol-lowing source code snippet from Listing 3. We evaluate the return of the API in order to store the unique identi-fier of the category for further requests. To amend the newly created category, we don’t have to change much in the previous snippet. We replace ?_response=true with the unique identifier of the category and change the

Listing 5$client->delete( '/api/v1/category/' . $categoryIdentifier, [ 'headers' => [...], ]);

Listing 4$client->patch( '/api/v1/category/' . $categoryIdentifier, [ 'headers' => [...], 'body' => json_encode(['name' => 'new Category updated']), ]);

Documentation is the most important Part of Your API Development & ManagementMilecia McGregor (Flipped Coding)

One of the biggest complaints from developers that work with APIs is the lack of good documentation. The most common reason for this is that docu-mentation is an afterthought for many

API developers. API developers have to manage their own deadlines, keep up with any architectural updates, and make sure their code works as expected on top of many other things. So how is documenta-tion more important than any of these things and when do you have the time to write it? In this talk, attendees will learn how to implement documenta-tion in every aspect of their API development and management. We’ll cover the reasons why you need documentation in the development phase and how it translate to the management phase. Then we’ll go over several strategies to make documentation easier for API developers to write and better for other de-velopers to use. This will highlight concerns like how documentation helps with scaling, how it helps keep a cleaner codebase, and how it helps make an API more maintainable over time

name to new category updated. The whole thing is then sent as a PATCH request. The snippet now in use can be found in Listing 4.

Last but not least, we want to delete the newly created category from the system. You can find the correspond-ing source code snippet in Listing 5.

ConclusionWith its new API structure, Shopware offers shop opera-tors, developers and agencies many new ways to estab-lish their e-commerce presence. It has never been easier to connect third-party systems to Shopware, let alone build your own storefronts in order to enable an even greater customer-specific approach. Despite all the new functions, there is no negative impact on performance. The current benchmarks for the API are already rather promising and enable ERP integrations to implement new synchronisation concepts.

Thomas Eiling works in the enterprise team of shop-ware AG and has been developing with Shopware since the first version. In addition to the actual program-ming in the team, he is also happy to take on DevOps tasks.

Page 26: The API revolution continues! API Whitepaper Selection...Looking to 2020 and beyond, the proportion of data produced and consumed in realtime is growing exponentially . 4 WITPAPR API

26www.apiconference.net

WHITEPAPER API Management

by Dr. Michael Sperber

Doing so leads to thinking about tests not in terms of individual examples, but general properties, and from there to an approach called property-based testing or QuickCheck. QuickCheck generates tests from proper-ties. It has its origins in functional programming and has been a crucial tool in eliminating bugs from many com-plex software projects. But QuickCheck does more than help find bugs: It encourages thinking about our API in terms of properties, and this style of thinking often leads to interesting domain insights and architectural improvements.

APIs and SpecificationsConsider the following API for a simple "smart" shop-ping cart, written in F# [1]:

type Interface = abstract member Add: int -> Item -> unit abstract member Total: unit -> Price

The first method, Add, takes an integer representing a count, and an item to be added to the cart, and returns nothing. (unit is used in F# similarly to void in other languages.) The second method returns a total price for all items in the cart. (That's why it's "smart" - it knows the prices of the items in it.)

We add a convenience module with functions for call-ing the methods:

module Interface = let add (cart: Interface) (count: int) (item: Item) = cart.Add count item let total (cart: Interface): Price = cart.Total ()

We also add a factory interface, creating a shopping-cart implementation from a catalog assigning a price to each item:

type Catalog = Map<Item, Price> type Factory = abstract member make: Catalog -> Interface

Now, the types give us some information about what kinds of values are input and output to the various op-erations, but not what those values actually are. How could we obtain that information? We could look at the implementation, but that might be messy and complex - databases and microservices might be involved. We could look at unit tests, which common development methodologies produce as a matter of course. These might look as follows, assuming there is a factory for shopping carts available as factory:

let test = let cart1 = factory.make Map.empty assertEquals (Interface.total cart1) (Price 0) let cart2 = factory.make (Map.ofList [Item "Milk", Price 100; Item "Egg", Price 20]) assertEquals (Interface.total cart2) (Price 0)

Taking executable specifications seriously

APIs have Properties – find and test them! Most approaches to testing test individual examples. This is often not enough to tease out rare but realistic edge cases. As a result, it takes a large number of individual tests to really be confident that our API is ready for deployment: This is a lot of work. What if we could generate test cases automatically?

Page 27: The API revolution continues! API Whitepaper Selection...Looking to 2020 and beyond, the proportion of data produced and consumed in realtime is growing exponentially . 4 WITPAPR API

27www.apiconference.net

WHITEPAPER API Management

(We assume a function assertEquals from some test frameworks, and that items and prices are constructed with the Item and Price constructors.)

These are trivial tests, but they do make a point: The first one says that the total a freshly created shopping cart is 0 - provided its catalog is empty. The second one says the same thing for a catalog containing only milk and eggs. The TDD community calls this an "execut-able specification" (or part of one), but there is still a lot missing. Looking at it, you might say "The total is 0 for all shopping carts, for all possible catalogs." This is in fact a property of the shopping-cart factory, and, even though may seem trivial, worth stating explicitly. This could look as follows:

let total0Correct = Prop.forAll Arb.catalog (fun catalog -> let cart = factory.make catalog Interface.total cart .=. Price 0)

The Prop and Arb modules come from the FsCheck package. [2] Prop.forAll returns a property (of type Property) that should hold for all values from a certain set. Arb.catalog means "arbitrary catalog", and Prop.forAll accepts a function for naming that catalog. The body of that function is a boolean expression that first creates a cart associated with the catalog, and then checks whether the total reported by the cart is indeed 0. As opposed to the two "example unit tests" above, this states that the total is 0 for all catalogs.

FsCheck provides a function called quick that checks whether the property holds. In F# Interative, it prints output as follows:

quick total0Correct;;  Ok, passed 100 tests.

FsCheck has just generated 100 tests from the descrip-tion of the property. We can instrument total0Correct to print out the catalogs of those tests:

 map [(Item "HF", Price 1); (Item "dK", Price 2)]  map [(Item "Bx", Price 1); (Item "QaY", Price 1); (Item "ca", Price 3)]  map [(Item "Qhwu", Price 2); (Item "t", Price 3)]  map [(Item "HS", Price 1); (Item "h", Price 3); (Item "uj", Price 1)]  map [(Item "IXVgqo", Price 1); (Item "Rgr", Price 5); (Item "Zgnh", Price 3)]  map [(Item "ZXhwG", Price 4); (Item "nwFUSd", Price 2)] ...

This means that FsCheck can generate many tests from a declarative description of a property. FsCheck's under-lying idea comes from one of the most influential works in functional programming, the paper QuickCheck. [3] The QuickCheck technique is so pervasively useful that QuickCheck implementations exist for most program-ming languages, even non-functional ones.

The total0Correct property shows two advantages over example-based unit tests like the two above: It separates the general property from the specific exam-ples, thus being a more informative part of an execut-able specification. Moreover, generating many test cases from a single property is much more likely to find bugs in the code.

Of course, total0Correct is quite simplistic. In Listing 1 is a more involved property.

Again, this is a property that holds for all catalogs. For each catalog, the property extracts the available items from the catalog as items. It then goes on to say that the property must also hold for all Arb.list (Arb.pickOneOf items). Translated to English, this means "an abitrary list of elements, each of which is picked from items" - or "an arbitrary list of items from the catalog". The property then proceeds to construct a cart as before, and uses List.iter to add one of each item to the catalog. The next line looks up the price for each item with the pro-

Listing 1let totalCorrect = Prop.forAll Arb.catalog (fun catalog -> let items = List.map fst (Map.toList catalog) Prop.forAll (Arb.list (Arb.pickOneOf items)) (fun items -> let cart = factory.make catalog List.iter (Interface.add cart 1) items let prices = List.map (Catalog.itemPrice catalog) items let total = List.fold Price.add (Price 0) prices Interface.total cart .=. total))

Flexible, hybrid API-led software architectures with KongSven Bernhardt (OPITZ CONSULTING Deutschland GmbH)

Kong is a lightweight, cloud-native API solution that makes it easier and faster than ever to connect APIs and micro-services in today’s hybrid, multi-cloud environments. With its agnostic, flexible

deployment approach, Kong can be used in today’s heterogeneous IT system landscapes to integrate a wide variety of data and systems – even across com-pany boundaries – using APIs. In addition to REST APIs, Kong also offers support for gRPC and Graph-QL, which broadens the possibilities to implement modern application architectures. In this presentation, we will discuss deployment patterns and use cases for Kong to demonstrate the flexibility of the platform. Using a practical example, aspects of the API development and deployment process as well as the integration in existing software development processes will be discussed.

Page 28: The API revolution continues! API Whitepaper Selection...Looking to 2020 and beyond, the proportion of data produced and consumed in realtime is growing exponentially . 4 WITPAPR API

28www.apiconference.net

WHITEPAPER API Management

vided Catalog.itemPrice function, calling the resulting list prices. The one ofter that uses Price.add to add all those prices yielding total, and the last line compares that total to the output of Interface.total.

Finding BugsWe can use the totalCorrect property to test a faulty im-plementation of the shopping-cart interface, producing this output from FsCheck (Listing 2).

This says that FsCheck found a counterexample that proves that the property does not hold. The counterex-ample consists of a catalog from the first Prop.forAll and a list of items from the second Prop forAll - map [(Item "t", Price 4)] and [Item "t"; Item "t"], respec-tively.

Thus, if the catalog contains just a single item called t priced at 4, adding that one t twice in a row results in a test failure.

But FsCheck does more. It shrinks the counterexam-ple it found to create a simpler one, and comes up with one that has a simpler name (the empty string instead of t) and a simpler price (4 instead of 1). As this is the simplest counterexample FsCheck could find, it means that just purchasing "one of the empty string" does not trigger the bug. This together with the statement Price 1 = Price 2 gives a clue as to what the cause of the prob-lem is. In fact, when Interface.add is called twice for the same item, it forgets about the first item.

Generating Test Data and Informing the BusinessConsider extending the shopping cart with idea of a dis-count. Imagine the business requirement is that a dis-count can be described as "buy X, only pay for Y", with Y less than X. Such a discount could be described by the following type alias:

type Discount = { receive: int; payFor: int }

The available discounts could be described by a map as-sociating an item with a discount:

type Discounts = Map<Item, Discount>

To take discounts into account, we add a method to the Factory interface:

type Factory = abstract member make: Catalog -> Interface abstract member make: Catalog * Discounts -> Interface

Now, imagine the business also requires that adding an item to a shopping cart should never decrease the price. We can codify this requirement as a property as follows (Listing 3).

There are now three nested Prop.forAll in the prop-erty: It must hold for all catalogs, but also for all dis-counts that are applicable for the catalog, and also any combination of counts and items already in the shop-ping cart before that one additional item is purchased. The subsequent lines prepare the shopping cart, calcu-late the total, add one item (for simplicity, it just uses the first item from the catalog), and calculate the total again, comparing them in the final line. Now, this prop-erty uses the custom function Arb.discounts to generate discounts for a catalog. It is defined as follows:

let discounts catalog = let items = Seq.map fst (Map.toSeq catalog) Arb.map (Arb.pickOneOf items) discount

Listing 3let addingItemIncreasesTotal = Prop.forAll Arb.catalog (fun catalog -> Prop.forAll (Arb.discounts catalog) (fun discounts -> Prop.forAll (Arb.countAndItems catalog) (fun countAndItems -> let cart = factory.make(catalog, discounts) List.iter (fun (count, item) -> Interface.add cart count item) countAndItems let itemPrice = Seq.head (Map.toSeq catalog) let totalBefore = Interface.total cart Interface.add cart 1 (fst itemPrice) let totalAfter = Interface.total cart totalAfter .>=. totalBefore)))

Listing 2quick totalCorrect;;  Falsifiable, after 2 tests (3 shrinks) (StdGen (957853183,296699855)):  Label of failing property: Price 1 = Price 2  Original:  map [(Item "t", Price 4)]  [Item "t"; Item "t"]  Shrunk:  map [(Item "", Price 1)]  [Item ""; Item ""]

Keynote: Building Digital Ecosystems with APIsMatthias Biehl ( API-University.com)

Digital ecosystems offer personalised digital products to customers and new opportunities for building competiti-ve advantage to companies. Digital ecosystems use APIs to tailor, integrate

and bundle the products of companies across indus-tries. This talk explores patterns for building digital ecosystems.

Page 29: The API revolution continues! API Whitepaper Selection...Looking to 2020 and beyond, the proportion of data produced and consumed in realtime is growing exponentially . 4 WITPAPR API

29www.apiconference.net

WHITEPAPER API Management

This generates a map with the keys picked from the items in the catalog, and the values generated as an arbi-trary discount. Here is its definition:

let discount = let arbCount = Arb.choose 1 20 let convertTo (receive, payFor) = { receive = receive; payFor = payFor } let convertFrom discount = (discount.receive, discount.payFor) let valid (receive, payFor) = payFor < receive Arb.convert convertTo convertFrom (Arb.filter valid (Arb.pair arbCount arbCount))

The key part here is the Arb.filter valid (Arb.pair arbN arbN) expression at the end: It says that a discount is constructed from a pair of arbitrary counts, as long as that pair is valid. An arbitrary count arbCount is defined in the first line as a integer between 1 und 20, and the valid function ensure the first business requirements - that customers always pay for less than they bought. The convertTo and convertFrom functions merely con-vert between these pairs and the Discount type.

Checking this property against a correct shopping-cart implementation shows a problem (Listing 4).

The cause is apparent from the shrunk counterexam-ple. It comes with a catalog with a single item, which is discounted as "buy 4, only pay for 1". If three items are already in the shopping cart (first, one was purchased, then two more), buying one more gets the cart over the threshold of 4 and decreases the price to 1.

The insight from this is that, in "buy X, only pay for Y", really Y must be X-1 and cannot be less, which can be codified and communicated back to the business.

This example again may be a bit simplistic, but dem-onstrates the value of property-based testing for explor-ing the requirements and resulting specification, and nailing down seemingly unlikely edge cases.

ConclusionProperties and property-based testing are powerful ad-ditions to the software developer's toolbox. The most

Taming the rising Complexity of Event-driven APIsDr. Matthew O‘ Riordan (Ably Realtime)

Driven by consumer expectations and the proliferation of IoT, by 2023 30 percent of all the world’s data will be consumed in real time – that’s 1.5 times the entire amount of data consumed

today. Corresponding to this, by 2020, 50 percent of APIs will be event-driven. In this talk, I explore the often unforeseen complexities involved in both publi-shing and consuming event-driven APIs as adoption increases. Whilst initiatives like AsyncAPI are making significant progress in helping to define machine and human readable specifications, developers still need to think hard about which dimensions they want their event-driven APIs to deliver on in the future, before they dive in. I will take you on a tour of different pro-tocols that each address event-driven API challenges with varying proficiency, diving into five key comple-xities: integrity vs. latency, throughput, push and pull subscriptions, downstream reliability and durability. At the end of this talk I hope you will see that there is no silver bullet, but there are insights you can take from real-world problems we’ve seen first-hand, and apply that to the event-driven realtime APIs you are publishing or plan to publish soon.

Listing 4quick (addingItemIncreasesTotal factory8);;  Falsifiable, after 1 test (4 shrinks) (StdGen (1040086835,296699864)):  Label of failing property: Price 1 >= Price 3  Original:  map [(Item "uI", Price 2)]  map [(Item "uI", { receive = 4   payFor = 2 })]  [(1, Item "uI"); (2, Item "uI")]  Shrunk:  map [(Item "", Price 1)]  map [(Item "", { receive = 4   payFor = 1 })]  [(1, Item ""); (2, Item "")]

immediate benefit is increased coverage of tests, and thus less bugs. A more important consequence is in the properties themselves: Expressing and thinking about properties encourages thinking about the specification of the system under development. This leads to better understanding of the business requirements, and fre-quently to simpler and more elegant designs.

Dr. Michael Sperber is the managing director of Ac-tive Group GmbH. He is an internationally recognized expert in functional programming. He has also written numerous specialist articles and books on the subject. Michael Sperber is the co-founder of the blog Funk-

tionale-programmierung.de and co-organizer of the developer conference BOB. He is also one of the primary authors of the iSAQB advanced curriculum "Functional Software Architecture".

Links & Literatur

[1] https://github.com/active-group/apicon-2020

[2] https://github.com/fscheck/FsCheck

[3] Koen Claessen, John Hughes: QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs. International Conference on Functional Programming, 2000. https://dl.acm.org/doi/10.1145/357766.351266

[4] John Hughes: Experiences with QuickCheck: Testing the Hard Stuff and Staying Sane. A List of Successes That Can Change the World, pp. 169-186. Springer, 2016.