© 2016 VMware Inc. All rights reserved.

“Together, or Not At All?”Interop WG: The Interoperability Standard for OpenStack

Mark T. Voelker, OpenStack ArchitectUpdated January 2017

2

Perhaps inherent in that ambition is the promise of interoperability.

Code against one set of capabilities and API’s, take your pick of many public clouds, distributions, appliances, and services.

As it turns out, even clouds that are on some level the same can look and act very differently.

OpenStack aims to be a “ubiquitous Open Source Cloud Computing platform that will meet the needs of public and private clouds, regardless of size…”

3

There are ~4,650 config options in the tc-approved-release OpenStack projects.And ~1,073 policy.json configurations (as of late 2015).

OpenStack has a lot of nerd knobs that can affect how a cloud behaves. In fact…

4

You can also put things in front of OpenStack that change the behavior users see…like firewalls.

…and load balancers, API gateways, SSL configurations…

5

The mechanics of what’s under the hood may change behavior too…

(take image formats supported by various virt drivers & storage platforms for example)

6

And of course different types of workloads use the cloud in different ways…

7Think you can code against that?

Mark T. Voelker (@marktvoelker)• OpenStack Architect @ VMware, Interop Working Group co-chair, Triangle OpenStack Meetup co-founder,

OpenStack Foundation Member #54, OpenStack ATC, Ex-Puppet OpenStack core dev, OpenStack East Steering Committee

• Fact: can be bribed with doughnuts• In copious (hah!) spare time: OpenStack solutions, Big Data, Massively Scalable Data Centers, DevOps,

making sawdust with extreme prejudice, raising two great kids with my awesome wife in North Carolina

“A computer nerd….is somebody who uses a computer in order to use a computer.” –Douglas Adams

9

[note: this talk will be slightly more entertaining if you’re a science fiction fan…

…otherwise it will merely be somewhat informative.]

10

Why Should Vendors Care About Interoperability Standards?• It’s good for your users

• It helps you promote your product

• It helps applications be developed for your platform

• It’s now required if you want to call your product OpenStack.

11

def list_images():  “”” A function to list images. Because all OpenStack Powered Platforms can do that…somehow.””"  if $cloud == ‘vendorA’:     # TODO: this also works for vendorX     list_images_via_nova_image_api()  elif $cloud == ‘vendorB’:     # TODO: this also worked for vendorY last week but now, um?     list_images_via_glance_v1()  elif $cloud == ‘vendorC’:     list_images_via_glance_v2()  else:     # I dunno what cloud this is, but it’s OpenStack Powered! So something must work. # Resort to trial and error since we don’t know.     try:        list_images_via_nova_image_api()     except NopeError:         # D’oh, guess that wasn’t it…         try:            list_images_via_glance_v1()         except StillNopeError:            # Aww…well third time’s the charm?             try:               list_images_via_glance_v2()            except NopeNopeNopeError:               rage_quit()

This function could also be called:

not_winning()

12

Someone should make a standard.

Situation:Interoperability is hard.

13

Soon

Soon:

Want to prevent that?

When the community works together with the Interop Working Group, we can.

To understand how, you need to know how OpenStack is governed.

14

OpenStack is primarily governed by two bodies:

Technical Committee

& the Board of Directors

(you decide which is which)

15

The Technical Committee provides:“technical leadership for OpenStack as a whole…..enforces OpenStack ideals (Openness, Transparency, Commonality, Integration, Quality…), decides on issues affecting multiple projects…..”

16

“The Board of Directors provides strategic & financial oversight of Foundation resources and staff.”

Which includes

these things

openstack.org/marketplace

17

Interop WG is a Board activity.

• One Director serves as co-chair, other co-chair elected by participants.

• It’s work and procedures must ultimately be approved by a vote of the Board, not the +2’s of it’s most trusted reviewers.

• It produces “Guidelines”, not infrastructure code.

• It can use the OpenStack trademark and logo as both a carrot and stick.

• It can make requirements for products that call themselves OpenStack.

18

So what’s a Guideline, then?

• A list of Capabilities that products must support.

• A list of Tests products must pass to prove it.

• A list of Designated Sections of OpenStack code they must use to provide those Capabilities.

(also: a list of exceptions & things that might be required in the future)

19

What’s the cadence of Guidelines?• New Guideline every 6 months• Each Guideline covers 3 OpenStack releases• Only the newest 2 can be used for new logo

program qualification

20

Things you won’t find in Interoperability Guidelines:

• Stuff that end users don’t see or can’t use:

• Admin-only API’s• RPC API’s• DB Schema• HA Requirements

• Stuff that’s intentionally pluggable:

• Virt/net/storage drivers• Middlewares• Specific databases

• Stuff that doesn’t have tests

• Stuff that’s being deprecated (usually…more on that in a minute)

21

How do we decide what gets in?

10 Guiding Principles12 Core CriteriaA giant list of tests

New Capabilities must be “advisory” (non-required) for one Guideline before becoming required.

22

Interop Criteria(Currently all weighted fairly equally with a few small deviations)

https://github.com/openstack/defcore/blob/master/doc/source/process/CoreCriteria.rst

23

Say more to me about these tests….

• Must be under TC governance• All tests today are in Tempest (per the TC’s request)• Must work on all releases covered by a Guideline• Typically run via the RefStack wrapper which reports results to

refstack.openstack.org and shows which Guidelines you meet

Details: http://openstack.org/interop

http://refstack.openstack.org

24

Sometimes things go amiss…Tests can be “flagged” (not required for the duration of the Guideline) in some cases:• Capability fails to meet Criteria

(e.g. was scored incorrectly)

• Test fails/is skipped due to an accepted bug in the project

• Test fails/is skipped due to an accepted bug in the test

• Test fails because it requires non-required Capabilities

• Test reflects an implementation choice that isn’t widely deployed even though the Capability itself is.

25

When does all this happen?

• Summit-3 months: preliminary draft

• Summit-2 months: ID new Capabilities

• Summit-1 month: Score Capabilities

• Summit: “Solid” draft

• Summit+1 month: Self-testing

• Summit+2 months: Test Flagging

• Summit+3 months: Board Approval

(Note: 2015 was weird in that we had a very accelerated schedule to get DefCore bootstrapped…above is what it looks like from now on.)

26

Why isn’t $my_favorite_thing in the current Interoperability Guideline?

27

28

• It didn’t meet Criteria (scored too low)

• It wasn’t scored in time (scoring is surprisingly hard to get right)

• It was admin-only or driver-specific

• That project isn’t yet widely deployed

• There wasn’t a test for it

• It didn’t score highly across all releases covered in that Guideline

• Nobody brought it up yet

29

I’m an OpenStack Developer

I have a really cool new feature...

How do I get it into the Interoperability Guidelines?

30

I’m the Interop WG co-chair.

I have a blog post you should read!

http://markvoelker.github.io/blog/defcore-for-devs/

• Document it well

• Ensure it has usable tests

• Foster adoption among users, SDK’s, & other projects

• Be patient: needs to be in 3 OpenStack releases

31

Do we get to offer feedback?

Absolutely!

• Feedback built into Interop WG scoring cycles

• Feedback encouraged for Advisory capabilities

• Feedback encouraged via flag requests

• Feedback via User Survey• Feedback via RefStack

community-visible results (you may also buy me a beer & bend my ear about interoperability anytime)

32

How do I make RefStack work for me?It’s actually not that hard.

Instructions here which boil down to:1. Download refstack-client.2. Run the “setup_env” script.3. Configure tempest for your cloud.4. Run refstack-client to execute tests.5. Upload results to refstack.openstack.org and review.

33

Why run all the tests & upload results?

Because more data is better.

This gives us additional data about what works in your cloud. With data from lots of clouds, we can make better scoring decisions in the future when considering adding Capabilities to Guidelines.

34

What if I don’t pass all the required tests?

Don’t panic.

• Figure out why some tests failed. • Was it environmental (e.g. a timeout due to

storage being slow)? Tweaking tempest config may help.

• Was it due to a bug in the test? • Was it due to a bug in OpenStack?

• Do you have grounds for requesting a flag?• Valid reasons for flagging a test and how to

do it can be found here.• Talk to us!

• We have an interest in helping you succeed.

• interop@openstack.org is here to help!• Catch us on IRC at #openstack-interop or

#openstack-refstack.

35

What’s Next?

• Work begins on 2017.08• More testing

• Vendors currently testing against just-released 2017.01• Vertical Programs

• Example: what would “OpenStack Powered NFV” look like?• New component programs and/or “mixin” programs

• Example: what would “OpenStack Powered Compute with Database” look like?

• Could these be a conduit to projects defining (early on, for themselves) what interoperability looks like to users of their project?

36

Sometimes the enemy is us.

• Sometimes projects are slow to adopt support for each others’ new API’s and features.

• Sometimes projects provide multiple ways to do the same things (and sometimes they’re mutually exclusive).

• Sometimes we don’t have good data about what’s really supported.

• Sometimes tests use admin credentials unnecessarily or lump many Capabilities into one test.

37

For this to work, we have to communicate as one community.

• Board of Directors/Interop Working Group

• Technical Committee/technical contributors

• End Users

• Vendors

• Ecosystem

38

But what interoperability issues should we attack first?

How about the ones in this report?

39

What interoperability issues should I, an end user, watch out for ?

Here’s that report again.

1. How we test interop2. Variance in API evolution3. External network connectivity4. API and Policy discoverability5. Understanding the Interop WG

40

Did you mention something about….

...verticals?

41

Think about the Linux ecosystem.

The kernel & userland you run in your datacenter…

42

…are probably a bit different than the one you have on your phone.

(or in your router, submarine, cash register, seatback entertainment unit, or drone)

43

We still think of both as “Linux.”

They’re just specialized for specific use cases.

We’re seeing specialization in OpenStack now too.

44

Within products addressing a particular use case, interoperability is still useful and important.

Otherwise things might not behave as expected when moving from one cloud or product to another.

45

We’re just getting started designing vertical programs. Much depends on the use case.

Some things a use case might need:

• Capabilities not commonly exposed in general purpose compute clouds

• Specific attributes in the API• Scenario tests (beyond API

availability, maybe beyond Tempest)

• Specific control plane design choices

• Performance criteria.• Admin API’s• Different OpenStack projects

46

The key is to take the general methodology and criteria we’ve already developed….

...and apply them in a way that’s specific to a vertical use case.

47

Some examples of things a general-purpose compute clouds might not need for interoperability that an NFV cloud might:

• SR-IOV support• High PPS network

flow rates• Provider network

support• NUMA topology

aware scheduling• CPU pinning• Active-active control

plane design choices

48

We’d like to get moving quickly on developing programs for vertical use cases.

There seems to be demand, and people ready to help. NFV seems like a good first target.

Buckle up!

49

I’ll be jumping in myself.

Speaking of which, we should really wrap up today’s talk and get on with it…

Want to learn more?

• 2017.01 Guideline• 2016.08 Guideline• Next Guideline draft• Public RefStack Server• OpenStack Interop Homepage• Core Criteria• Interop WG procedural overview• Lexicon of Interop WG terms• Interop WG wiki & meeting Info• How to submit patches

51“Do what I do. Hold on tight and pretend it’s a plan!”

52

Thank you.

[with apologies to fine the folks at the BBC’s “Doctor Who”]

(please don’t have me arrested)