Embed Size (px)
DESCRIPTION
Tempest is an Openstack test suite which runs against all the OpenStack service endpoints. It makes sure that all the OpenStack components work together properly and that no APIs are changed. Tempest is a "gate" for all commits to OpenStack repositories and will prevent merges if tests fail.
Citation preview
OpenStack QA and Tempest
David Kranz
May 6, 2013
Monday, May 06, 2013
Outline
OpenStack QA
Tempest
Future directions
How to contribute
Monday, May 06, 2013
Current OpenStack QA Landscape
Upstream (OpenStack QA project)
Tempest (functional and integration tests)
Grenade (upgrade testing)
Gerrit/Jenkins (continuous integration and gating)
Projects
Unit and functional tests
Packagers and Deployers
Testing in real multi-node environments
Stress and performance testing
Monday, May 06, 2013
What is Tempest?
Comprehensive test suite for OpenStack
API validation
Integration testing
Can be configured to run against real
deployments
Runs as part of the gating process or on a
periodic basis
Written in Python
Monday, May 06, 2013
What is Devstack?
Opinionated single-node deployment
Creates complete OpenStack system
All components configurable
Used by developers, testers and by jenkins
The jenkins “gate” uses devstack and Tempest
https://github.com/openstack-dev/devstack.git
Monday, May 06, 2013
What is “the gate”?
Tests that run on every commit before merging
Includes per-project unit tests
Tempest runs on all commits to all projects
Integration tests help make sure commits do
not break other projects
Blocks incompatible API changes
https://wiki.openstack.org/wiki/Governance/Appro
ved/APIStability
Monday, May 06, 2013
Running Tempest with Devstack
Stack.sh runs devstack and creates
/opt/stack/etc/tempest.conf
Tempest will spin up images, servers, volumes,
etc., do things to them, bring them down.
Takes about 45 minutes to run all tests
Edit tempest.conf to run against real deployment
Monday, May 06, 2013
git clone git://github.com/openstack-dev/devstack.git
cd devstack
./stack.sh
cd /opt/stack/tempest
./run_tests.sh [tempest/tests/...]
Example of Tempest Test Case
Monday, May 06, 2013
def test_create_with_existing_server_name(self):
# Creating a server with a name that already exists is allowed
server_name = rand_name('server')
resp, server = self.create_server(name=server_name, wait_until='ACTIVE')
id1 = server['id']
resp, server = self.create_server(name=server_name, wait_until='ACTIVE')
id2 = server['id']
self.assertNotEqual(id1, id2, "Did not create a new server")
resp, server = self.client.get_server(id1)
name1 = server['name']
resp, server = self.client.get_server(id2)
name2 = server['name']
self.assertEqual(name1, name2)
Test cases use unittest2
From tempest/tests/compute/servers/test_servers.py
Tempest in the Gate
Three tempest runs per commit
MySQL
PostGresSQL
Quantum
Full tempest run
Spins up blank Ubuntu 12.04 node
Installs with devstack
Runs 700+ tests
Takes about 40 minutes in gate
Monday, May 06, 2013
Opentack QA Roadmap
Move more testing upstream (shared effort!)
Support multi-node testing in jenkins (triple-o)
Integrate reports from downstream tempest runs using various real hardware configurations
Performance testing
Better upgrade testing
Add newly incubated projects to Tempest
Use Tempest as validation suite for the OpenStack trademark
Improve Tempest performance by parallel execution
Make stress testing reliable
Monday, May 06, 2013
Where Tempest needs Help
New tests to cover rest of API
Strong on Nova, weaker on other projects
Lots of extensions untested
New integrated projects: ceilometer and heat
Improving performance
Upgrade testing
Stress testing
Monday, May 06, 2013
Getting Involved with Tempest
Hang out with us in #openstack-qa
Weekly QA meetings Thursday 17:00 UTC
Get on openstack-qa mailing list
http://lists.openstack.org/cgi-
bin/mailman/listinfo/openstack-qa
Always looking for help filling test gaps
Monday, May 06, 2013
Questions?
Monday, May 06, 2013
