Embed Size (px)
Citation preview
Getting Started with Contributing to OpenStack
Mike [email protected] on freenode
An Introductory Crash Course on OpenStack Development
Who Am I?
● Mike Turek● Software Engineer at IBMIBM in the Linux
Technology Center● Started in June 2014 after finishing my Master's
in Computer Science at Binghamton University● Currently focused on Power enablement for
OpenStack's Ironic project.● Specifically CI testing using baremetal nodes.
What to Expect
● Overview of OpenStack and a handful of its integrated projects. A focus on Nova.
● Overview of DevStack and its role in OpenStack
● Overview of Gerrit and some best practices when working with the community.
● A demonstration of DevStack and a sample bug.
Main OpenStack Components
● OpenStack Compute (Nova) Provision and manage instances
● OpenStack Network (Neutron) Manages network resources.
● OpenStack Storage (Cinder & Swift) Block and object storage respectively
● Shared Services (Glance, Keystone, Heat, Trove, etc.)
Basic OpenStack Structure
Nova
● Manages instances, flavors, images, and network resources
● Spun off many of the integrated projects. Glance is the image service Neutron is the network service (Will eventually
deprecate nova-net) Ironic replaces the nova baremetal driver
Nova Resources
Instances Virtual Machines – the most common usage. Several
drivers accomplishing this (libvirt, vmware, hyperv). Physical Hardware – Originally just the baremetal driver
in Nova. Now accomplished by the Ironic project. Containers – Currently accomplished via Nova-Docker
and Libvirt-LXC drivers. Whatever else might come along! Nova is extensible.
Flavors Specifies what combinations of resources an instance
can use Images
Specifies the OS image to be used to build the instance.
More on Shared Services
● Glance Storage and retrieval of images
● Keystone Identity service that projects talk to in order to
authenticate requests.
● Oslo Common functions used throughout OpenStack
● Ironic Allows provisioning baremetal machines as simply
as virtual machines
Governance
● Each project has a project team lead (PTL) Makes key decisions about the project. Priorities for
a cycle, etc.
● Each project has core reviewers Well respected community members get higher
voting power. You need two core reviewers to approve a patch.
OpenStack is...
● Open source Apache 2.0 license
● Written in Python For the most part
● On a six month release cycle Named in alphabetical order, each referring to
something related to that release's summit location
● Not simple to deploy Vastly configurable, not simple to deploy
DevStack
● Quickly and easily deploy OpenStack● Creates an all in one environment
All services run on the same machine
● OpenStack integrated project Important to developers and the Tempest project
● NOT production ready Intended for creating volatile environments Great for development and testing
DevStack local.conf
● This will provide everything we need for today.
● The [[local|localrc]] section local refers to when in the stacking process to
source this information. (One of the first things to happen)
localrc refers to the file using these options.
[[local|localrc]]ADMIN_PASSWORD=supersecretDATABASE_PASSWORD=$ADMIN_PASSWORDRABBIT_PASSWORD=$ADMIN_PASSWORDSERVICE_PASSWORD=$ADMIN_PASSWORDSERVICE_TOKEN=tokenizemecaptain
The DevStack Toolbox
● stack.sh Sets up your environment using local.conf
● unstack.sh Tears down the environment
● rejoin-stack.sh Devstack runs each component in foreground mode
in a separate screen. After reboot screens must be recreated and components restarted.
● create-stack-user.sh Creates a user account suitable for running
DevStack.
Gerrit
● A code collaboration tool● Adds the concept of a store of pending
changes● Provides a forum for code reviews
Each patch has a web page where you can review, comment, and vote on patches
● Integrates with git via the git-review plugin● Won't be going into setup, for help see
https://wiki.openstack.org/wiki/Documentation/HowTo/FirstTimers#Set_up_git_and_git_review
Gerrit
https://review.openstack.org/Documentation/intro-quick.html#_the_life_and_times_of_a_change
The Gerrit Review System
● Patches come in and get voted on by Core reviewers (-2, -1, 0, +1, +2) Reviewers (-1, 0, +1) CI Systems (-1, 0, +1)
● Each patch needs two +2 votes to get approved Fix will then be committed A -2 from a core will always block the patch Fix will eventually be merged into the in progress
branch.
Launchpad
● OpenStack uses Launchpad for bug tracking● Each project has a launchpad page
https://bugs.launchpad.net/nova
● Where you can find, report, and discuss bugs.● Also used for blueprints
Launchpad Blueprints
● A tool for planning larger features and contributions
● Same flow as a patch● Usually submitted towards the beginning of a
cycle.● Specifies problem, approach, alternatives, and
impacts● Submitter is usually the implementer.
More on Finding Bugs
● Bugs can be organized by Priority, Status, and other qualities within Launchpad
● Highly recommend the advanced search Lets you filter out bugs that have people assigned Show certain priorities and tags (low-hanging-fruit)
● Bugs with Triage status Solution has been discussed but needs someone to
implement it. Can be a good place to start
Best Practices
● Keep patches small. A larger patch may require a blueprint
● Make your commit messages clear. A commit message should succinctly explain
What your patch does. Why your patch is needed.
● Keep discussions on the review page. E-mails and IRC chats can be helpful, but in the
end its best to keep discussions public and organized.
Nobody's Perfect
● Patches rarely are accepted on their first submission
● Expect to get -1's Just the community saying that a change needs to
be made to your patch Nothing personal and not a big deal Discuss with the community how to resolve their
concerns and resubmit
Other Tips
● Communities will sometimes do bug days or sprints. A good way to find and discuss bugs Can be a fun way to contribute and get to know the
community Following IRC chats and mailing lists will help you
find out when these are happening
● Code reviews are appreciated A great way to build trust within the community and
learn the code base.
Need Help?Talk to the Community!● Each project has a mailing-list
Great place to discuss features and bugs
● Each project has a freenode IRC channel Development discussions, not usage questions.
● Each project has a weekly meeting. Where to follow what's important to the community Open section to bring up concerns and bugs.
● See each project's https://wiki.openstack.org page for information. Example - https://wiki.openstack.org/wiki/Nova
DemonstrationSlides are provided for audience use and will not be shown during the presentation. If you'd like a
copy of this deck, please let me know!
Python Clients
● Provide a simple interface to the OpenStack services
● All follow the following format <project_name> <openstack_options> <command>
<command_options> <positional_arguments>
nova osusername mjturek boot flavor 3 image ubuntu mjturek_instance
● Help commands for each project and each command nova help nova help list
Sample Bug
● Let's review a sample bug. https://bugs.launchpad.net/nova/+bug/1336977
● A simple usability bug.● Disclaimer – This demo will be on a bug from
the Juno time frame. This means that I needed to use the stable/juno branch rather than the current upstream branch for this demo.
Create a branch
● Each project uses git● Branch within the directory of the project you're
modifying. For nova, branch within /opt/stack/nova
● To create a branch git checkout -b mjturek/1336977
Replicate the Bug
● Bug report indicates that uninformative log happens when there's a NoValidHost exception
● Could overload the system until it can't provide a host.
● Let's just force the NoValidHost to occur via code.
Where to Force NoValidHost
● Notice that the bug report shows an exception occurring “WARNING nova.scheduler.driver”
● This is the class where the exception is reported
● So we know it's happening during scheduling.● We're using the FilterScheduler
Found in nova.conf
Where to Force NoValidHost
● Let's modify the FilterScheduler to throw the exception Find where it normally throws a NoValidHost Force it to happen
● Should now see the Exception in the logs and the instance in an error state
Uh oh...
● The 'nova boot' succeeded...● It seems that our changes didn't take.● Need to restart the process for changes to take
effect.● Enter 'screen -x'
screen● screen is a fullscreen window manager.● DevStack deploys each OpenStack service in a
process within one terminal window.● Some useful commands
Ctrl+A D – detach from screen (return to terminal) Ctrl+A N – Go to the next screen Ctrl+A P – Go to previous screen Ctrl+A [0-9] – Go to respective screen number Ctrl+A [ – Copy mode (Lets you scroll back through
logs) Ctrl+A “ – Enter a window number to switch to.
Restart the Process
● We modified nova code. Go through each screen and restart ones starting with n-
● Switch to each screen Ctrl+C to stop the process Up one entry (up-arrow on keyboard) Press enter.
● Let's try another 'nova boot'
There's the Exception
● Look through the nova screens (screens that start with n-) and you should find the exception.
● We're now seeing what the bug reporter is seeing.
● Let's improve the logging message!
Find Where to Modify
● Remember the warning we see? “WARNING nova.scheduler.driver”
● Again, this is the class where the exception is reported
● Found in '/opt/stack/nova/nova/scheduler/driver.py'
● Searching for the exception shows where the modification needs to happen.
The Fix
● It seems that when a NoValidHost exception happens, we simply set the state to error, and throw a warning at the user.
● NoValidHost exceptions have a 'reason' string in them.
● Let's report that 'reason' string!
Clean Up
● We don't want to submit that exception being forced! This would break nova!
● We only forced the exception for testing, so remove it.
● A quick test should show instances are succeeding again.
● It's time to submit, but first we need the git-review plugin
Submit Your Patch
● Running 'git diff' will show your changes● Now commit your changes
git commit -a
● If after initially committing you need to make more changes, you can update your commit with git commit -a –amend
● You will be asked to enter a commit message
The Commit Message
More informative novascheduler logging
This patch adds a warning log entry for when a NoValidHostexception causes a scheduling error. The NoValidHostexception's message will be logged before reporting thatthe machine went to error. Previously, all other exceptionswould show a stack trace. However, since the NoValidHost caseis skipped, a user may not understand why his/her machinewent to an ERROR state. This patch seeks to remove anyconfusion.
ClosesBug: 1336977
● The first line has a 50 character limit● The body has a 72 character limit per line● Ending section has tags for Closes-Bug: <bug number>, Implements:
<blueprint>, Change ID: <uuid generated by git review>● For specifics, see:https://wiki.openstack.org/wiki/GitCommitMessages#Summary_of_Git_commit_message_structure
Submit
● Now that you've committed your changes to your branch, you can submit it for review
● Make sure you're within the project's directory and type git review
● Upon submission you'll receive a link to your review page.
Debugging
● When you require more robust debugging● Import your favorite python debugger (pdb,
pudb) and set breakpoints where needed● Restart the processes where you've added
breakpoints● Pdb will come up within the screen of the
respective process
