Embed Size (px)
Citation preview
Developing your own Swift middleware
OpenStack Summit Atlanta, May 2014
About me
● Christian Schwede
● Developer @ eNovance
● Mostly working on Swift, testing and automation
● Started using Swift in 2012
Agenda
● Using middlewares to extend Swift functionality
● Introduction to wsgi, middlewares and paste.deploy
● Developing our own middleware for Swift Proxy
● Testing & Packaging
● References
Writing Swift middlewares?
Swift middlewares?
● Most of Swift features are implemented as a middleware
○ logging, tempurl, dlo, ratelimit, tempauth, quotas, ...
● Flexibility to extend existing functionality
● No need to fork or modify existing code
● Several additional middlewares outside of Swift
wsgi and middlewares?
wsgi
● simple interface between web servers & web applications
● defined in PEP0333 and PEP3333
● application
○ callable object (with a __call__ method)
● server
○ invokes the callable once for each client request
Client Server App
application objects
● must accept two positional arguments
○ environ: Python dictionary
○ start_response: callable
■ status = „200 OK“
■ headers = [(„header_name“, „header_value“), ]
environ
● REQUEST_METHOD
● PATH_INFO
● QUERY_STRING
● HTTP_HEADERNAME
● wsgi.input
from wsgiref.simple_server import make_server
def myapp(environ, start_response):
body = []
body.append("PATH_INFO: '%s'\n" % environ.get('PATH_INFO'))
headers = [('Content-Type', 'text/plain')]
start_response('200 OK', headers)
return body
srv = make_server('localhost', 8000, myapp)
srv.serve_forever()
Sample server
middleware
● can act as a server for an application and vice versa
● run multiple applications side-by-side
● authentication
● rerouting a request
● content processing
Client Server AppMiddleware
class SummitMiddleware(object):
def __init__(self, app, *args, **kwargs):
self.app = app
def __call__(self, env, start_response):
response = self.app(env, start_response)
if env.get('PATH_INFO') == '/echo':
length = int(env.get('CONTENT_LENGTH') or 0)
return env.get('wsgi.input').read(length)
return response
srv = make_server(‘localhost’, 8000, SummitMiddleware(myapp))
Sample middleware
Testing, packaging & deploying
class FakeApp(object):
def __call__(self, env, start_response):
start_response('200 OK', [])
return ""
class TestSummitMiddleware(unittest.TestCase):
def test_simple_request(self):
environ = {'REQUEST_METHOD': 'PUT'}
req = Request.blank('/echo', environ, body="Hello World")
mw = SummitMiddleware(FakeApp())
resp = req.get_response(mw)
self.assertEqual("Hello World", resp.body)
Testing a middleware
paste.deploy
● load WSGI applications and servers from URI
● uses INI-style configuration files
● separates config from code
● Paste Script can serve applications from config files
● widely used in Openstack
[app:sample]
use = egg:sample#app
[filter:middleware]
use = egg:sample#middleware
suffix = /echoresponse
[pipeline:main]
pipeline = middleware sample
[server:main]
use = egg:Paste#http
port = 8000
config.ini for paste.deploy
setup(name='sample',
packages=['sample', ],
zip_safe=False,
entry_points={
'paste.app_factory': ['app=sample.app:app_factory'],
'paste.filter_factory': ['middleware = sample.middleware:
filter_factory']
})
setup.py
Writing a Swift middleware
Preview middleware
● PUT
○ create a small preview image and store it separate
● GET
○ return preview if QUERY_STRING contains “preview”
● DELETE
○ also delete preview if exists
Useful helpers
● swift.common.utils.split_path
path = /v1/AUTH_account/test/img.jpg
ver, acc, cont, obj = split_path(path)
● swift.common.wsgi.make_subrequest
○ add middleware after authentication middleware
● swift.common.swob.wsgify
○ decorator
@wsgify
def __call__(self, request):
# request.params
# request.path_info
# request.method
# request.environ
# request.body
return self.app
wsgify?
@wsgify
def __call__(self, req):
try:
(ver, acc, con, obj) = split_path(req.path_info, 4, 4, True)
except ValueError:
return self.app
preview_path= '/%s/%s/%s_%s/%s' % (ver, acc, con, self.suffix, obj)
if req.method == 'GET' and request.params.has_key('preview'):
req.path_info = preview_path
Return preview
Extract preview
if req.method == 'PUT':
preview = create_preview(request.body)
if preview:
sub = wsgi.make_subrequest(
request.environ, path=preview_path, body=preview)
sub.get_response(self.app)
Delete preview
if req.method == 'DELETE':
sub = wsgi.make_subrequest(req.environ, path=preview_path)
sub.get_response(self.app)
return self.app
References
3rd party middlewares
● swauth - github.com/gholt/swauth
● swift3 - github.com/stackforge/swift3
● CDMI - github.com/osaddon/cdmi
● Swift informant - github.com/pandemicsyn/swift-informant
● Swift Origin Server - github.com/dpgoetz/sos
● Ceilometer - ceilometer/objectstore/swift_middleware.py
● github.com/enovance/swift-middleware-sample
● docs.openstack.org/developer/swift/
○ middleware.html
○ development_middleware.html
○ development_auth.html
○ associated_projects.html
● legacy.python.org/dev/peps/pep-3333
Something to read
