BEAKER - MAKING SESSIONS AND CACHING ROOMMATES

Alessandro Molina@__amol__

amol@turbogears.org

Who am I

CTO @ AXANT.it, mostly Python company

(with some iOS and Android)

TurboGears2 dev team member

Took over Beaker maintenace in 2015

Mostly contributed to web python libraries: Formencode, MING MongoDB ODM, ToscaWidgets2

Whats Beaker Framework for handling Caching and

Sessions in web applications

Used by many different frameworks:

TurboGears, Bottle, Pyramid, etc...

Created by Pylons Author to solve the

dogpile effect.

DogPile!

The Good Handles the DogPile effect

Provides many backends: Memcache, File,

MongoDB, Redis, SQLAlchemy

A single tool to handle both caching and

sessions with common backends.

The Bad File based synchronization, what about

distributed solutions?

Session as a big BLOB of data doesnt

perform well in case of big BLOBs of data

Updated for too long without breaking

backward compatibility: Too many APIs

Getting on project: PY3 Support Beaker supported Python3 using 2to3

This lead to some bugs (not everything got

properly converted)

Made really hard to run the test suite and

maintain the project.

So my first action as new maintainer...

Rewriting for Py3 a project that involves serializing/deserializing

Porting to PY3: #1 Snakes speak ?Beaker uses BASE64 to encode pickled data

in sessions.

>>> import base64, pickle

>>> data = dict(key='value', otherkey='othervalue')

>>> base64.b64encode(pickle.dumps(data))

'KGRwMApTJ290aGVya2V5JwpwMQpTJ290aGVydmFsdWUnCnAyCnN

TJ2tleScKcDMKUyd2YWx1ZScKcDQKcy4='

BASE64: rfc4648Here are a few requirements that

determine which alphabet should be

used:

o Handled by humans.

Human handle?

TEXT!

Human handle?

TEXT!

Human handle?

BYTES!>>> import base64, pickle

>>> base64.b64encode(pickle.dumps(HELLO))

b'gANYBQAAAEhFTExPcQAu'

Porting to PY3: #2 Session Cookies? Session ID is stored in cookies

CookieSession stores even the whole

session in a cookie Great idea, makes really simple to scale

HTTP Headers are plain text

We are pickling data and base64 so already ASCII

Just need to encrypt it to avoid people from

messing with them.

HTTP CookiesHTTP has allowed field content with text

in the ISO-8859-1 charset [ISO-8859-1],

supporting other charsets only through

use of [RFC2047] encoding. In practice,

most HTTP header field values use only a

subset of the US-ASCII charset [USASCII].

Newly defined header fields SHOULD limit

their field values to US-ASCII octets

Encoding/Encripting data on Py3 Beaker expected to be able to perform

text operations on the result of base64

Relied on AES to encrypt cookies, pbkdf2

to generates keys and base64/binascii

to make it text.

Reads the encrypt_key from config file

4 lines of code == 10 doubts Even Python has no clear idea of what is

text and what are bytes: Both BASE64 and BINASCII work with ASCII

BINASCII accepts unicode (text)

BASE64 accepts bytes

Between Python3.2 and Python3.3 the binascii

changed behaviour of accepting unicode vs bytes

ConfigParser returns text on py3, bytes on py2

Now crying in a corner...

Lets go for something simpler... Review patch to caching decorator

To cache a function, just apply a decorator

that calls function and caches the result

Caching decorator generates cache key Cache Key from the function name

convert parameters to strings

add parameters to cache key

Getting the Cache Key def cache(self, *args, **kwargs):

"""Decorate a function to cache itself with supplied parameters

:param args: Used to make the key unique for this function, as in region()

above.

:param kwargs: Parameters to be passed to get_cache(), will override defaults

Example::

# Assuming a cache object is available like:

cache = CacheManager(dict_of_config_options)

def populate_things():

@cache.cache('mycache', expire=15)

def load(search_term, limit, offset):

return load_the_data(search_term, limit, offset)

return load('rabbits', 20, 0)

"""

return _cache_decorate(args, self, kwargs, None)

Well, maybe...

Unknown parameters Decorator gets *args and **kwargs

Makes sense... doesnt know the real

function arguments

Not so much in fact func(1) and func

(a=1) end up with two different cache

keys

Avoid the doubt Beaker solved this by accepting only

positional arguments on cached functions .. note::

The function being decorated must only be called with

positional arguments.

Users not so happy, breaks people code

when they introduce caching: https://github.com/bbangert/beaker/issues/62

What to do? Leave it as is As been like that for years

Add **kwargs and just document the issue

Add **kwargs and try to be smart using

inspect.getargspec / inspect.

getcallargs but that would be slow

Leave it as is at least isnt broken

Has been a great trip! Making sessions and caching in a single

solution seemed simple, but has actually a

lot of corner case

Beaker Co-Author thrown in the towel and

created dopgile.cache which is based on

beaker code but is much simpler and

doesnt provide sessions

Still is incredibily convenient There are more shared features than

differences (especially in cookie based

marshalled sessions).

Its really convenient, write backends once

and have support for both sessions and

caching on them.

Questions?