Django Heresies

Simon Willison EuroDjangoCon4th May 2009

@simonwhttp://simonwillison.net/

http://www.flickr.com/photos/ianlloyd/264753656/

DJANGO IS AWESOME

Selling bacon on the internet

Pulitzer prize winning journalism

Saving children’s lives in Kenya

Heresy“An opinion at variance with theorthodox or accepted doctrine”

Templates

{% if %} tags SUCK

{% if %} tags SUCK• Every time you {% endifnotequal %},

God kicks the Django Pony

{% if %} tags SUCK• Every time you {% endifnotequal %},

God kicks the Django Pony

Don’t invent a programming languageThe template system intentionally doesn’t allow the following:

• Assignment to variables • Advanced logic

The goal is not to invent a programming language. The goal is to offer just enough programming-esque functionality, such as branching and looping, that is essential for making presentation-related decisions.

http://docs.djangoproject.com/en/dev/misc/design-philosophies/

“

”

{% if photo.width > 390 %}...{% endif %}

'''A smarter {% if %} tag for django templates.

While retaining current Django functionality, it also handles equality, greater than and less than operators. Some common case examples::

{% if articles|length >= 5 %}...{% endif %} {% if "ifnotequal tag" != "beautiful" %}...{% endif %}'''

http://www.djangosnippets.org/snippets/1350/

Chris Beaven (aka SmileyChris)

{% load smartif %} replaces the Django {% if %} tag

'''A smarter {% if %} tag for django templates.

While retaining current Django functionality, it also handles equality, greater than and less than operators. Some common case examples::

{% if articles|length >= 5 %}...{% endif %} {% if "ifnotequal tag" != "beautiful" %}...{% endif %}'''

http://www.djangosnippets.org/snippets/1350/

Chris Beaven (aka SmileyChris)

{% load smartif %} replaces the Django {% if %} tag

♡♡

♡

Silencing errors

• 2003: “template authors shouldn’t be able to break the site”

• 2008: “I can't think of a single time this feature has helped me, and plenty of examples of times that it has tripped me up.”

• Silent {{ foo.bar }} is OK, silent tags are evil

• django-developers: http://bit.ly/silentfail

Project layout

http://www.flickr.com/photos/macrorain/2789698166/

Relocatable

TEMPLATE_DIRS = ( # Don't forget to use absolute paths, # not relative paths.)

import osOUR_ROOT = os.path.realpath( os.path.dirname(__file__))...TEMPLATE_DIRS = os.path.join(OUR_ROOT, 'templates')

Relocatable

TEMPLATE_DIRS = ( # Don't forget to use absolute paths, # not relative paths.)

import osOUR_ROOT = os.path.realpath( os.path.dirname(__file__))...TEMPLATE_DIRS = os.path.join(OUR_ROOT, 'templates')

local_settings.py

• svn:ignore local_settings.py ?

• Can’t easily test your production settings

• Configuration isn’t in source control!

Environments

zoo/configs/common_settings.py

zoo/configs/alpha/app.wsgizoo/configs/alpha/manage.pyzoo/configs/alpha/settings.py

zoo/configs/testing/app.wsgizoo/configs/testing/manage.pyzoo/configs/testing/settings.py

zoo/configs/alpha/settings.py

from zoo.configs.common_settings import *

DEBUG = TrueTEMPLATE_DEBUG = DEBUG

# Database settingsDATABASE_NAME = 'zoo_alpha'DATABASE_USER = 'zoo_alpha'

Reusable code

http://www.flickr.com/photos/ste3ve/521083510/

Generic views

def object_detail(request, queryset, object_id=None, slug=None, slug_field='slug', template_name=None, template_name_field=None, template_loader=loader, extra_context=None, context_processors=None, template_object_name='object', mimetype=None ):

object_detail drawbacks

• You can’t swap the ORM for something else (without duck typing your own queryset)

• You have to use RequestContext

• You can’t modify something added to the context; you can only specify extra_context

• That’s despite a great deal of effort going in to making the behaviour customisable

newforms-admin

• De-coupled admin from the rest of Django

• A new approach to customisation

• Powerful subclassing pattern

class Entry(models.Model): title = models.CharField(max_length=255) author = models.ForeignKey('auth.User')

class EntryAdmin(admin.ModelAdmin): exclude = ('author',) def queryset(self, request): queryset = super(EntryAdmin, self).queryset(request) return queryset.filter(author = request.user) def save_model(self, request, obj, form, change): obj.author = request.user obj.save() def has_change_permission(self, request, axj=None): if not obj: return True # access to change list return obj.author == request.user has_delete_permission = has_change_permission

admin.site.register(Entry, EntryAdmin)

Fine grained permissions

class Entry(models.Model): title = models.CharField(max_length=255) author = models.ForeignKey('auth.User')

class EntryAdmin(admin.ModelAdmin): exclude = ('author',) def queryset(self, request): queryset = super(EntryAdmin, self).queryset(request) return queryset.filter(author = request.user) def save_model(self, request, obj, form, change): obj.author = request.user obj.save() def has_change_permission(self, request, obj=None): if not obj: return True # access to change list return obj.author == request.user has_delete_permission = has_change_permission

admin.site.register(Entry, EntryAdmin)

Fine grained permissions

class Entry(models.Model): title = models.CharField(max_length=255) author = models.ForeignKey('auth.User')

class EntryAdmin(admin.ModelAdmin): exclude = ('author',) def queryset(self, request): queryset = super(EntryAdmin, self).queryset(request) return queryset.filter(author = request.user) def save_model(self, request, obj, form, change): obj.author = request.user obj.save() def has_change_permission(self, request, obj=None): if not obj: return True # access to change list return obj.author == request.user has_delete_permission = has_change_permission

admin.site.register(Entry, EntryAdmin)

Fine grained permissions

class Entry(models.Model): title = models.CharField(max_length=255) author = models.ForeignKey('auth.User')

class EntryAdmin(admin.ModelAdmin): exclude = ('author',) def queryset(self, request): queryset = super(EntryAdmin, self).queryset(request) return queryset.filter(author = request.user) def save_model(self, request, obj, form, change): obj.author = request.user obj.save() def has_change_permission(self, request, obj=None): if not obj: return True # access to change list return obj.author == request.user has_delete_permission = has_change_permission

admin.site.register(Entry, EntryAdmin)

Fine grained permissions

class Entry(models.Model): title = models.CharField(max_length=255) author = models.ForeignKey('auth.User')

class EntryAdmin(admin.ModelAdmin): exclude = ('author',) def queryset(self, request): queryset = super(EntryAdmin, self).queryset(request) return queryset.filter(author = request.user) def save_model(self, request, obj, form, change): obj.author = request.user obj.save() def has_change_permission(self, request, obj=None): if not obj: return True # access to change list return obj.author == request.user has_delete_permission = has_change_permission

admin.site.register(Entry, EntryAdmin)

Fine grained permissions

class Entry(models.Model): title = models.CharField(max_length=255) author = models.ForeignKey('auth.User')

class EntryAdmin(admin.ModelAdmin): exclude = ('author',) def queryset(self, request): queryset = super(EntryAdmin, self).queryset(request) return queryset.filter(author = request.user) def save_model(self, request, obj, form, change): obj.author = request.user obj.save() def has_change_permission(self, request, obj=None): if not obj: return True # access to change list return obj.author == request.user has_delete_permission = has_change_permission

admin.site.register(Entry, EntryAdmin)

Fine grained permissions

Objects can be views

• A Django view is a function that takes a request object and returns a response object

A Django view is a callable that takes a request object and returns a response object

Objects can be views

• A Django view is a function that takes a request object and returns a response object

• A Django view is a callable that takes a request object and returns a response object

• Just define __call__() on the class

Example: restview.pyDjango Snippets: http://bit.ly/restview

class ArticleView(RestView): def GET(request, article_id): return render("article.html", { 'article': get_object_or_404( Article, pk = article_id), })

def POST(request, article_id): form = ... return HttpResponseRedirect(request.path)

from django.http import HttpResponse

class RestView(object): def __call__(self, request, *args, **kwargs): if not hasattr(self, method): return self.method_not_allowed(method) return getattr(self, method)(request, *args, **kwargs) def method_not_allowed(self, method): response = HttpResponse('Not allowed: %s' % method) response.status_code = 405 return response

django_openid

• Next generation of my django-openid project

• Taken a lot longer than I expected

• Extensive use of class-based customisation

GitHub: http://github.com/simonw/django-openid

Consumer

LoginConsumer

SessionConsumerCookieConsumer

consumer.py

AuthConsumer

auth.py

RegistrationConsumer

registration.py

Suggestions from django_openid

• Every decision should use a method

• Every form should come from a method

• Every model interaction should live in a method

• Everything should go through a render() method

render()

class Consumer(object): ... base_template = 'django_openid/base.html' ... def render(self, request, template, context=None): context = context or {} context['base_template'] = self.base_template return TemplateResponse( request, template, context ) ...

render()

class Consumer(object): ... base_template = 'django_openid/base.html' ... def render(self, request, template, context=None): context = context or {} context['base_template'] = self.base_template return TemplateResponse( request, template, context ) ...

render()

class Consumer(object): ... base_template = 'django_openid/base.html' ... def render(self, request, template, context=None): context = context or {} context['base_template'] = self.base_template return TemplateResponse( request, template, context ) ...

TemplateResponse

class MyCustom(BaseView):

def index(self): response = super(MyCustom, self).index() # response is a TemplateResponse response.context['counter'] += 1 response.template = 'some/other/template.html' return response

# Two classesSimpleTemplateResponse(template, context)TemplateResponse(request, template, context)

TemplateResponse

• Subclasses can re-use your logic and extend or modify your context

• So can middleware and unit tests

• GZip Middleware writes to response.content, needs work arounds

• Should HttpResponse be immutable?

Ticket #6735, scheduled for Django 1.2

Storing state on self in a class-based generic view is not thread safe

Storing state on self in a class-based generic view is not thread safe

Testing

Django Core

• Excellent testing culture

• Dubious “find... | grep... | xargs wc -l”:

• 74k lines of code

• 45k lines of tests

• “No new code without tests”

• Coverage = 54.4%, increasing over time

Django community?

• ... not so good

• even though django.test.client is great

• Many reusable apps lack tests

• need more psychology!

nose is more fun

• nosetests --with-coverage

• (coming to a SoC project near you)

• nosetests --pdb

• nosetests --pdb-failures

Test views directly

• Hooking up views to a URLconf just so you can test them is fiddly

• ... and sucks for reusable apps

• A view function takes a request and returns a response

RequestFactory

rf = RequestFactory()

get_request = rf.get('/hello/')post_request = rf.post('/submit/', { 'foo': 'bar'})delete_request = rf.delete('/item/1/')

http://bit.ly/requestfactory

The HttpRequest constructor isn’t doing anything useful at

the moment...

A web-based interface?

• Testing would be more fun with pretty graphs

• ... and animated progress meters

• ... and a “test now” button

• ... maybe the Django pony could smile at you when your tests pass

• Cheap continuous integration: run tests every time a file changes on disk?

settings.py is the root of all evil

Why did PHP magic_quotes suck?

• They made it impossible to write reusable code

• What if your code expects them to be on, but a library expects them to be off?

• Check get_magic_quotes_gpc() and unescape... but what if some other library has done that first?

settings.py problems

• Middleware applies globally, even to those applications that don’t want it

• Anything fixed in settings.py I inevitably want to dynamically alter at runtime

• TEMPLATE_DIRS for mobile sites

• DB connections

• How about per-application settings?

>>> from django import dbTraceback (most recent call last): File "<stdin>", line 1, in <module> ...ImportError: Settings cannot be imported, because environment variable DJANGO_SETTINGS_MODULE is undefined.

Grr

Turtles all the way down

http://www.flickr.com/photos/raceytay/2977241805/

“an infinite regression belief about cosmology and the

nature of the universe”

The Django Contract• A view is a callable that takes a request object and

returns a response object

The Django Contract• A view is a callable that takes a request object and

returns a response object

• Primary URLconf: selects a view based on regular expressions

The Django Contract• A view is a callable that takes a request object and

returns a response object

• Primary URLconf: selects a view based on regular expressions

• Application: sometimes has its own URLconf include()d in to the primary

The Django Contract• A view is a callable that takes a request object and

returns a response object

• Primary URLconf: selects a view based on regular expressions

• Application: sometimes has its own URLconf include()d in to the primary

• Middleware: a sequence of globally applied classes process_request/process_response/process_exception

The Django Contract• A view is a callable that takes a request object and

returns a response object

• Primary URLconf: selects a view based on regular expressions

• Application: sometimes has its own URLconf include()d in to the primary

• Middleware: a sequence of globally applied classes process_request/process_response/process_exception

• Site: a collection of applications + settings.py + urls.py

Extended Contract• A view is a callable that

takes a request and returns a response

• URLconf: a callable that takes a request and returns a response

• Application: a callable that takes a request and returns a response

• Middleware: a callable that takes a request and returns a response

• Site: a callable that takes a request and returns a response

Let’s call them “turtles”

http://www.flickr.com/photos/enil/2440955556/

Three species of turtle

• Django request / response

• WSGI

• HTTP

• What if they were interchangeable?

Three species of turtle• Django request / response

• WSGI

• Christopher Cahoon, SoC

• django_view_from_wsgi_app() http://bit.ly/djwsgi

• django_view_dec_from_wsgi_middleware()

• HTTP

• paste.proxy

Micro frameworks

http://www.flickr.com/photos/remiprev/336851630/

juno 782 lines http://github.com/breily/juno

newf 144 lines http://github.com/JaredKuolt/newf

mnml 205 lines http://github.com/bradleywright/mnml

itty 406 lines http://github.com/toastdriven/itty

djnghttp://github.com/simonw/djng

(First commit at 5.47am this morning)

A micro framework that depends on a macro framework

class Router(object):    """ Convenient wrapper around Django's urlresolvers, allowing them to be used from normal application code.

  from django.conf.urls.defaults import url router = Router( url('^foo/$', lambda r: HttpResponse('foo'), name='foo'), url('^bar/$', lambda r: HttpResponse('bar'), name='bar') ) request = RequestFactory().get('/bar/') print router(request) """    def __init__(self, *urlpairs):        self.urlpatterns = patterns('', *urlpairs)        self.resolver = urlresolvers.RegexURLResolver(r'^/', self)

        def handle(self, request):        path = request.path_info        callback, callback_args, callback_kwargs = self.resolver.resolve(path)        return callback(request, *callback_args, **callback_kwargs)

        def __call__(self, request):        return self.handle(request)

Re-imagining the core Django APIs, minus

urls.py and settings.py

http://github.com/simonw/djangopeople.net

http://www.flickr.com/photos/morgantj/2639793944/

Thank you