When Good Code Goes Bad:

Tools and Techniques
for Troubleshooting
Plone


Plone Conference 2008

David GlickWeb DeveloperONE/Northwest

What we're going to cover

1. Preparation for debugging

2. Debugging tools

3. 2 case studies

4. Tips & tricks

Newbie programmer's reaction:

Absolute terror

Experienced programmer's reaction:

Absolute terror

Maybe it's a little bug...

...but who knows
what's in there, really!

What could be wrong?

Trivial typo

Logic error

Failure to consider certain input values

Faulty assumptions about the state of the system

Deep voodoo

Mental preparation

50% of debugging is not getting fazedthe other 60% is being clever as all get outyeah, debugging requires you to give 110%

Remember: You are in control.

computers always do what they are told: try to figure out where there is a mismatch between your assumptions about what the computer is doing, and what it's actually doing

Remember: You are not alone.

- Google- Plone mailing lists- IRC: #plone

Use the source, Luke.

- open source- Python is very readable

Getting ready...

Don't use debugging tools
in production.

Security

Performance

- security- performance- zc.buildout is best practice for creating a repeatable zope environment, which makes creating a development sandbox trivial

Run zope in foreground mode.

Buildout:

$ bin/instance fg

Pre-buildout:

$ zopectl fg

Turn on verbose security.

Buildout:

in buildout.cfg, [instance] section:

verbose-security on

Pre-buildout:

in zope.conf:

verbose-security on

security-policy-implementation python

Stop ignoring errors in /error_log

Debugging tools

1. What is that thing?

Firebug (Firefox)

http://tinyurl.com/2z4tvg

IE Web Developer Toolbar (IE)
http://tinyurl.com/iedevtoolbar

WebKit (Safari)
http://webkit.org

Google Chrome inspector

Investigating HTML,
CSS, and Javascript

>

Investigating Plone
viewlets and portlets

plone.app.gloworm
http://tinyurl.com/gloworm

Investigating
the current page context

Products.Clouseau
http://tinyurl.com/clouseau

2. What code controls this?

Seek and ye shall find!

Grep

Multi-file search and replace feature
in editor of choice

Code indexers (etags, glimpse)

Where is the code?

From a filesystem page template:

(gives you the dotted path of module + class of obj)

From any page template:

(gives you the class name of obj)

collective.recipe.omelette

Before:

After:

http://tinyurl.com/collective-omelette

The same traits that make eggs ideal for distributing code make them a pain for writing and debugging code.

3. What's that code doing?

Debugging is an art, not a science

Logging

context.plone_log(my_var)

Output will show up on console

Useful for when some code is being run repeatedly and you want to make sure it's operating on the correct values

pdb: the Python debugger

pdb: the Python debugger

Trigger with:

import pdb; pdb.set_trace()

Don't forget to remove the set_trace!

4. What's Zope doing right now?

DebugInfo page (built in):
/ControlPanel/DebugInfo/manage_main in ZMI

DeadlockDebugger product
http://tinyurl.com/deadlockdebugger

5. Why is this so slow?

YSlow frontend performance

http://developer.yahoo.com/yslow/

PTProfiler page templates

http://plone.org/products/ptprofiler

ZopeProfiler Python calls

http://pypi.python.org/pypi/Products.ZopeProfiler

What does it look like
in real life?

Case #1:

plone_newbie4022's Quick Installer problem

(from #plone, 2008-09-08)

Reading a traceback

module

Line #

method

Morerecent

Snapshot of what code is executing

Doesn't include the values

Reading a traceback (cont'd)

For page templateexpressions

Template file location

The actual TAL expression
that was called

Location of expression
within the template

The actual Error message!

Bonus (but irrelevant) error

The Error
happened
here!

Common Python exceptions

AttributeError

e.g. obj.nonexistent_attribute

ImportError

e.g. import nonexistent_module

IndexError

e.g. my_list[42],
if my_list doesn't have >=43 elements

Common Python exceptions

KeyError

e.g. my_dict['nonexistent_key']

TypeError

e.g. foo = ('tuple',) + 'string'

ValueError

e.g. int('string')

Common Zope exceptions

Unauthorized

NotFound

Redirect

TraversalError

ComponentLookupError failed zope 3 lookup

ConflictError ZODB conflict

Lessons from Case #1

The problem is sometimes (usually?)
not where you think the problem is.

Think carefully about both:

What the code is doing

What the state is of the objects in your ZODB

Often you don't have to change any code to fix a bug, once you understand what is going on.

What does it look like
in real life?

Case #2:

some (only somewhat) contrived

catalog trouble

plone.reload

Reload Python code without restarting Zope.

Just go to /@@reload.

pdb commands

l / list: print the code. l 42 == list line #42

w / where: print a traceback

n / next: run the next command

s / step: enter a method that is being called

r / return: execute until the current method completes

c / continue: continue execution indefinitely
(end pdb session)

advanced pdb commands

b / breakpoint: b 42 == set a breakpoint on line 42

cl / clear: cl 1 == remove breakpoint #1

j / jump: j 42 == jump to line 42

u / up: move up in the call stack

d / down: move down in the call stack

Full pdb docs: http://www.python.org/doc/2.5.2/lib/module-pdb.html

pdb idioms

pp obj pretty-print an object

dir(obj) list an object's attributes and methods

obj.__dict__ -- list an object's data attributes

obj.__doc__ -- print an object's docstring

args list the arguments passed to the current method

retval print the value that will be returned from the current method

!expr force expr to be treated as a Python expression rather than a pdb command

Lessons from Case #2

If you don't have a traceback, finding the source of misbehavior can be hard. Be persistent.*

* Bad Zope pun not intended.

Try the most obvious things first.

(Assume there's a bug in your own code,
rather than in Zope/Plone.)

Tips and Tricks

PDBDebugMode product

Automatically get a pdb whenever there is a python exception

Invoke a pdb on any context via /@@pdb view

Invoke a pdb on anything
via /path/to/anything?pdb_runcall=1

Debugging python scripts

Use the zdb product
http://www.simplistix.co.uk/software/zope/zdb

from Products.zdb import set_trace; set_trace()

Ensures that you can (l)ist
the code of the Python script

Doesn't work for form controller Python scripts (.cpy) though

When stepping into calls in restricted python, you will first enter guarded_getattr, which you can (r)eturn out of and then the (n)ext call will be the one you want

Tab key auto-completion in pdb

Add the following to ~/.pdbrc:

import rlcompleter

pdb.Pdb.complete = \ rlcompleter.Completer(locals()).complete

Why aren't my changes coming alive?

template, image & stylesheet customizations:

put CSS, Javascript, KSS registries in debug mode, or at least re-save them

make sure you don't have something in portal_skins/custom or portal_view_customizations that's overriding your filesystem item

flush browser cache

Why aren't my changes coming alive?

changes to python code or ZCML:

restart Zope, or use plone.reload

Why aren't my changes coming alive?

changes to GenericSetup profiles:

re-run profile or reinstall product (don't need to restart zope)

installing a new product:

if not in the Products namespace, did you add a ZCML slug?

Debugging KSS

Use the FireKiss add-on for Firebug:
http://tinyurl.com/firekiss

Configure KSS debug log at /@@kss_devel_mode/ui

Be aware that KSS will time out if you try to put a pdb set_trace in your server-side KSS action code

Common red herrings

Not authorized to access binding.

A deferred permission error from earlier in code execution.

See http://tinyurl.com/unauthorizedbinding

Quick Installer swallows import errors

(Fixed in Plone 3.1.2)

AttributeError: getGroups

Usually means you moved the site to a new Zope, whose acl_users needs to be converted to be PAS-enabled.

You can trigger that conversion by adding a brand new Plone site.

No traversable adapter found.

Usually a symptom of Five failing to load.

Start Zope in foreground mode and watch the console for the real traceback.

Credits

Cave openings, screenshots David Glick

First computer bug US Navy / public domain

Flickr users:

rotten apples alisonedunn CC Attribution

gearing up for spelunking jarvist CC Attribution

pause button Jobot Da Robot CC Attribution Share Alike

debug yourself joshlewis CC Attribution NC Share Alike

no fish waldopepper CC Attribution NC

These slides are licensed under the following license: Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License