The Worst Code I Ever Wrote

The worst code I ever wrote (maybe)

Tomas Doran

@bobtfish

tdoran@yelp.com

15/11/2013

!•‘Working’ != ‘done’

– I’m a terrible person – Working now doesn’t mean you can maintain it

•Functions for reuse – and my horrible propensity for inline templating

•exec {} abuse – Lets fit an entire shell script in the command => “…”

•Looping – Hacks you have probably used. – Doing it sanely with puppet >= 3.2

Key lessons

tl;dl: This talk is about bad puppet code, and how and why to do it better. The worst code I ever wrote is a lie ;) We’re going to concentrate on 2 main topics - functions and looping, with a nod to exec abuse.

!

• Legacy user management code –Old non-modular code –Have to fit in –Can’t change everything

• Built in sshkey {} – Not suitable for our requirements

Managing ssh keys - gitolite

We’re rapidly updating our puppet codebase, but the user management code is one of the most complex parts, so we wanted to be minimally invasive.. We needed to generate a very custom authorized keys files to integrate gitolite authentication into our current model.

sshkey resource

The built in sshkey {} isn’t perfect for everyone. Most of the solutions on forge aren’t generic

http://forge.puppetlabs.com/hetzeneckerl/ssh_authorized_key

These are the core issues. But this code didn’t work for me.

!

• Legacy user management code –Old non-modular code –Have to fit in

!

• Almost – But not quite suitable for our requirements

http://forge.puppetlabs.com/nightfly/ssh_keys

Doing it myself as a concat {} seemed sane, especially given other people’s evidence..

!

• Legacy user management code –Old non-modular code –Have to fit in –Can’t change everything

!

• Built in sshkey {} – Not suitable for our requirements

!

• Hack what we want with define

Managing ssh keys - gitolite

So, we’re gonna use a bunch of defines and concat {}, easy?

!

gitolite::user { ‘tdoran’: key_source => $key_source; }

– Add in current user management code – $keysource = puppet:///files/users/tdoran/authorized_keys

/etc/authorized_keys.d/git.pub – Use concat – Multiple gitolite instances!

Inputs and outputs

- API for creating a key we drop into our current user management path - Eventual generated file, from a concat {} resource - Note we can have multiple gitolite instances on one box, that makes everything much harder.

So how does it work?

$864

First, get an array of key lines, with names prepended

Split the key into an array

$864

We split the file of keys into an array of lines. We remove comments, whitespace only lines etc We capture the entire key passed in.

Split the key into an array

$864

We interpolate the username in front of that key, space separating them

Split the key into an array

$864

End up with an array of strings which are username and key joined by a space We then abuse a define as a loop by calling it with the array of lines as $title

!

Add user to every instance – [“${username} ${restofkey}”]

– [“${instance} “]

– [“${instance} ${username} ${restofkey}”]

– Use this as $name in a define to iterate

– ssh/authorized_keys.d/${instance}.pub

Prepend instance name

We then have an array of instances, and an array of users/keys. Our array_prefix function makes the product of both lists. Complexity due to iterating over two lists $key_lines and $instances

!

Add user to every instance – [“${username} ${restofkey}”]

– [“${instance} “]

– [“${instance} ${username} ${restofkey}”]

– Use this as $name in a define to iterate

– ssh/authorized_keys.d/${instance}.pub

Prepend instance name

We then have an array of instances, and an array of users/keys. Our array_prefix function makes the product of both lists. Complexity due to iterating over two lists $key_lines and $instances

!

Add user to every instance – [“${username} ${restofkey}”]

– [“${instance} “]

– [“${instance} ${username} ${restofkey}”]

– Use this as $name in a define to iterate

– ssh/authorized_keys.d/${instance}.pub

Prepend instance name

We then have an array of instances, and an array of users/keys. Our array_prefix function makes the product of both lists. Complexity due to iterating over two lists $key_lines and $instances

!

Add user to every instance – [“${username} ${restofkey}”]

– [“${instance} “]

– [“${instance} ${username} ${restofkey}”]

– Use this as $name in a define to iterate

– ssh/authorized_keys.d/${instance}.pub

Prepend instance name

We then have an array of instances, and an array of users/keys. Our array_prefix function makes the product of both lists. Complexity due to iterating over two lists $key_lines and $instances

!

Add user to every instance – [“${username} ${restofkey}”]

– [“${instance} “]

– [“${instance} ${username} ${restofkey}”]

– Use this as $name in a define to iterate

– ssh/authorized_keys.d/${instance}.pub

This is gross

Prepend instance name

We then have an array of instances, and an array of users/keys. Our array_prefix function makes the product of both lists. Complexity due to iterating over two lists $key_lines and $instances

My original next code

$864

Stare in horror!

This was hideous. Whilst looping without define abuse is hard, there’s just no excuse for this.

Sins

$864

Hideous abuse of define - one $name for each $key_lines

LOL?

?We then split the line back up.

Sins

$864

Hideous abuse of variables

LOL??

Shift the user name off the array. Lol, who knew you could do that? Use the remaining data joined back up

Sins

$864

Hideous abuse of interpolation

LOL??

LOL??

So, here, I have a double quoted string containing erb, and I’m then using double quote (not erb) interpolation.. Niiice.

Sins

$864

Hideous abuse of quoting

LOL?

?

LOL?

?

LOL??

EVERY TYPE OF QUOTES!!!

Sins

$864

Hideous abuse of quoting

LOL?? LOL??

Sins

$864

Hideous abuse of bash

LOL??

ssh-keygen can’t be fed from stdin without disgusting trickery (modern bash specific)

Sins

$864

So complex I’m tempting the template!

LOL??

LOL??

And yes, it’s so complex I template the thing I’m interpolating into a template..

I am a bad person

$864

LOL?

I am a bad person

$864

LOL?

I am a bad person

$864

LOL?

Thankfully

I am a bad person

$864

LOL?

Someone stopped me ;)

!• Please?

– I’m a terrible person – Working now doesn’t mean you can maintain it

• Functions for reuse – Full power of ruby – Composable – There should be more libraries of functions

Don’t abuse inline_template()

Do what I’m saying, not what I do :)

More sane

Except each user can have multiple keys, and we want to prevent any key being reused by multiple users.

!• What functions do I even have?

– Hard to know what functions you have imported – stdlib, builtins, ….your $modulepath here…

• What do these even do? – Often need to read the code to determine – Lots of functions packaged with modules - don’t do

this!

• Millions of twisty little functions, all alike • Millions of twisty little functions, all different

– You know what this reminds me of?

Issues with functions

Everyone’s favorite language

If all you have is a hammer, everything looks like a nail

has a few functions

Unfortunately, there are no consistent naming conventions

arguably

And it’s hard to work out which function you actually want

a few too many..

The problem is, functions in a global namespace aren’t all that modular/composable. You can call functions from functions - but dependency hell

we’re almost half way through….

Shall I stop now? :) puppet problem is worse than this - what functions you have depends on what modules are in your codebase.

Get docs for project’s functions

I wrote a thing to help with this - this will document _your_ functions for you. gist at end of talk.

!

• /etc/groups on system !

• gitolite groups have a different format: @admins = tdoran joebloggs!!

• Use exec {… add mess here … }

gitolite group file generation

This is another great example of what not to do.

exec {} abuse

$864

Syntax highlighting won’t save you now - puny human!

This. DO NOT DO THIS. Please?

exec {} abuse

$864

Syntax highlighting won’t save you now - puny human!

exec {} abuse

$864

Syntax highlighting won’t save you now - puny human!

exec {} abuse

$864

Syntax highlighting won’t save you now - puny human!

Arggh, the backslashes!

This madness actually worked

$864

LOL?

This madness actually worked

$864

LOL?!

•exec {} abuse – Lets fit an entire shell script in the command => “…”

– Don’t do it! :)

– Drop a perl/python/ruby script (or two) instead, call them from the exec.

– Think about writing a type/provider

– Lots of examples of parsedfile available

Which would you rather have to debug?

$864

OR THIS!?!

$864

Which one’s going to stay debugged?

$864

OR THIS?!?

$864

!This would work fine.

Need multiple instances :(

Looping sanely

—parser future

—parser future

$864

—parser future

$864

!

• Can’t do this at Yelp (yet)

• Still using 2.7, because variable scope

!

”There should be more libraries of functions”

- Me, about 5m ago

– https://github.com/bobtfish/puppet-sshkey_utilities

– https://github.com/bobtfish/puppet-better_file

– https://gist.github.com/bobtfish/7403811

– Whole module to follow (eventually)

Hypocrisy avoidance

!

• Where was that code? – https://github.com/bobtfish/puppet-sshkey_utilities – https://github.com/bobtfish/puppet-better_file

• Did you say you were hiring? – YES

– Hamburg. – London. – San Francisco.

• Questions?

That’s all folks