Functional Programming in Python

Reuven M. Lerner, PhD • reuven@lerner.co.il October 22nd, 2014

Who am I?• Programmer, consultant, developer, trainer

• Long-time Python (+Ruby) user

• Linux Journal columnist

• Author Practice Makes Python!

• PhD in Learning Sciences from Northwestern

• Read (lots) more at http://lerner.co.il/

2

Functional programming• A different way of thinking about programming

• A different way of writing programs

• A different way of approaching problems

• Often provides new, clean, elegant solutions…

• … but again, it requires that you think about things very differently.

4

5

Function

Input

Output

… and if you can avoid internal state, even better!

No external state changes in this function…

Why work this way?• Simpler functions — easier to write and maintain

• Easer to test functions, and predict their results

• Easier to combine functions in new, different ways

• You can split work across threads, processors, or even machines without any ill effects

• Certain things can be expressed easily, elegantly

6

The FP perspective

7

f1 f2 f3 f4 f5Input Output

OO or functional?

• Python supports both styles

• It’s common to mix them in one program

• My current approach: Use objects in general, but functional techniques inside of methods

8

Sequences and iterables• There are three built-in sequences in Python:

strings, tuples, and lists

• Many other objects are "iterable," meaning that you can stick them into a "for" loop and get one item at a time

• Examples: files and dicts

• This discussion mainly has to do with iterables

9

Where are iterables?• Strings, lists, and tuples (obviously)

• Dicts, sets, and files

• Complex data structures: Lists of lists, lists of dicts, dicts of dicts (and others)

• Results from database queries

• Information from a network feed

• Many, many objects return iterables

10

Transformation• You often want to turn one iterable into another

• For example, you might want to turn the list

[0,1,2,3,4] # range(5)

• into the list

[0,1,4,9,16]

• We can transform the first list into the other by applying a Python function.

11

12

def square(x): return x*x

Each list item

New list item

The usual solution>>>> input = range(5)

>>>> def square(x):

return x*x

>>> output = [ ]

>>> for x in input:

output.append(square(x))

>>> output

[0, 1, 4, 9, 16]

13

What's wrong with this?• Nothing is wrong.

• But functional programming looks at this, and says:

• Why create a new variable ("output")?

• Why are you building it, one step at a time?

• Why are you modifying the state of "output"?

• Why do it in such a clumsy way?

14

The elegant way

"I want to apply my function, square, to each and every element of the input list, and get a list back."

15

16

def square(x): return x*x

Input list

Output list

List comprehensions

• In Python, we do this with a "list comprehension":

[square(x) for x in range(5)]

• Or if you prefer:

[x*x for x in range(5)]

17

Ints to strings• I can't say

', '.join(range(5))

• because str.join's input must contain strings.

• Solution:

', '.join([str(x) for x in range(5)])

18

Lowercase all words• I can transform a sentence into all lowercase:

words = 'This is a sentence for my Python class'.split()

[word.lower() for word in words]

• Or even:

' '.join([word.lower() for word in words])

19

Filenames to files• I can get a list of filenames from os.listdir:

os.listdir('/etc')

• I can get a file object for each of these:

[open('/etc/' + filename)

for filename in os.listdir('/etc')]

20

File contents

• If I want to get the names of users on my Unix system, I can say

[ line.split(":")[0]

for line in open('/etc/passwd') ]

21

Filtering

• If I want to get the names of users on my Unix system, I can say

[ line.split(":")[0]

for line in open('/etc/passwd')

if line[0] != '#' ]

22

Pig Latin!def plword(word):

vowels = 'aeiou'

if word[0] in vowels:

return word + 'way'

else:

return word[1:] + word[0] + 'ay'

23

Translation

' '.join([plword(word)

for word in open('column-215') ])

24

Bottom line• Comprehensions are Python's answer to the

traditional functions "map" and "filter".

• Once you start to think in terms of "mapping" one sequence to another, you'll see this pattern everywhere.

• You'll reduce the number of assignments you make. Your programs will be shorter and easier to understand. And you'll be able to stack these manipulations, without worrying about side effects.

25

Functions

• Where are the functions in functional programming?

• Let's write some functions! Let's use some functions!

26

Calling functions• We call functions in Python with ()

• No parentheses — no function call! x = len('abc')

type(x)

int

x = len

type(x)

builtin_function_or_method

27

Parentheses• In Python, something that can be executed with

parentheses is known as "callable".

• 'a'() will result in an error, that a string isn't "callable."

• Normally, we say that functions and classes are callable. How do we know? They have a __call__ attribute!

28

Dispatch table

def foo(): return "foo"

def bar(): return "bar"

d = {'foo':foo, 'bar':bar}

29

Sorting

• We know that we can use list.sort to sort a list:

words= 'This is a sentence for my online Python class'.split()

words.sort()

30

Changing sort's behavior• The optional "key" parameter lets you pass a

function that changes sort's behavior

• Define a function, and pass it to "key".

• Or use an existing function:

words.sort(key=str.lower)

words.sort(key=len)

31

The real lesson: You can do it, too!

• Take a function as a parameter

• Invoke the function within your function

• Now you have a more generic function, whose behavior can be modified!

32

33

Function

Input 1

Output

Use input 2

input 2

Example

• Let's implement transform_values!

>>> d = {'a':1, 'b':2, 'c':3}

>>> transform_values(lambda x: x*x, d)

{'a': 1, 'b': 4, 'c': 9}

34

Summary• Functions are data, too! Treating them as such

makes your programs more flexible, generic, and easy to understand

• Python's comprehensions are a modern version of classic "map" and "filter" functions.

• Composing functions isn't the only solution, but it's a really flexible and useful one

• It becomes easier with practice!

35

Questions? Comments? And remember…

• Early-bird discount on my book!

• http://lerner.co.il/practice-makes-python

• Use the WEBINAR coupon code for 10% off!

• Courses are at http://lerner.co.il/courses

• Contact me: reuven@lerner.co.il

• Follow me: @reuvenmlerner

36