Concurrency in the Go Programming Language

8/13/2019 Concurrency in the Go Programming Language

1/18

An overview of Go

by Dan Callahan

Let’s start at the beginning: What is Go?

Go is a “fast, statically typed, compiled language that feels like a dynamically typed,

interpreted language! "t is a systems language competiti#e $ith C, C%%, and &a#a, but

designed to deal gracefully $ith concurrency 'o $it:

“Go makes much more sense for the class of problems that C%% $as originally intended to

sol#e! ()ruce *ckel

Go $as released by Google in late +-, and .ersion / should land in the first fe$ months

of +/+

Why should you care?

0ou should care about Go for fi#e reasons:

/ "t has an impeccable pedigree

+ "t embodies an e1tremely pragmatic design philosophy

2 "t implements many inno#ati#e ideas

3 "t can already be used for real $ork

4 "t has an absolutely fantastic mascot

Go’s Pedigree

Go is being de#eloped by )ell Labs #eterans augmented by a handful of other brilliant

engineers 'he core team includes:

• Rob Pike 56'789, lan - from )ell Labs;

• Ken Thompson 5C, 6ni1, lan - from )ell Labs;

• Russ o! 5lan - from )ell Labs,


2/18

)y taking these lessons into account, Go hopes to offer the de#eloper8time efficiency of a

language like ython and the run8time efficiency of C

7or e1ample:

• *#en though Go is a systems language, it uses garbage collection• Go has been designed to compile extremely @uickly

• Go’s synta1 is clean and minimal

• Go offers first8class functions

• 7unctions in Go are closures

• 7unctions in Go can return multiple #alues

• 7unction definitions can ha#e named return parameters

• Go offers sane error handling, in contrast to many languagesA model of thro$ing and

catching e1ceptions

Let’s look at a fe$ of these more closely

Quick Compile Times

Go programs are structured in a manner that drastically limits the number of passes re@uired

to produce a binary By / G> dual8core laptop compiles pkg/math 5E+ files, 4 FLC; inunder / second:

$ time make --quietmake --quiet 0.89s user 0.08s system 97% cpu 0.991 total

Clean Syntax

)elo$ is a simple “hello, $orld! in Go

package main

import "fmt"

func main!

fmt.#rintln"ello 世界"!&

'he anatomy of this program should be self8e#ident: "t declares that it is a member of the

package main, imports any other packages it needs, and then defines a function called main,

$hich is e1ecuted $hen the compiled program is in#oked

"t’s $orth pointing out the use of 6nicode )y definition, Go source files are encoded in

6'789, $hich makes it easy to use 6nicode literals >a#e you e#er $anted to name a #ariable? Well, no$ you can

+


3/18

First Class Functions

"n Go, functions can be created or passed around like any other structure "n the code belo$,

"’#e defined t$o unary functions: 'ou(le and square *ach accepts a single int for input

and returns a single int

"’#e also defined apply, $hich accepts a t$o arguments: a function, and an integer "t then

calls the function and returns an integer

package main

import "fmt"

func 'ou(le) int! int return ) * )&

func square) int! int return ) + )&

func applyf funcint! int ) int! int return f)!&

func main! fmt.#rintlnapply'ou(le ,!! fmt.#rintlnapplysquare ,!!&

)ecause Go is strongly and statically typed, function declarations must e1plicitly name thetypes of their inputs and return #alues 'hus, func applyf funcint! int ) int! int

creates a ne$ function named apply $hich takes t$o inputs, a funcint! int called f and

an int called ) "t then returns an int

0ou can read funcint! int the same $ay: "t’s a function that accepts an integer and

returns an integer

With that information, the compiler is able to #erify that e#erything in the program is of the

e1pected type "t’s $orth noting that Go does not do implicit or automatic type coercion

Closures

H closure is Iust a function that can reference #ariables that $ould typically be considered

outside of its scope 'o illustrate, the code belo$ calculates the first fe$ 7ibonacci numbers

using a closure

package main

func fi(! func! int a ( 0 1 return func! int

a ( ( a*( return ( &

2


4/18

&

func main! f fi(! printlnf! f! f! f! f!!&

'hat might look a bit $eird 'o re#ie$, note that named function definitions ha#e three parts:

/ 'he func key$ord

+ 'he name of the function, its inputs, and their types, like square) int!

2 'he function’s return type, like int

=o func fi(! func! int abo#e is creating a function named fi( that doesn’t ha#e any

inputs, and $hich returns a func! int H func! int is, in turn, a function that doesn’t

ha#e any inputs, and $hich returns an integer

'hus, $hen $e in#oke fi(! $e get another function When $e in#oke that , $e get an

integer

)ecause functions in Go are closures, the function returned by fi(! has access to the #alues

of a and (, e#en though they $ere defined outside of the inner function itself

Go’s #nnova"ions

'he follo$ing items aren’t necessarily uni@ue to Go, but they are distinguishing features

• Go ships $ith its o$n pretty printer, gofmt• Go ships $ith an old8to8ne$ Go compiler, gofi1

• Go sports a brilliantly light$eight type system

• Go has per#asi#e, primiti#e support for concurrency

• Go makes it easy to defer e1ecution of a function until the end of a block

'he type system and concurrency features are $orthy of their o$n dedicated articles, so let’s

look at the other items

Gofmt, the Go Pretty-Printer

Go is some$here bet$een C and ython in the rigidity of its synta1 Whitespace is

insignificant and semicolons can be omitted, but the opening brace of a construct must be on

the same line as that construct

=till, Go maintains high degree of readability across authors thanks to gofmt, a command line

tool that pretty8prints Go source code in a single, consistent style

'ake the follo$ing #ersion of the 7ibonacci program from abo#e "t still compiles and runs

perfectly, but it has semicolons e#ery$here, horribly inconsistent indentation, and e#en

combines t$o statements on one line $ith a ( ( a*( return (

package main

3


5/18

func fi(! func! int a ( 0 1 return func! int a ( ( a*( return ( & &

func main! f fi(!printlnf! f! f! f! f!!&

ne pass through gofmt, and out pops:

package main

func fi(! func! int a ( 0 1 return func! int a ( ( a*( return ( &&

func main! f fi(! printlnf! f! f! f! f!!&

Gofix, the Old-to-Ne Go Compiler

Hs Go is still under rapid de#elopment, it’s not uncommon for components of the standard

library to change in back$ards8incompatible $ays


6/18

package main

import "url"

func main! url url.#arse"http//e)ample.com/search4qfoo"!

printlnurl.5uery!.6et"q"!!&

Jote that import "http" changed into import "url" and http.#arse23 $as changed

into url.#arse ne of the more subtle details is that my #ariable, url, $as automatically

renamed to url to a#oid clashing $ith the ne$ import statement >o$ cool is that?

"f you’re coming from ython, you can think of gofi) as being similar to to

!eferred "xecution

'ake the follo$ing program:

package main

func goo'(ye! println"6oo'(ye:"!&

func main! goo'(ye! println";our"! println"


7/18

println"


8/18

>eroku has also been using Go to build Dooer , a consistent, highly8a#ailable data store 'o

@uote Feith


9/18


10/18

13etBs go: D 0000Coffee is rea'y: D 000


11/18

i there, " lo#e cakeK!

in your bro$ser, since http.an'le;unc"/" han'ler! ensures that all re@uests get passed off to the han'ler function, $hich simply displays that message Digging in to the

source for the http package $ill re#eal that it takes each incoming re@uest, spins up a

goroutine to handle it, and then goes back to listening

'hus, a great use of goroutines $ithout needing to kno$ ho$ to communicate $ith them

$hile they run =till, there are situations $here you need to communicate $ith the goroutines,

as per the coffee N tea bre$ing code abo#e Luckily, Go pro#ides channels to do Iust that

//


12/18

Wha" is a channel?

H channel is a type8safe pipe for communicating bet$een different parts of your program

Let’s start $ith an impro#ed #ersion of the bre$ing program as an e1ample:

package main

import "time"

func ?s2ea'y>hat string secon's int@ ch chanJ- (ool! time.Aleepsecon's + 1e9! // nanosecon's println>hat "is rea'y:"! ch J- true&

func main! ch makechan (ool! println"3etBs go:"! go ?s2ea'y"Coffee" ch! go ?s2ea'y"orl'" K se' -e Bs/hello/goo'(ye/gB K >c -c 1@

/+


13/18

'he shell commands abo#e $ork like they do since echo Iust takes a string, slaps a Ln

character on the end, and spits it back out Bean$hile >c -c reads in data and counts the

number of characters 'hus, it counts /+ in the first instance 54 % 4 % a space % a ne$line;

=imilarly, the se' in#ocation takes its input and replaces any occurances of “hello! $ith

“goodbye!, lengthening the original string by +

What if $e $anted to simulate that sort of a pipeline in Go? 7irst, let’s define functions to

replace echo, se', and >c *ach one $ill take a channel for input, and a channel for output:

package main

import "rege)p" "strconF"!

func echoin chan string out chan string!

out J- J-in * "Ln"!&

func se'in chan string out chan string! re rege)p.MustCompile"hello"! out J- re.2eplaceIllAtringJ-in "goo'(ye"!&

func >cin chan string out chan string! out J- strconF.?toalenJ-in!!&

"n the abo#e functions, echo simply reads from its input channel, adds a ne$line to the end of

it $ith J-in * "Ln", and then sends the result to the out channel

'he se' function $orks similarly, first compiling a regular e1pression that matches hello,

and then calling that rege1’s 2eplaceIllAtring method 'he method is told to grab a string

from the in channel and use “goodbye! as the replacement for matches 'he result gets sent

to the out channel

Lastly, >c simply reads from the input channel 5J-in;, then gets the length of the input $ith

len, and lastly con#erts the integer length into a string $ith strconF.?toa before sending

the result on the out channel

'he main function is simple, but a bit #erbose

func main! ch1 makechan string! ch makechan string! ch makechan string! ch@ makechan string!

go echoch1 ch! go se'ch ch! go >cch ch@!

ch1 J- "hello >orl'" printlnJ-ch@!

/2


14/18

&

"t simply makes four channels, and hooks them up so that the output of echo is the input of

se', and the output of se' is the input of >c

*ach of those subfunctions get started in their o$n goroutines, and then sit and $ait for input'he string “hello $orld! gets sent on ch1, $hich flo$s through echo, then se', then >c and

out on ch@ 'he result, “/3!, gets printed:

$ /pipes11@

*1plicitly specifying the input and output channels is needlessly #erbose We ha#e to gi#e

each function an input channel, but if it made and returned its o$n output channel, then $e

could tri#ially chain together as many functions as $e $ant

Let’s do that:

package main

import "rege)p" "strconF"!

func echoin chan string! chan string out makechan string! go func! out J- J-in * "Ln"!

&! return out&

func se'in chan string! chan string out makechan string! re rege)p.MustCompile"hello"! go func! out J- re.2eplaceIllAtringJ-in "goo'(ye"! &! return out&

func >cin chan string! chan string out makechan string! go func! out J- strconF.?toalenJ-in!! &! return out&

'his might look comple1, but it’s really not too different from the pre#ious code 'he

signatures changed to Iust accepting a single chan string, each function starts $ith creating

an output channel, and each function ends by returning that channel "n the middle, the “real

$ork! gets $rapped in an anonymous function $hich gets in#oked in a goroutine 'his

means that $hen you call something like echo, it immediately returns its ne$ly8made outputchannel, and as a side effect, spins off a goroutine to process any data that gets sent on the in

/3


15/18

channel "t’s basically saying “kay, "’m ready and $aiting on in When " get something, "’ll

gi#e it back to you on this channel that "’m returning!

Jote that because functions in Go are closures, $e didn’t ha#e to e1plicitly pass in a

reference to the input channel, in, $hen $e defined an anonymous $orker function cse'echoin!!! inJ-"hello >orl'" printlnJ- out!&

We make an input channel, then $e get an output channel by composing our filters: out

>cse'echoin!!! Hgain, because the output of echoin! is a channel, $e can use it as

the input to se', and so on Hnd it $orks e1actly like you $ould e1pect:

$ ./pipes1@

7unctions that start goroutines and return channels are a some$hat uni@ue and interesting Go

idiom

)ow do channels and gorou"ines work "oge"her?Channels and goroutines $ork together to implement a much more sane model of

concurrency than traditional threads and shared memory 'he central idea is that, for

unbuffered channels, the act of reading and $riting across a channel forces t$o goroutines to,

at that moment, be synchronied ther$ise, things are free to e1ecute concurrently or in

parallel

'his paradigm is often e1pressed as:

Do not communicate by sharing memoryO instead, share memory by communicating (

*ffecti#e Go

7or a some$hat silly e1ample of channels and goroutines $orking together, check out this

prime number sie#e:

package main

func 6eneratech chanJ- int! for i i** ch J- i &&

func ;ilterin J-chan int out chanJ- int prime int!

/4


16/18

for i J-in if i%prime : 0 out J- i & &

&

func main! ch makechan int! go 6eneratech! for i 0 i J 10 i** prime J-ch printprime "Ln"! ch1 makechan int! go ;ilterch ch1 prime! ch ch1 &&

6enerate simply starts at + and sends e#er increasing numbers out on a gi#en channel 7irst

+, then 2, then 3, and so on

;ilter sits in bet$een t$o channels and silently drops any number that’s di#isible by its

prime input parameter ther$ise, it Iust passes the number along H ;ilterin out !,

for instance, $ould not let any multiples of + through it 'hus, $hen hooked up to the output

of 6enerate, first 2, then 4, then E, $ould pop out the other end

Can you see $here this is going? We $ant to build a pipeline that successi#ely filters each

prime, so that $ith each number popping out the end, $e sa#e it, and then add on a filter for

its o$n multiples:

G6enerateH -N

G6enerateH -N G;ilter!H -N

G6enerateH -N G;ilter!H -N G;ilter!H -N ,

and so on

>ence the loop in the main function, $hich successi#ely reads from the rightmost end of the

pipeline, prints that #alue, and then adds on a filter for it

$ ./primes,711117199

/


17/18

)ecause all of the $ork is happening in goroutines, as soon as any t$o adIacent goroutines

are ready, a ne$ #alue can begin tra#ersing through the pipeline 'he adIacent goroutines

synchronie upon e1changing data

hannels can also be mul"iple!ed

Hny number of goroutines can $rite to a single channel, and any number of goroutines can

read from a single channel Go e#en has a special select statement $hich, gi#en a list of

channels, reads from and e1ecutes a block for a random, $aiting channel 'his allo$s

channels to be easily de8multiple1ed

7or another e1ample, let’s use the random selection amongst $aiting channels to implement a

ridiculous random number generator:

package main

func main! ch1 makechan int! ch makechan int!

go func! for J-ch1 & &!

for select case ch1 J- 0 print0! case ch1 J- 1 print1! case ch J- print! & &&

*1ecuting this prints out a random series of /’s and ’s Why? Well, first $e set up t$o

channels for integers, and then kick off a goroutine that does nothing but drain one of those

channels

'hen $e loop infinitely, selecting one of the ready cases =ince the goroutine is al$ays

$aiting to consume any #alue sent on ch1, either of the t$o case statements that send on ch1

could run )y definition, select chooses one those at random, creating a stream of random

data

)ecause nothing is e#er $aiting to read from ch, the case that sends on ch is ne#er

e1ecuted

'hese features make it easy to implement things like @ueues, signaling, etc

/E


18/18

*roader Applica"ions

*#en if you ne#er plan on using Go, Iust learning about goroutines and channels can pro#ide

a rich mental model for designing concurrent systems "’m particularly fond of this @uote

describing the benefits of good notation:

)y relie#ing the brain of all unnecessary $ork, a good notation sets it free to concentrate on

more ad#anced problems, and, in effect, increases the mental po$er of the race (Hlfred

Jorth Whitehead, Hn "ntroduction to Bathematics, /-//

Hnd if you’d like to go straight to the notation’s source, goroutines and channels take their

inspiration from C H oare’s “Communicating =e@uential rocesses!, a formal language

for describing patterns of interaction in concurrent systems

Py"hon and oncurrency

Puite a fe$ proIects e1ist to make concurrent programming easier in ython "n particular:

• )oth yC= and ython8C= implement C= in ython, and are some$hat acti#e

'he ython8C= proIect is currently planning on merging into yC=

• 'he ))C’s Famaelia is a frame$ork for doing *rlang8style concurrency in ython

• =tackless ython is a fork of Cython $ith added support for “tasklets! and channels

+"her Resources

6sing C= is 'ony >oare’s book on C=, a#ailable as a free D7

'he Go $ebsite at golangorg has absolutely fantastic resources, including great docs, a

bro$ser8based “ playground! N pastebin, an interacti#e tour of the language, a great blog, and

much more

redi"s

Documents

Concurrency in the Go Programming Language