Building  your  first  Go  App

1Go mascot designed by Renée French and copyrighted under the Creative Commons Attribution 3.0 license.

@spf13

• Author of Hugo, Cobra, Viper & More

• Chief Developer Advocate for MongoDB

•Gopher 2

http://spf13.com

My  Go  Story

• I write a medium size blog • Frustrated with Wordpress performance & maintenance

• Existing Static Site Generators had complicated installations & were very slow

• Began writing a S.S.G. in Go called Hugo 1.5 years ago4

hugo.spf13.com

Hugo• Hugo is 2nd fastest growing Static Site Generator (40-50 * a week on github)

• Hugo is one of the fastest Static Site Generators. 1000x faster than Jekyll

• Easiest Installation of any SSG. Download and run • 2nd most contributors of any Go project (Docker 1st)6

http://github.com/spf13/hugo

Plan

• Introduce Go Language • Introduce Tools & Libraries • Build our application • Tell the world how awesome we are 8

Ground  Rules

• Workshops are hard, everyone works at a different pace

• We will move on when about 50% are ready

• Slides are online, feel free to work ahead or catch up

9

http://spf13.com/

presentation/first-go-app/

10

Installing  Go

• You should already have Go installed • If you don’t, do it NOW

• Installation Guide 11

Installing  MongoDB

• You should already have MongoDB installed • If you don’t, do it NOW

• Installation Guide 12

Git  &  Mercurial

• Lastly we need Git & Mercurial • http://git-scm.com/downloads • http://mercurial.selenic.com/wiki/Download

13

Introduction    to  Go

14

Why  Another  Language?

• Software is slow • Sofware is hard to write • Software doesn’t scale well

15

Go  is  Fast

• Go execution speed is close to C • Go compile time rivals dynamic interpretation

16

Go  is  Friendly• Feels like a dynamic language in many ways • Very small core language, easy to remember all of it

• Single binary installation, no dependencies • Extensive Tooling & StdLib

17

Go  is  Concurrent• Concurrency is part of the language • Any function can become a goroutine • Goroutines run concurrently, communicate through channels

• Select waits for communication on any of a set of channels

18

Go’s  Inspiration• C: statement and expression syntax • Pascal: declaration syntax • Modula 2, Oberon 2: packages • CSP, Occam, Newsqueak, Limbo, Alef: concurrency • BCPL: the semicolon rule • Smalltalk: methods • Newsqueak: <-, := • APL: iota 19

… AND lessons good and bad from all those plus: C++, C#, Java, JavaScript, LISP, Python, Scala, ...

— txxxxd

“Most  of  the  appeal  for  me  is  not  the  

features  that  Go  has,  but  rather  the  

features  that  have  been  intentionally  left  

out.”20

— Rob Pike

“Why  would  you  have  a  language  that  is  not  theoretically  

exciting?  Because  it’s  very  useful.”

21

package main !

import "fmt" !

func main() { fmt.Println(“Hello, 世界") }

Hello  World

22

Package

• Not classes or objects • No subpackages • Fundamental building block in Go • Visibility is package-level, not type-level • Programs run “main” package 23

package main !import "fmt" !func main() { fmt.Println("Hello, 世界") }

import

• Provides access to a package • Package consists of types, functions, etc… • No circular dependencies

24

package main !import "fmt" !func main() { fmt.Println("Hello, 世界") }

“fmt"• Package path is just a string

• Standard library sits at the root (no path) • Core language is very small • Most functionality is in Stdlibs 25

package main !import "fmt" !func main() { fmt.Println("Hello, 世界") }

func

• Declaring a function • Same syntax for methods, anonymous functions & closures

• main.main() is the function run when the program is executed

26

package main !import "fmt" !func main() { fmt.Println("Hello, 世界") }

Main

• No void • No return value • No function args (command line is in os package)

27

package main !import "fmt" !func main() { fmt.Println("Hello, 世界") }

{ }• Braces not spaces • Feels like C ( & most languages ) • Newline must be after brace

(semicolons are inserted during compiliation)

28

package main !import "fmt" !func main() { fmt.Println("Hello, 世界") }

fmt

• Accessing the package imported above

• Everything visible in the package will be accessible through the name (or alias if provided)

29

package main !import "fmt" !func main() { fmt.Println("Hello, 世界") }

 Println

• Println, not println • Println, not print_ln • Capital for export • Variadic function using reflection

30

package main !import "fmt" !func main() { fmt.Println("Hello, 世界") }

“Hello,  世界”

• UTF-8 input source • Strings are immutable • Strings are UTF-8 encoded • String is a built-in type

31

package main !import "fmt" !func main() { fmt.Println("Hello, 世界") }

Rob  Pike  did  it  better

http://confreaks.com/videos/3419-gophercon2014-opening-day-keynote

32

Tons  more…

• types • constants • methods • interfaces • pointers

• control flow • conditionals • tools • packages • concurrency 33

Effective  Go

34

Lets  write  some  code  

!

Introduce  the    go  tools

35

package main !

import "fmt" !

func main() { fmt.Println("Hello,World") }

Hello  World

36

hello.go

❯ go run hello.goHello,World

Go  Run

37

❯ go build hello.go !

❯ls -lh 1.7M Jun 17 22:44 hello 72B Jun 17 22:43 hello.go !

❯ ./helloHello, World

Go  Build

38

Go  Fmt

• Automatically formats Go source code • Ends stylistic debates • Integration with editors (vim, emacs, others) • Can also refactor code (see http://spf13.com/post/go-fmt )

39

go  Test  

• Enables easy testing of application • Integrates with the testing package • Supports benchmark, functional & unit style testing

• Combine with ‘looper’ to have realtime feedback 40

package main !

import "testing" !

func TestOne(t *testing.T) { one := false if !one { t.Errorf(“Test Failed”) } }

Hello  Test

41

hello_test.go

❯ go test ./... --- FAIL: TestOne (0.00 seconds) hello_test.go:8: Test failed FAIL FAIL _/Code/firstApp 0.012s !

!

❯ go test ./... ok _/Code/firstApp 0.012s

Go  Test

42

Go  Planet

43The planet logo is based on the Go mascot designed by Renée French and copyrighted under the Creative Commons Attribution 3.0 license.

Why  Planet?

• CLI application • Web application

• Good introduction

• Right sized • Database (MongoDB)

• Concurrency 45

Steps

0. Env Setup 1. Commands 2. Configuration 3. Working with Feeds

4. DB as Storage 5-6. Web Server 7-8. DB -> Templates 9. & 10. Add Polish

46

Step  0  

Setting  up  our  environment

47

Go  PAth  !

$GOPATH48

Go  path

• The GOPATH environment variable specifies the location of your workspace (and dependencies)

• It must not be the same path as your Go installation

49

❯ mkdir $HOME/go!

❯ export GOPATH=$HOME/go

Setting  up

50

export GOPATH=$USER/go export GOROOT=`go env GOROOT` PATH=$PATH:$GOPATH/bin

Linux  &  Mac

51

Add this to your ~/.bashrc (or equivalent)

Windows• Control panel > System

52

Go  PAth

• /home/user/gocode/ ($GOPATH) • src/ (put your source code here)

• bin/ (binaries installed here)

• pkg/ (installed package objects)53

Creating  our  project

54

Project  Dir

• Make a directory inside of $GOPATH/src • I like mine to be in $GOPATH/src/github.com/spf13/PROJECTNAME

• It can not be a symlink • We will call this our working directory or $wd

55

package main !

import "fmt" !

func main() { fmt.Println(“My Project") }

main.go

56

❯ go run main.goMy Project

Run  it

57

58

I’m on the #gopath with @spf13 at #OSCONhttp://j.mp/onGoPath

Tweet  Break

Step  1  

Creating  the  Command(s)

59

Defining  our  first  command

60

Structs

• Short for structure • Objectish … blend of data & methods, but no inheritance

• Really cheap. Struct{} is free (0 bytes).61

Functions

• Can have multiple return values • No overloading • No optional parameters (but variadic)

62

First  class  Functions

• Function literals are anonymous functions in Go • Can assign to a variable or field • Can use as function parameters or return values

• Can be created as a goroutine 63

pointers• Function calls copy arguments into the function • Pointers reference a location in memory where a value is stored rather than the value itself

• In Go a pointer is represented using the * • The & operator gives us the address of a variable • Go automatically dereferences pointers when using “.” 64

cobra

• A CLI Commander • Easy management of commands & flags

• Used by bitbucket, openshift & lots more

65http://fav.me/d5gkby7

go get -u github.com/spf13/cobra

Getting  Cobra

66

❯ mkdir $wd/commands

Commands  Dir

67

// Copyright © 2014 Steve Francia <spf@spf13.com>. // // Use of this source code is governed by an Apache2 // license that can be found in the LICENSE file. !package commands !import ( "fmt" "os" ! "github.com/spf13/cobra" )

planet.go  (1)

68

Path  =  Urlgo get uses it to install

69

var RootCmd = &cobra.Command{ Use: "dagobah", Short: `Dagobah is an awesome planet style RSS aggregator`, Long: `Dagobah provides planet style RSS aggregation. It is inspired by python planet. It has a simple YAML configuration and provides it's own webserver.`, Run: func(cmd *cobra.Command, args []string) { fmt.Println("Dagobah runs") }, }

planet.go  (2)

70

executing  cobra’s  command

71

Variable  Assignment

• ‘:=‘ is the short variable declaration operatorDeclares, creates & assigns simultaneously while infering type

• Assignment via ‘=‘ requires prior declaration • var foo int = 10 is the same as foo := 10 72

Error  Handling• Errors are not exceptional. Should be part of the language • Typically one of the return values • No exceptions in Go fd, err := os.Open("test.go") if err != nil { log.Fatal(err) } 73

(  Don’t)  Panic• Should rarely be used if ever • Performs a full stack trace dump for end users • Use only in the most dire circumstances (like package can’t find FileSystem)

• Can use ‘recover’, but only inside deferred functions74

func Execute() { err := RootCmd.Execute() if err != nil { fmt.Println(err) os.Exit(-1) } }

planet.go  (3)

75

// Copyright © 2014 Steve Francia <spf@spf13.com>. // // Use of this source code is governed by an Apache2 // license that can be found in the LICENSE file. !package main !import "github.com/spf13/firstGoApp-Planet/commands" !func main() { commands.Execute() }

main.go

76

Make sure to use your path

❯ go run main.goDagobah runs !

❯ go run main.go help

Running  it

77

Getting  Help

• https://godoc.org/github.com/spf13/cobra • https://github.com/spf13/firstGoApp-Planet/tree/step1

78

Creating #CobraCommands with @spf13 at #OSCON http://j.mp/cobracmd

Tweet  Break

Step  2  

Easy  Configuration80

Handing  Config  files

81

Package  Identifiers

• Export with Capital name • No automatic getters & setters • var field … func Field() … func SetField() are appropriate names

82

init()

• Each source file can have one (or multiple) • Niladic function to setup state (function without return value)

• Called after all variable declarations in the package

83

Scoping

• Very granular scoping. • Variables declared inside { } only survive in that cotrol structure (for, if, etc)

• The := trap if := is used to declare x and x already exists in an outer scope a new variable x will be created (shadowing)

84

package main !import ( "errors" "fmt" ) !var Foo int !func Zero() (int, error) { return 0, errors.New("yo") } !!

func main() { ! Foo = 10 ! fmt.Println("outer:", Foo) // 10 ! if Foo, err := Zero(); err != nil { fmt.Println("inner:", Foo) // 0 } ! fmt.Println("outer again:", Foo) // still 10 }

:=  Trap

85http://play.golang.org/p/i4L9Ao1P65

Viper

• A configuration manager • Easy loading of config files • Registry for application settings • Designed to work well with (or without) Cobra

86

go get -u github.com/spf13/viper

Getting  Viper

87

// Copyright © 2014 Steve Francia <spf@spf13.com>. // // Use of this source code is governed by an Apache2 // license that can be found in the LICENSE file. !package commands !import ( "fmt" "os" ! "github.com/spf13/cobra" "github.com/spf13/viper" )

planet.go  (1)

88

var CfgFile string !func init() { cobra.OnInitialize(initConfig) RootCmd.PersistentFlags().StringVar(&CfgFile, "config", "", "config file (default is $HOME/dagobah/config.yaml)") } !func initConfig() { if CfgFile != "" { viper.SetConfigFile(CfgFile) } viper.SetConfigName("config") viper.AddConfigPath("/etc/dagobah/") viper.AddConfigPath("$HOME/.dagobah/") viper.ReadInConfig() }

planet.go  (2)

89

Creating  our  config  file

90

appname: "Dagobah" feeds: - "http://spf13.com/index.xml" - "http://dave.cheney.net/feed" - "http://www.goinggo.net/feeds/posts/default" - "http://blog.labix.org/feed" - "http://blog.golang.org/feed.atom" !

$HOME/.dagobah/config.yaml

91

trying  it  out

92

var RootCmd = &cobra.Command{ Use: "...", Short: `...`, Long: `...`, Run: rootRun, } !func rootRun(cmd *cobra.Command, args []string) { fmt.Println(viper.Get("feeds")) fmt.Println(viper.GetString("appname")) }

planet.go  (3)

93

❯ go run main.go[http://spf13.com/index.xml http://dave.cheney.net/feed http://www.goinggo.net/feeds/posts/default http://blog.labix.org/feed http://blog.golang.org/feed.atom]Dagobah

Running  it

94

I got bit by the #goviper at #OSCON http://j.mp/go-viper

Tweet  Break

https://github.com/spf13/

firstGoApp-Planet/tree/

step296

Step  3  

Working  with  Feeds97

Finding  Libraries

98

Go-Search.org

99

Go  wiki

100

go-pkg-rss

• Fetch Rss and Atom feeds from the internet • Supports RSS .92, 1.0, 2.0 & Atom 1.0 • Respects TTL, SkipDays, CacheTimeout, etc in the feeds

101

go get -u github.com/jteeuwen/go-pkg-rss

Getting  Go-PKG-RSS

102

boilerplate  stuff

103

import ( "time" ... ) !func addCommands() { RootCmd.AddCommand(fetchCmd) } !func Execute() { addCommands() ... }

planet.go

104

// Copyright © 2014 Steve Francia <spf@spf13.com>. // // Use of this source code is governed by an Apache2 // license that can be found in the LICENSE file. !package commands !import ( "fmt" "os" "time" ! rss "github.com/jteeuwen/go-pkg-rss" "github.com/spf13/cobra" "github.com/spf13/viper" )

Fetch.Go  (1)

105

Creating  our  own  type

106

Named  Types

• Can be any known type (struct, string, int, slice, a new type you’ve declared, etc)

• Can have methods declared on it • Not an alias. Explicit type.

107

Slices  &  arrays

• Array is fixed in length • Slice can be viewed as a dynamic array (can grow) • Slice is a view of an array • Slice is the core data structure in Go for lists of data

108

type Config struct { Feeds []string Port int }

Fetch.Go  (2)

109

Defining  the  fetch  

command110

New(    )

• new(T) Does 2 things at once: • Allocates new zeroed value of type T • Returns a pointer to the new value ( *T) • Used for all non-reference types

111

Make(  )

• Creates slices, maps, and channels only • These types are special “reference” types • Makes a initialized (not zeroed) value of type T • Does not return a pointer

112

Composite  Literals

• An expression that creates a new value each time it is evaluated

• We’ve already used these quite a bit

• Can also be create values of arrays, slices, and maps • Pseudo Constructor • eg File{fd: fd, name: name} 113

Constructors

• Not built into Go • Use Composite Literals when possible • If initialization is needed use a factory • Convention is to use New___() • … or New() when only one type is exported in the package

114

var fetchCmd = &cobra.Command{ Use: "fetch", Short: "Fetch feeds", Long: `Dagobah will fetch all feeds listed in the config file.`, Run: fetchRun} !func init() { fetchCmd.Flags().Int("rsstimeout", 5, "Timeout (in min) for RSS retrival") viper.BindPFlag("rsstimeout", fetchCmd.Flags().Lookup("rsstimeout")) }

Fetch.go  (3)

115

Fetching  in  a  goroutine

116

GoRoutines

• A function executing concurrently with other goroutines in the same address space

• Lightweight • Not threads, coroutines, or processes

117

Channels• Channels are lightweight • Synchronize goroutines by communication rather than locking shared memory

• Channel <- Sender —- Writes block when channel is full

• <- Channel —- Reads blocks waiting for something to be read • Great talk on channels given at GopherConhttp://confreaks.com/videos/3422-gophercon2014-a-channel-compendium

118

Marshaling  into

• We can’t return a T (no generics)

• We can reflect on any value of any type

• Marshal accepts the address of the value so it can manipulate the value directly

119

func fetchRun(cmd *cobra.Command, args []string) { Fetcher() !

sigChan := make(chan os.Signal, 1) signal.Notify(sigChan, os.Interrupt) <-sigChan }

Fetch.go  (4)

120

func Fetcher() { var config Config if err := viper.Marshal(&config); err != nil { fmt.Println(err) } !

for _, feed := range config.Feeds { go PollFeed(feed.Url) } }

Fetch.go  (5)

121

Building  our  fetcher

122

Learning  about  go-pkg-rss

• How do we learn how to use the package?123

Learning  about  go-pkg-rss

• github.com/jteeuwen/go-pkg-rss • Readme is pretty lacking • godoc.org/github.com/jteeuwen/go-pkg-rss

124

godoc.org/github.com/jteeuwen/go-pkg-rss125

Go  Search

126

For

• For, Foreach & While are all spelled ‘for’ in Go for init; condition; post { } // Like a C for

for condition { } // Like a C while

for { } // Like a C for(;;)

127

func PollFeed(uri string) { timeout := viper.GetInt("RSSTimeout") if timeout < 1 { timeout = 1 } feed := rss.New(timeout, true, chanHandler, itemHandler) ! for { if err := feed.Fetch(uri, nil); err != nil { fmt.Fprintf(os.Stderr, "[e] %s: %s", uri, err) return } ! fmt.Printf("Sleeping for %d seconds on %s\n", feed.SecondsTillUpdate(), uri) time.Sleep(time.Duration(feed.SecondsTillUpdate() * 1e9)) } }

Fetch.GO  (6)

128

func chanHandler(feed *rss.Feed, newchannels []*rss.Channel) { fmt.Printf("%d new channel(s) in %s\n", len(newchannels), feed.Url) } !func itemHandler(feed *rss.Feed, ch *rss.Channel, newitems []*rss.Item) { fmt.Printf("%d new item(s) in %s\n", len(newitems), feed.Url) }

Fetch.go  (7)

129

❯ go run dagobah.go fetch --rsstimeout=1 15 new item(s) in http://spf13.com/index.xml 1 new channel(s) in http://spf13.com/index.xml Sleeping for 300 seconds on http://spf13.com/index.xml 25 new item(s) in http://www.goinggo.net/feeds/posts/default 1 new channel(s) in http://www.goinggo.net/feeds/posts/default Sleeping for 300 seconds on http://www.goinggo.net/feeds/posts/default 10 new item(s) in http://dave.cheney.net/feed 1 new channel(s) in http://dave.cheney.net/feed Sleeping for 299 seconds on http://dave.cheney.net/feed

Running  it

130

Tweet  Break

I wrote my first goroutine at #OSCON with @spf13

https://github.com/spf13/

firstGoApp-Planet/tree/

step3132

Step  4  

Storing  in   the  Database

133

Pretty

• github.com/kr/pretty • pretty.Println(X,Y,Z)

134

MongoDB

• Open Source • Document Database • Document == Struct or Document == Map • Works seamlessly with Go (and other languages)

135

MongoDB

•Fast & Scalable • Used by Disney, IBM, Metlife, eBay, Forbes, Craigslist, Cisco, Stripe

136

Mgo• Developed by the open source community • Heavily used throughout Go community • "mgo is the best database driver I've ever used." — Patrick Crosby, Founder of StatHat

• "mgo and Go are a pair made in heaven. — Brian Ketelsen, Author of GopherTimes.com

137

go get -u gopkg.in/mgo.v2

Getting  mgo

138

boilerplatestuff

139

// Copyright © 2014 Steve Francia <spf@spf13.com>. // // Use of this source code is governed by an Apache2 // license that can be found in the LICENSE file. !package commands !import ( "fmt" "os" ! "github.com/spf13/viper" "gopkg.in/mgo.v2" ) !var mongodbSession *mgo.Session !func init() { RootCmd.PersistentFlags().String("mongodb_uri", "localhost", "host where mongoDB is") viper.BindPFlag("mongodb_uri", RootCmd.PersistentFlags().Lookup(“mongodb_uri")) CreateUniqueIndexes() }

MongoDB.go  (1)

140

Setting  up  a  DB  session

141

func DBSession() *mgo.Session { if mongodbSession == nil { uri := os.Getenv("MONGODB_URI") if uri == "" { uri = viper.GetString("mongodb_uri") ! if uri == "" { log.Fatalln("No connection uri for MongoDB provided") } } ! var err error mongodbSession, err = mgo.Dial(uri) if mongodbSession == nil || err != nil { log.Fatalf("Can't connect to mongo, go error %v\n", err) } ! mongodbSession.SetSafe(&mgo.Safe{}) } return mongodbSession }

MongoDB.go  (2)

142

setting  up  some  

shortcuts143

!func DB() *mgo.Database { return DBSession().DB(viper.GetString("dbname")) } !func Items() *mgo.Collection { return DB().C("items") } !func Channels() *mgo.Collection { return DB().C("channels") }

MongoDB.go  (3)

144

Setting  up  indexes

145

func CreateUniqueIndexes() { idx := mgo.Index{ Key: []string{"key"}, Unique: true, DropDups: true, Background: true, Sparse: true, } ! if err := Items().EnsureIndex(idx); err != nil { fmt.Println(err) } ! if err := Channels().EnsureIndex(idx); err != nil { fmt.Println(err) } }

MongoDB.go  (4)

146

Preparing  the  data

147

Data  Prep

• Feeds are dirty • Go doesn’t allow “patching” of structs (not how structs work)

• MongoDB works with native types & named types

148

Tags  (Annotations)

• String literal following a field declaration • Provides additional information during reflection

• Used by JSON, BSON, YAML, etc 149

for  x,y  :=  range

• Provides a way to iterate over an array, slice, string, map, or channel.

• Like Foreach or Each in other languages • x is key, y is value… can ignore by setting to _

150

Append

• Append is really awesome, feels like push, but does a lot more.

• Append will grow a slice (both length & capacity)

• … and copy the underlying array if needed 151

type Itm struct { Date time.Time Key string ChannelKey string Title string FullContent string Links []*rss.Link Description string Author rss.Author Categories []*rss.Category Comments string Enclosures []*rss.Enclosure Guid *string `bson:",omitempty"` Source *rss.Source PubDate string Id string `bson:",omitempty"` Generator *rss.Generator Contributors []string Content *rss.Content Extensions map[string]map[string][]rss.Extension }

fetch.go

152

func itmify(o *rss.Item, ch *rss.Channel) Itm { var x Itm x.Title = o.Title x.Links = o.Links x.ChannelKey = ch.Key() x.Description = o.Description x.Author = o.Author x.Categories = o.Categories x.Comments = o.Comments x.Enclosures = o.Enclosures x.Guid = o.Guid x.PubDate = o.PubDate x.Id = o.Id x.Key = o.Key() x.Generator = o.Generator x.Contributors = o.Contributors x.Content = o.Content x.Extensions = o.Extensions x.Date, _ = o.ParsedPubDate() ! if o.Content != nil && o.Content.Text != "" { x.FullContent = o.Content.Text } else { x.FullContent = o.Description } ! return x }

fetch.go

153

type Chnl struct { Key string Title string Links []rss.Link Description string Language string Copyright string ManagingEditor string WebMaster string PubDate string LastBuildDate string Docs string Categories []*rss.Category Generator rss.Generator TTL int Rating string SkipHours []int SkipDays []int Image rss.Image ItemKeys []string

Cloud rss.Cloud TextInput rss.Input Extensions map[string]map[string][]rss.Extension Id string Rights string Author rss.Author SubTitle rss.SubTitle }

fetch.go

154

func chnlify(o *rss.Channel) Chnl { var x Chnl x.Key = o.Key() x.Title = o.Title x.Links = o.Links x.Description = o.Description x.Language = o.Language x.Copyright = o.Copyright x.ManagingEditor = o.ManagingEditor x.WebMaster = o.WebMaster x.PubDate = o.PubDate x.LastBuildDate = o.LastBuildDate x.Docs = o.Docs x.Categories = o.Categories x.Generator = o.Generator x.TTL = o.TTL

x.Rating = o.Rating x.SkipHours = o.SkipHours x.SkipDays = o.SkipDays x.Image = o.Image x.Cloud = o.Cloud x.TextInput = o.TextInput x.Extensions = o.Extensions x.Id = o.Id x.Rights = o.Rights x.Author = o.Author x.SubTitle = o.SubTitle !... ! return x }

fetch.go

155

... !

!

var keys []string for _, y := range o.Items { keys = append(keys, y.Key()) } x.ItemKeys = keys !

...

fetch.go

156

Storing  Feeds  &  Items

157

The  Blank  Identifier  ‘_’

• Used to ignore return values from a function

• Used with imports as an alias • Allows import of unused packages so init

functions can be executed158

func chanHandler(feed *rss.Feed, newchannels []*rss.Channel) { fmt.Printf("%d new channel(s) in %s\n", len(newchannels), feed.Url) for _, ch := range newchannels { chnl := chnlify(ch) if err := Channels().Insert(chnl); err != nil { if !strings.Contains(err.Error(), "E11000") { fmt.Printf("Database error. Err: %v", err) } } } }

fetch.go

159

func itemHandler(feed *rss.Feed, ch *rss.Channel, newitems []*rss.Item) { fmt.Printf("%d new item(s) in %s\n", len(newitems), feed.Url) for _, item := range newitems { itm := itmify(item, ch) if err := Items().Insert(itm); err != nil { if !strings.Contains(err.Error(), "E11000") { fmt.Printf("Database error. Err: %v", err) } } } }

fetch.go

160

Running  MongoDB

161

❯ mongodMongoDB starting : pid=51054 port=27017 dbpath=/data/db

Running  MongoDB

162

❯ mongo

Mongo  Shell

163

❯ go run dagobah.go fetch --rsstimeout=1 15 new item(s) in http://spf13.com/index.xml 1 new channel(s) in http://spf13.com/index.xml Sleeping for 300 seconds on http://spf13.com/index.xml 25 new item(s) in http://www.goinggo.net/feeds/posts/default 1 new channel(s) in http://www.goinggo.net/feeds/posts/default Sleeping for 300 seconds on http://www.goinggo.net/feeds/posts/default 10 new item(s) in http://dave.cheney.net/feed 1 new channel(s) in http://dave.cheney.net/feed Sleeping for 299 seconds on http://dave.cheney.net/feed

Running  planet

164

> use dagobah > db.items.findOne() { "_id" : ObjectId("53b4c08bddbc460a933cf3ed"), "date" : ISODate("2014-07-01T00:00:00Z"), "key" : "http://spf13.com/post/go-pointers-vs-references", "channelkey" : "Recent Content on Hacking Management", "title" : "Pointers vs References", "links" : [ { "href" : "http://spf13.com/post/go-pointers-vs-references", "rel" : "", "type" : "", "hreflang" : ""

Looking  at  the  Data

165

_ID

• MongoDB’s primary key • Default is to use an “object_id” • Object_id guaranteed to be unique across your entire cluster

• MongoDB will create it for you on insert 166

Tweet  Break

Feeding mgo at #OSCON with @spf13 http://j.mp/mongomgo

https://github.com/spf13/

firstGoApp-Planet/tree/

step4168

Step  5  

Building  a   web  Server

169

Lots  of  choices• BeegoHigh-performance web framework

• GinMartini like with better performance

• GojiA minimalistic web framework

• Gorilla A web toolkit

• httprouter A high performance router

• Mango modular web-app framework like Rack

• Martini modular web apps & services

• net/httpStandard library web package

• pat Sinatra style, by the author of Sinatra.

• RevelA high-productivity web framework

• tigertonic framework for JSON web services

• traffic Sinatra inspired web framework for Go

• web.go A simple framework to write webapps in Go.

170

Gin

• Compatible with net/http • Uses Httprouter (really fast) • Familiar/Friendly Interface • Works well with JSON

• Had to pick something 171

go get -u github.com/gin-gonic/gin

Getting  Gin

172

boilerplate  stuff

173

// Copyright © 2014 Steve Francia <spf@spf13.com>. // // Use of this source code is governed by an Apache2 // license that can be found in the LICENSE file. !package commands !import ( "github.com/gin-gonic/gin" "github.com/spf13/cobra" "github.com/spf13/viper" )

Server.Go  (1)

174

Defining  the  Server  command

175

var serverCmd = &cobra.Command{ Use: "server", Short: "Server for feeds", Long: `Dagobah will serve all feeds listed in the config file.`, Run: serverRun, } !func init() { serverCmd.Flags().Int("port", 1138, "Port to run Dagobah server on") viper.BindPFlag("port", serverCmd.Flags().Lookup("port")) }

server.go  (2)

176

func addCommands() { RootCmd.AddCommand(fetchCmd) RootCmd.AddCommand(serverCmd) }

planet.go

177

Defining  our  first  route

178

func serverRun(cmd *cobra.Command, args []string) { r := gin.Default() !

r.GET("/ping", func(c *gin.Context) { c.String(200, "pong") }) !

port := viper.GetString("port") fmt.Println("Running on port:", port) r.Run(":" + port) }

server.go  (3)

179

❯ go run dagobah.go server

Running  it

180

Check  it  outhttp://localhost:1138 181

Tweet  Break

I wrote a webserver in go using Gin with @spf13 at #OSCON http://j.mp/gin-go

https://github.com/spf13/

firstGoApp-Planet/tree/

step5183

Step  6  

Building  a   Dynamic  Server

184

Binary  Problem?

• Go ships as a single binary • Need to load files from within the binary • Don’t want to embed by hand

185

go.rice

• Nicely allows local loads in dev, and embedded loads when executing binary

• Stable, but not very mature yet • Best option I know of

186

go get -u github.com/GeertJohan/go.rice

Getting  Go.Rice

187

Serving  Static  Files

188

Get  your  static  &  

templates  on

c.spf13.com/OSCON/step6-static.zip189

import ( "fmt" "html/template" "log" "net/http" ! "github.com/GeertJohan/go.rice" "github.com/gin-gonic/gin" "github.com/spf13/cobra" "github.com/spf13/viper" )

server.go

190

func serverRun(cmd *cobra.Command, args []string) { ... r.GET("/static/*filepath", staticServe) !

port := viper.GetString("port") fmt.Println("Running on port:", port) r.Run(":" + port) }

server.go

191

func staticServe(c *gin.Context) { static, err := rice.FindBox("static") if err != nil { log.Fatal(err) } ! original := c.Request.URL.Path c.Request.URL.Path = c.Params.ByName("filepath") fmt.Println(c.Params.ByName("filepath")) http.FileServer(static.HTTPBox()).ServeHTTP(c.Writer, c.Request) c.Request.URL.Path = original }

server.go

192

Loading  &  Serving    Templates

193

func loadTemplates(list ...string) *template.Template { templateBox, err := rice.FindBox("templates") if err != nil { log.Fatal(err) } templates := template.New("") ! for _, x := range list { templateString, err := templateBox.String(x) if err != nil { log.Fatal(err) } ! // get file contents as string _, err = templates.New(x).Parse(templateString) if err != nil { log.Fatal(err) } } return templates }

server.go

194

func serverRun(cmd *cobra.Command, args []string) { r := gin.Default() templates := loadTemplates("full.html") r.SetHTMLTemplate(templates) ! r.GET("/ping", func(c *gin.Context) { c.String(200, "pong") }) ! r.GET("/", homeRoute) r.GET("/static/*filepath", staticServe)! port := viper.GetString("port") fmt.Println("Running on port:", port) r.Run(":" + port) }

server.go

195

func homeRoute(c *gin.Context) { obj := gin.H{"title": "Go Rules"} c.HTML(200, “full.html", obj) }

server.go

196

❯ go run dagobah.go server

Running  it

197

Check  it  outhttp://localhost:1138 198

Tweet  Break

No more mr rice guy with @spf13 at #OSCON http://j.mp/go-rice

https://github.com/spf13/

firstGoApp-Planet/tree/

step6200

Step  7  

Connecting  the  Data  to  The  templates

201

Go  Templates

• Really nice to work with • Very simple, yet very powerful • Safe • Go Template Primer

202

Understanding  the  data

203

> db.items.find().sort({ "date" : -1 }).limit(1).pretty() { "_id" : ObjectId("53b4c08bddbc460a933cf3ed"), "date" : ISODate("2014-07-01T00:00:00Z"), "key" : "http://spf13.com/post/go-pointers-vs-references", "channelkey" : "Recent Content on Hacking Management", "title" : "Pointers vs References", "links" : [ { "href" : "http://spf13.com/post/go-pointers-vs-references", "rel" : "", "type" : "", "hreflang" : "" }

Looking  at  the  Schema

204

Listing  POsts

205

passing  data    into  the  template

• Go’s templates accept a variety of data types • Gin’s default is ‘H’ : map[string]interface{} • Feels natural • All (exported) methods bound to values in the map are accessible inside the template

206

func homeRoute(c *gin.Context) { var posts []Itm results := Items().Find(bson.M{}).Sort("-date").Limit(20) results.All(&posts) ! obj := gin.H{"title": "Go Rules", "posts": posts} c.HTML(200, "full.html", obj) }

server.go

207

... <section id="latest-list" class="ui divided inbox selection list active tab" data-tab="recent"> {{ range $item := .posts }} {{ if $item.WorthShowing }} <a class="item" href=“#{{$item.Key | urlquery }}"> <div class="description">{{ $item.Title }}</div> <div class="right floated ui label small">{{ $item.Date.Format "Jan 2, 2006" }}</div> </a> {{ end }} {{ end }} </section> ...

Full.html

208

Adding  Helpers  to  itm

209

func (i Itm) FirstLink() (link rss.Link) { if len(i.Links) == 0 || i.Links[0] == nil { return } return *i.Links[0] } !func (i Itm) WorthShowing() bool { if len(i.FullContent) > 100 { return true } return false }

fetch.go

210

displaying  POsts

211

... <main> {{ with .message}}! <h1 class="ui header large"> {{.}} </h1>! {{ end }}!! {{ range $item := .posts }}! {{ if $item.WorthShowing }}! <article data-key="{{$item.Key}}">! <header>! <a name="{{$item.Key}}">! <h1 class="ui header">{{$item.Title}}</h1>! <section class="meta-tags">! <a class="ui label large blue author" href="/channel/{{$item.ChannelKey}}">{{$item.Author.Name}}</a>! <span class="large ui label date">{{ $item.Date.Format "Jan 2, 2006" }}</span>! </section>! </header>! <section class="main-content">! {{$item.FullContent | html }}! </section>! <footer>! {{with $item.FirstLink}}! <a class="ui basic button" href="{{.Href}}">Source</a>! {{end}}! <div class="ui divider"></div>! </footer>! </article>! {{ end }}! {{ else }}! No Articles! {{ end }}! </main> ...

full.html

212

Adding  Template  Functions

213

func loadTemplates(list ...string) *template.Template { templateBox, err := rice.FindBox("templates") if err != nil { log.Fatal(err) } ! templates := template.New("") ! for _, x := range list { templateString, err := templateBox.String(x) if err != nil { log.Fatal(err) } ! // get file contents as string _, err = templates.New(x).Parse(templateString) if err != nil { log.Fatal(err) } }

! funcMap := template.FuncMap{ "html": ProperHtml, "title": func(a string) string { return strings.Title(a) }, } ! templates.Funcs(funcMap) ! return templates }

server.go

214

func ProperHtml(text string) template.HTML { if strings.Contains(text, "content:encoded>") || strings.Contains(text, "content/:encoded>") { text = html.UnescapeString(text) } return template.HTML(html.UnescapeString(template.HTMLEscapeString(text))) }

server.go

215

Looking  Goodhttp://localhost:1138 216

Tweet  Break

Conquering Go Templates with @spf13 at #OSCON

https://github.com/spf13/

firstGoApp-Planet/tree/

step7218

Step  8  

More  Routes219

Adding  Helpers

220

func (c Chnl) HomePage() string { if len(c.Links) == 0 { return "" } !

url, err := url.Parse(c.Links[0].Href) if err != nil { log.Println(err) } return url.Scheme + "://" + url.Host }

fetch.go

221

!func four04(c *gin.Context, message string) { c.HTML(404, "full.html", gin.H{"message": message, "title": message}) }

server.go

222

func AllChannels() []Chnl { var channels []Chnl r := Channels().Find(bson.M{}).Sort("-lastbuilddate") r.All(&channels) return channels }

MongoDB.go

223

Listing  channels

224

func homeRoute(c *gin.Context) { var posts []Itm results := Items().Find(bson.M{}).Sort("-date").Limit(20) results.All(&posts) ! obj := gin.H{"title": "Go Rules", "posts": posts, "channels": AllChannels()} c.HTML(200, "full.html", obj) }

server.go

225

... <section id="channel-list" class="ui large inverted vertical menu tab" data-tab=“channels"> <a class="item js-navigate" href="/" >! <i class="external home icon"></i>! <span class="ui name" href=""> All </span>! </a>! {{ range $channel := .channels }}! <div class="item js-navigate-channel" data-href="/channel/{{$channel.Key}}" data-key="{{$channel.Key}}" style="cursor:pointer;">! <a href="{{$channel.HomePage}}" class="ui float right"><i class="external url sign icon"></i></a>! <span class="ui name" href=""> {{$channel.Title}} </span>! <span class="ui label">{{ len $channel.Links }}</span>! </div>! {{end}}! </section> ...

full.html

226

displaying  channels

227

func serverRun(cmd *cobra.Command, args []string) { ... r.GET("/channel/*key", channelRoute) r.Run(":" + port) }

server.go

228

func channelRoute(c *gin.Context) { key := c.Params.ByName("key") if len(key) < 2 { four04(c, "Channel Not Found") return } ! key = key[1:] ! fmt.Println(key) ! var posts []Itm results := Items().Find(bson.M{"channelkey": key}).Sort("-date").Limit(20) results.All(&posts) ! if len(posts) == 0 { four04(c, "No Articles") return } ...

server.go  -  pt  1

229

... var currentChannel Chnl err := Channels().Find(bson.M{"key": key}).One(&currentChannel) if err != nil { if string(err.Error()) == "not found" { four04(c, "Channel not found") return } else { fmt.Println(err) } } ! obj := gin.H{"title": currentChannel.Title, "header": currentChannel.Title, "posts": posts, "channels": AllChannels()} ! c.HTML(200, "full.html", obj) }

server.go  -  pt  2

230

displaying  A  post

231

func serverRun(cmd *cobra.Command, args []string) { ... r.GET("/post/*key", channelRoute) r.Run(":" + port) }

server.go

232

func postRoute(c *gin.Context) { key := c.Params.ByName("key") ! if len(key) < 2 { four04(c, "Invalid Post") return } ! key = key[1:] ! var ps []Itm r := Items().Find(bson.M{"key": key}).Sort("-date").Limit(1) r.All(&ps) ! if len(ps) == 0 { four04(c, "Post not found") return } ...

server.go  -  Pt  1

233

...! var posts []Itm results := Items().Find(bson.M{"date": bson.M{"$lte": ps[0].Date}}).Sort("-date").Limit(20) results.All(&posts) ! obj := gin.H{"title": ps[0].Title, "posts": posts, "channels": AllChannels()} ! c.HTML(200, "full.html", obj) }

server.go  -  Pt  2

234

Looking  Goodhttp://localhost:1138 235

Tweet  Break

Making my own planet in #go with @spf13 at #OSCON http://j.mp/go-planet

https://github.com/spf13/

firstGoApp-Planet/tree/

step8237

Step  9  

Advanced  Routes238

Congrats

• You’ve made it really far • Now the training wheels are off • Let’s try to add pagination & search

240

Hints

•You will need a new index •We’re talking full text here

•You will also need a new route241

... r.GET("/", homeRoute) r.GET("/post/*key", postRoute) r.GET("/search/*query", searchRoute) r.GET("/static/*filepath", staticServe) r.GET("/channel/*key", channelRoute) ...

server.go

242

Step  10  !

Add  Polish243

What’s  next  ?

• Add ‘truncate’ command to remove old posts and channels

• Change root command to run fetcher & server

• Add infinite scroll capabilities 244

What’s  next  ?

• Make it so users can provide their own static files & templates

• Documentation • Blog posts

245

In  Conclusion

246Go mascot designed by Renée French and copyrighted under the Creative Commons Attribution 3.0 license.

Thank  yOU247

What  have  we  done  ?

• Written our first lines of Go

• Written our first Go package

• Written our own web server

• Written our first Go application

• Learned a lot & had fun doing it 248

@spf13

• Author of Hugo, Cobra, Viper & More

• Chief Developer Advocate for MongoDB

•Gopher 249