View
135
Download
5
Category
Tags:
Preview:
DESCRIPTION
This presentation was given as a Workshop at OSCON 2014. New to Go? This tutorial will give developers an introduction and practical experience in building applications with the Go language. Gopher Steve Francia, Author of [Hugo](http://hugo.spf13.com), [Cobra](http://github.com/spf13/cobra), and many other popular Go packages breaks it down step by step as you build your own full featured Go application. Starting with an introduction to the Go language. He then reviews the fantastic go tools available. With our environment ready we will learn by doing. The remainder of the time will be dedicated to building a working go web and cli application. Through our application development experience we will introduce key features, libraries and best practices of using Go. This tutorial is designed with developers in mind. Prior experience with any of the following languages: ruby, perl, java, c#, javascript, php, node.js, or python is preferred. We will be using the MongoDB database as a backend for our application. We will be using/learning a variety of libraries including: * bytes and strings * templates * net/http * io, fmt, errors * cobra * mgo * Gin * Go.Rice * Cobra * Viper
Citation preview
Building your first Go App
1Go mascot designed by Renée French and copyrighted under the Creative Commons Attribution 3.0 license.
@spf13
• Author of Hugo, Cobra, Viper & More
• Chief Developer Advocate for MongoDB
•Gopher 2
My Go Story
• I write a medium size blog • Frustrated with Wordpress performance & maintenance
• Existing Static Site Generators had complicated installations & were very slow
• Began writing a S.S.G. in Go called Hugo 1.5 years ago4
Hugo• Hugo is 2nd fastest growing Static Site Generator (40-50 * a week on github)
• Hugo is one of the fastest Static Site Generators. 1000x faster than Jekyll
• Easiest Installation of any SSG. Download and run • 2nd most contributors of any Go project (Docker 1st)6
Plan
• Introduce Go Language • Introduce Tools & Libraries • Build our application • Tell the world how awesome we are 8
Ground Rules
• Workshops are hard, everyone works at a different pace
• We will move on when about 50% are ready
• Slides are online, feel free to work ahead or catch up
9
http://spf13.com/
presentation/first-go-app/
10
Installing Go
• You should already have Go installed • If you don’t, do it NOW
• Installation Guide 11
Installing MongoDB
• You should already have MongoDB installed • If you don’t, do it NOW
• Installation Guide 12
Git & Mercurial
• Lastly we need Git & Mercurial • http://git-scm.com/downloads • http://mercurial.selenic.com/wiki/Download
13
Introduction to Go
14
Why Another Language?
• Software is slow • Sofware is hard to write • Software doesn’t scale well
15
Go is Fast
• Go execution speed is close to C • Go compile time rivals dynamic interpretation
16
Go is Friendly• Feels like a dynamic language in many ways • Very small core language, easy to remember all of it
• Single binary installation, no dependencies • Extensive Tooling & StdLib
17
Go is Concurrent• Concurrency is part of the language • Any function can become a goroutine • Goroutines run concurrently, communicate through channels
• Select waits for communication on any of a set of channels
18
Go’s Inspiration• C: statement and expression syntax • Pascal: declaration syntax • Modula 2, Oberon 2: packages • CSP, Occam, Newsqueak, Limbo, Alef: concurrency • BCPL: the semicolon rule • Smalltalk: methods • Newsqueak: <-, := • APL: iota 19
… AND lessons good and bad from all those plus: C++, C#, Java, JavaScript, LISP, Python, Scala, ...
— txxxxd
“Most of the appeal for me is not the
features that Go has, but rather the
features that have been intentionally left
out.”20
— Rob Pike
“Why would you have a language that is not theoretically
exciting? Because it’s very useful.”
21
package main !
import "fmt" !
func main() { fmt.Println(“Hello, 世界") }
Hello World
22
Package
• Not classes or objects • No subpackages • Fundamental building block in Go • Visibility is package-level, not type-level • Programs run “main” package 23
package main !import "fmt" !func main() { fmt.Println("Hello, 世界") }
import
• Provides access to a package • Package consists of types, functions, etc… • No circular dependencies
24
package main !import "fmt" !func main() { fmt.Println("Hello, 世界") }
“fmt"• Package path is just a string
• Standard library sits at the root (no path) • Core language is very small • Most functionality is in Stdlibs 25
package main !import "fmt" !func main() { fmt.Println("Hello, 世界") }
func
• Declaring a function • Same syntax for methods, anonymous functions & closures
• main.main() is the function run when the program is executed
26
package main !import "fmt" !func main() { fmt.Println("Hello, 世界") }
Main
• No void • No return value • No function args (command line is in os package)
27
package main !import "fmt" !func main() { fmt.Println("Hello, 世界") }
{ }• Braces not spaces • Feels like C ( & most languages ) • Newline must be after brace
(semicolons are inserted during compiliation)
28
package main !import "fmt" !func main() { fmt.Println("Hello, 世界") }
fmt
• Accessing the package imported above
• Everything visible in the package will be accessible through the name (or alias if provided)
29
package main !import "fmt" !func main() { fmt.Println("Hello, 世界") }
Println
• Println, not println • Println, not print_ln • Capital for export • Variadic function using reflection
30
package main !import "fmt" !func main() { fmt.Println("Hello, 世界") }
“Hello, 世界”
• UTF-8 input source • Strings are immutable • Strings are UTF-8 encoded • String is a built-in type
31
package main !import "fmt" !func main() { fmt.Println("Hello, 世界") }
Rob Pike did it better
http://confreaks.com/videos/3419-gophercon2014-opening-day-keynote
32
Tons more…
• types • constants • methods • interfaces • pointers
• control flow • conditionals • tools • packages • concurrency 33
Effective Go
34
Lets write some code
!
Introduce the go tools
35
package main !
import "fmt" !
func main() { fmt.Println("Hello,World") }
Hello World
36
hello.go
❯ go run hello.goHello,World
Go Run
37
❯ go build hello.go !
❯ls -lh 1.7M Jun 17 22:44 hello 72B Jun 17 22:43 hello.go !
❯ ./helloHello, World
Go Build
38
Go Fmt
• Automatically formats Go source code • Ends stylistic debates • Integration with editors (vim, emacs, others) • Can also refactor code (see http://spf13.com/post/go-fmt )
39
go Test
• Enables easy testing of application • Integrates with the testing package • Supports benchmark, functional & unit style testing
• Combine with ‘looper’ to have realtime feedback 40
package main !
import "testing" !
func TestOne(t *testing.T) { one := false if !one { t.Errorf(“Test Failed”) } }
Hello Test
41
hello_test.go
❯ go test ./... --- FAIL: TestOne (0.00 seconds) hello_test.go:8: Test failed FAIL FAIL _/Code/firstApp 0.012s !
!
❯ go test ./... ok _/Code/firstApp 0.012s
Go Test
42
Go Planet
43The planet logo is based on the Go mascot designed by Renée French and copyrighted under the Creative Commons Attribution 3.0 license.
Why Planet?
• CLI application • Web application
• Good introduction
• Right sized • Database (MongoDB)
• Concurrency 45
Steps
0. Env Setup 1. Commands 2. Configuration 3. Working with Feeds
4. DB as Storage 5-6. Web Server 7-8. DB -> Templates 9. & 10. Add Polish
46
Step 0
Setting up our environment
47
Go PAth !
$GOPATH48
Go path
• The GOPATH environment variable specifies the location of your workspace (and dependencies)
• It must not be the same path as your Go installation
49
❯ mkdir $HOME/go!
❯ export GOPATH=$HOME/go
Setting up
50
export GOPATH=$USER/go export GOROOT=`go env GOROOT` PATH=$PATH:$GOPATH/bin
Linux & Mac
51
Add this to your ~/.bashrc (or equivalent)
Windows• Control panel > System
52
Go PAth
• /home/user/gocode/ ($GOPATH) • src/ (put your source code here)
• bin/ (binaries installed here)
• pkg/ (installed package objects)53
Creating our project
54
Project Dir
• Make a directory inside of $GOPATH/src • I like mine to be in $GOPATH/src/github.com/spf13/PROJECTNAME
• It can not be a symlink • We will call this our working directory or $wd
55
package main !
import "fmt" !
func main() { fmt.Println(“My Project") }
main.go
56
❯ go run main.goMy Project
Run it
57
58
I’m on the #gopath with @spf13 at #OSCONhttp://j.mp/onGoPath
Tweet Break
Step 1
Creating the Command(s)
59
Defining our first command
60
Structs
• Short for structure • Objectish … blend of data & methods, but no inheritance
• Really cheap. Struct{} is free (0 bytes).61
Functions
• Can have multiple return values • No overloading • No optional parameters (but variadic)
62
First class Functions
• Function literals are anonymous functions in Go • Can assign to a variable or field • Can use as function parameters or return values
• Can be created as a goroutine 63
pointers• Function calls copy arguments into the function • Pointers reference a location in memory where a value is stored rather than the value itself
• In Go a pointer is represented using the * • The & operator gives us the address of a variable • Go automatically dereferences pointers when using “.” 64
cobra
• A CLI Commander • Easy management of commands & flags
• Used by bitbucket, openshift & lots more
65http://fav.me/d5gkby7
go get -u github.com/spf13/cobra
Getting Cobra
66
❯ mkdir $wd/commands
Commands Dir
67
// Copyright © 2014 Steve Francia <spf@spf13.com>. // // Use of this source code is governed by an Apache2 // license that can be found in the LICENSE file. !package commands !import ( "fmt" "os" ! "github.com/spf13/cobra" )
planet.go (1)
68
Path = Urlgo get uses it to install
69
var RootCmd = &cobra.Command{ Use: "dagobah", Short: `Dagobah is an awesome planet style RSS aggregator`, Long: `Dagobah provides planet style RSS aggregation. It is inspired by python planet. It has a simple YAML configuration and provides it's own webserver.`, Run: func(cmd *cobra.Command, args []string) { fmt.Println("Dagobah runs") }, }
planet.go (2)
70
executing cobra’s command
71
Variable Assignment
• ‘:=‘ is the short variable declaration operatorDeclares, creates & assigns simultaneously while infering type
• Assignment via ‘=‘ requires prior declaration • var foo int = 10 is the same as foo := 10 72
Error Handling• Errors are not exceptional. Should be part of the language • Typically one of the return values • No exceptions in Go fd, err := os.Open("test.go") if err != nil { log.Fatal(err) } 73
( Don’t) Panic• Should rarely be used if ever • Performs a full stack trace dump for end users • Use only in the most dire circumstances (like package can’t find FileSystem)
• Can use ‘recover’, but only inside deferred functions74
func Execute() { err := RootCmd.Execute() if err != nil { fmt.Println(err) os.Exit(-1) } }
planet.go (3)
75
// Copyright © 2014 Steve Francia <spf@spf13.com>. // // Use of this source code is governed by an Apache2 // license that can be found in the LICENSE file. !package main !import "github.com/spf13/firstGoApp-Planet/commands" !func main() { commands.Execute() }
main.go
76
Make sure to use your path
❯ go run main.goDagobah runs !
❯ go run main.go help
Running it
77
Getting Help
• https://godoc.org/github.com/spf13/cobra • https://github.com/spf13/firstGoApp-Planet/tree/step1
78
Step 2
Easy Configuration80
Handing Config files
81
Package Identifiers
• Export with Capital name • No automatic getters & setters • var field … func Field() … func SetField() are appropriate names
82
init()
• Each source file can have one (or multiple) • Niladic function to setup state (function without return value)
• Called after all variable declarations in the package
83
Scoping
• Very granular scoping. • Variables declared inside { } only survive in that cotrol structure (for, if, etc)
• The := trap if := is used to declare x and x already exists in an outer scope a new variable x will be created (shadowing)
84
package main !import ( "errors" "fmt" ) !var Foo int !func Zero() (int, error) { return 0, errors.New("yo") } !!
func main() { ! Foo = 10 ! fmt.Println("outer:", Foo) // 10 ! if Foo, err := Zero(); err != nil { fmt.Println("inner:", Foo) // 0 } ! fmt.Println("outer again:", Foo) // still 10 }
:= Trap
85http://play.golang.org/p/i4L9Ao1P65
Viper
• A configuration manager • Easy loading of config files • Registry for application settings • Designed to work well with (or without) Cobra
86
go get -u github.com/spf13/viper
Getting Viper
87
// Copyright © 2014 Steve Francia <spf@spf13.com>. // // Use of this source code is governed by an Apache2 // license that can be found in the LICENSE file. !package commands !import ( "fmt" "os" ! "github.com/spf13/cobra" "github.com/spf13/viper" )
planet.go (1)
88
var CfgFile string !func init() { cobra.OnInitialize(initConfig) RootCmd.PersistentFlags().StringVar(&CfgFile, "config", "", "config file (default is $HOME/dagobah/config.yaml)") } !func initConfig() { if CfgFile != "" { viper.SetConfigFile(CfgFile) } viper.SetConfigName("config") viper.AddConfigPath("/etc/dagobah/") viper.AddConfigPath("$HOME/.dagobah/") viper.ReadInConfig() }
planet.go (2)
89
Creating our config file
90
appname: "Dagobah" feeds: - "http://spf13.com/index.xml" - "http://dave.cheney.net/feed" - "http://www.goinggo.net/feeds/posts/default" - "http://blog.labix.org/feed" - "http://blog.golang.org/feed.atom" !
$HOME/.dagobah/config.yaml
91
trying it out
92
var RootCmd = &cobra.Command{ Use: "...", Short: `...`, Long: `...`, Run: rootRun, } !func rootRun(cmd *cobra.Command, args []string) { fmt.Println(viper.Get("feeds")) fmt.Println(viper.GetString("appname")) }
planet.go (3)
93
❯ go run main.go[http://spf13.com/index.xml http://dave.cheney.net/feed http://www.goinggo.net/feeds/posts/default http://blog.labix.org/feed http://blog.golang.org/feed.atom]Dagobah
Running it
94
https://github.com/spf13/
firstGoApp-Planet/tree/
step296
Step 3
Working with Feeds97
Finding Libraries
98
Go-Search.org
99
Go wiki
100
go-pkg-rss
• Fetch Rss and Atom feeds from the internet • Supports RSS .92, 1.0, 2.0 & Atom 1.0 • Respects TTL, SkipDays, CacheTimeout, etc in the feeds
101
go get -u github.com/jteeuwen/go-pkg-rss
Getting Go-PKG-RSS
102
boilerplate stuff
103
import ( "time" ... ) !func addCommands() { RootCmd.AddCommand(fetchCmd) } !func Execute() { addCommands() ... }
planet.go
104
// Copyright © 2014 Steve Francia <spf@spf13.com>. // // Use of this source code is governed by an Apache2 // license that can be found in the LICENSE file. !package commands !import ( "fmt" "os" "time" ! rss "github.com/jteeuwen/go-pkg-rss" "github.com/spf13/cobra" "github.com/spf13/viper" )
Fetch.Go (1)
105
Creating our own type
106
Named Types
• Can be any known type (struct, string, int, slice, a new type you’ve declared, etc)
• Can have methods declared on it • Not an alias. Explicit type.
107
Slices & arrays
• Array is fixed in length • Slice can be viewed as a dynamic array (can grow) • Slice is a view of an array • Slice is the core data structure in Go for lists of data
108
type Config struct { Feeds []string Port int }
Fetch.Go (2)
109
Defining the fetch
command110
New( )
• new(T) Does 2 things at once: • Allocates new zeroed value of type T • Returns a pointer to the new value ( *T) • Used for all non-reference types
111
Make( )
• Creates slices, maps, and channels only • These types are special “reference” types • Makes a initialized (not zeroed) value of type T • Does not return a pointer
112
Composite Literals
• An expression that creates a new value each time it is evaluated
• We’ve already used these quite a bit
• Can also be create values of arrays, slices, and maps • Pseudo Constructor • eg File{fd: fd, name: name} 113
Constructors
• Not built into Go • Use Composite Literals when possible • If initialization is needed use a factory • Convention is to use New___() • … or New() when only one type is exported in the package
114
var fetchCmd = &cobra.Command{ Use: "fetch", Short: "Fetch feeds", Long: `Dagobah will fetch all feeds listed in the config file.`, Run: fetchRun} !func init() { fetchCmd.Flags().Int("rsstimeout", 5, "Timeout (in min) for RSS retrival") viper.BindPFlag("rsstimeout", fetchCmd.Flags().Lookup("rsstimeout")) }
Fetch.go (3)
115
Fetching in a goroutine
116
GoRoutines
• A function executing concurrently with other goroutines in the same address space
• Lightweight • Not threads, coroutines, or processes
117
Channels• Channels are lightweight • Synchronize goroutines by communication rather than locking shared memory
• Channel <- Sender —- Writes block when channel is full
• <- Channel —- Reads blocks waiting for something to be read • Great talk on channels given at GopherConhttp://confreaks.com/videos/3422-gophercon2014-a-channel-compendium
118
Marshaling into
• We can’t return a T (no generics)
• We can reflect on any value of any type
• Marshal accepts the address of the value so it can manipulate the value directly
119
func fetchRun(cmd *cobra.Command, args []string) { Fetcher() !
sigChan := make(chan os.Signal, 1) signal.Notify(sigChan, os.Interrupt) <-sigChan }
Fetch.go (4)
120
func Fetcher() { var config Config if err := viper.Marshal(&config); err != nil { fmt.Println(err) } !
for _, feed := range config.Feeds { go PollFeed(feed.Url) } }
Fetch.go (5)
121
Building our fetcher
122
Learning about go-pkg-rss
• How do we learn how to use the package?123
Learning about go-pkg-rss
• github.com/jteeuwen/go-pkg-rss • Readme is pretty lacking • godoc.org/github.com/jteeuwen/go-pkg-rss
124
godoc.org/github.com/jteeuwen/go-pkg-rss125
Go Search
126
For
• For, Foreach & While are all spelled ‘for’ in Go for init; condition; post { } // Like a C for
for condition { } // Like a C while
for { } // Like a C for(;;)
127
func PollFeed(uri string) { timeout := viper.GetInt("RSSTimeout") if timeout < 1 { timeout = 1 } feed := rss.New(timeout, true, chanHandler, itemHandler) ! for { if err := feed.Fetch(uri, nil); err != nil { fmt.Fprintf(os.Stderr, "[e] %s: %s", uri, err) return } ! fmt.Printf("Sleeping for %d seconds on %s\n", feed.SecondsTillUpdate(), uri) time.Sleep(time.Duration(feed.SecondsTillUpdate() * 1e9)) } }
Fetch.GO (6)
128
func chanHandler(feed *rss.Feed, newchannels []*rss.Channel) { fmt.Printf("%d new channel(s) in %s\n", len(newchannels), feed.Url) } !func itemHandler(feed *rss.Feed, ch *rss.Channel, newitems []*rss.Item) { fmt.Printf("%d new item(s) in %s\n", len(newitems), feed.Url) }
Fetch.go (7)
129
❯ go run dagobah.go fetch --rsstimeout=1 15 new item(s) in http://spf13.com/index.xml 1 new channel(s) in http://spf13.com/index.xml Sleeping for 300 seconds on http://spf13.com/index.xml 25 new item(s) in http://www.goinggo.net/feeds/posts/default 1 new channel(s) in http://www.goinggo.net/feeds/posts/default Sleeping for 300 seconds on http://www.goinggo.net/feeds/posts/default 10 new item(s) in http://dave.cheney.net/feed 1 new channel(s) in http://dave.cheney.net/feed Sleeping for 299 seconds on http://dave.cheney.net/feed
Running it
130
Tweet Break
I wrote my first goroutine at #OSCON with @spf13
https://github.com/spf13/
firstGoApp-Planet/tree/
step3132
Step 4
Storing in the Database
133
Pretty
• github.com/kr/pretty • pretty.Println(X,Y,Z)
134
MongoDB
• Open Source • Document Database • Document == Struct or Document == Map • Works seamlessly with Go (and other languages)
135
MongoDB
•Fast & Scalable • Used by Disney, IBM, Metlife, eBay, Forbes, Craigslist, Cisco, Stripe
136
Mgo• Developed by the open source community • Heavily used throughout Go community • "mgo is the best database driver I've ever used." — Patrick Crosby, Founder of StatHat
• "mgo and Go are a pair made in heaven. — Brian Ketelsen, Author of GopherTimes.com
137
go get -u gopkg.in/mgo.v2
Getting mgo
138
boilerplatestuff
139
// Copyright © 2014 Steve Francia <spf@spf13.com>. // // Use of this source code is governed by an Apache2 // license that can be found in the LICENSE file. !package commands !import ( "fmt" "os" ! "github.com/spf13/viper" "gopkg.in/mgo.v2" ) !var mongodbSession *mgo.Session !func init() { RootCmd.PersistentFlags().String("mongodb_uri", "localhost", "host where mongoDB is") viper.BindPFlag("mongodb_uri", RootCmd.PersistentFlags().Lookup(“mongodb_uri")) CreateUniqueIndexes() }
MongoDB.go (1)
140
Setting up a DB session
141
func DBSession() *mgo.Session { if mongodbSession == nil { uri := os.Getenv("MONGODB_URI") if uri == "" { uri = viper.GetString("mongodb_uri") ! if uri == "" { log.Fatalln("No connection uri for MongoDB provided") } } ! var err error mongodbSession, err = mgo.Dial(uri) if mongodbSession == nil || err != nil { log.Fatalf("Can't connect to mongo, go error %v\n", err) } ! mongodbSession.SetSafe(&mgo.Safe{}) } return mongodbSession }
MongoDB.go (2)
142
setting up some
shortcuts143
!func DB() *mgo.Database { return DBSession().DB(viper.GetString("dbname")) } !func Items() *mgo.Collection { return DB().C("items") } !func Channels() *mgo.Collection { return DB().C("channels") }
MongoDB.go (3)
144
Setting up indexes
145
func CreateUniqueIndexes() { idx := mgo.Index{ Key: []string{"key"}, Unique: true, DropDups: true, Background: true, Sparse: true, } ! if err := Items().EnsureIndex(idx); err != nil { fmt.Println(err) } ! if err := Channels().EnsureIndex(idx); err != nil { fmt.Println(err) } }
MongoDB.go (4)
146
Preparing the data
147
Data Prep
• Feeds are dirty • Go doesn’t allow “patching” of structs (not how structs work)
• MongoDB works with native types & named types
148
Tags (Annotations)
• String literal following a field declaration • Provides additional information during reflection
• Used by JSON, BSON, YAML, etc 149
for x,y := range
• Provides a way to iterate over an array, slice, string, map, or channel.
• Like Foreach or Each in other languages • x is key, y is value… can ignore by setting to _
150
Append
• Append is really awesome, feels like push, but does a lot more.
• Append will grow a slice (both length & capacity)
• … and copy the underlying array if needed 151
type Itm struct { Date time.Time Key string ChannelKey string Title string FullContent string Links []*rss.Link Description string Author rss.Author Categories []*rss.Category Comments string Enclosures []*rss.Enclosure Guid *string `bson:",omitempty"` Source *rss.Source PubDate string Id string `bson:",omitempty"` Generator *rss.Generator Contributors []string Content *rss.Content Extensions map[string]map[string][]rss.Extension }
fetch.go
152
func itmify(o *rss.Item, ch *rss.Channel) Itm { var x Itm x.Title = o.Title x.Links = o.Links x.ChannelKey = ch.Key() x.Description = o.Description x.Author = o.Author x.Categories = o.Categories x.Comments = o.Comments x.Enclosures = o.Enclosures x.Guid = o.Guid x.PubDate = o.PubDate x.Id = o.Id x.Key = o.Key() x.Generator = o.Generator x.Contributors = o.Contributors x.Content = o.Content x.Extensions = o.Extensions x.Date, _ = o.ParsedPubDate() ! if o.Content != nil && o.Content.Text != "" { x.FullContent = o.Content.Text } else { x.FullContent = o.Description } ! return x }
fetch.go
153
type Chnl struct { Key string Title string Links []rss.Link Description string Language string Copyright string ManagingEditor string WebMaster string PubDate string LastBuildDate string Docs string Categories []*rss.Category Generator rss.Generator TTL int Rating string SkipHours []int SkipDays []int Image rss.Image ItemKeys []string
Cloud rss.Cloud TextInput rss.Input Extensions map[string]map[string][]rss.Extension Id string Rights string Author rss.Author SubTitle rss.SubTitle }
fetch.go
154
func chnlify(o *rss.Channel) Chnl { var x Chnl x.Key = o.Key() x.Title = o.Title x.Links = o.Links x.Description = o.Description x.Language = o.Language x.Copyright = o.Copyright x.ManagingEditor = o.ManagingEditor x.WebMaster = o.WebMaster x.PubDate = o.PubDate x.LastBuildDate = o.LastBuildDate x.Docs = o.Docs x.Categories = o.Categories x.Generator = o.Generator x.TTL = o.TTL
x.Rating = o.Rating x.SkipHours = o.SkipHours x.SkipDays = o.SkipDays x.Image = o.Image x.Cloud = o.Cloud x.TextInput = o.TextInput x.Extensions = o.Extensions x.Id = o.Id x.Rights = o.Rights x.Author = o.Author x.SubTitle = o.SubTitle !... ! return x }
fetch.go
155
... !
!
var keys []string for _, y := range o.Items { keys = append(keys, y.Key()) } x.ItemKeys = keys !
...
fetch.go
156
Storing Feeds & Items
157
The Blank Identifier ‘_’
• Used to ignore return values from a function
• Used with imports as an alias • Allows import of unused packages so init
functions can be executed158
func chanHandler(feed *rss.Feed, newchannels []*rss.Channel) { fmt.Printf("%d new channel(s) in %s\n", len(newchannels), feed.Url) for _, ch := range newchannels { chnl := chnlify(ch) if err := Channels().Insert(chnl); err != nil { if !strings.Contains(err.Error(), "E11000") { fmt.Printf("Database error. Err: %v", err) } } } }
fetch.go
159
func itemHandler(feed *rss.Feed, ch *rss.Channel, newitems []*rss.Item) { fmt.Printf("%d new item(s) in %s\n", len(newitems), feed.Url) for _, item := range newitems { itm := itmify(item, ch) if err := Items().Insert(itm); err != nil { if !strings.Contains(err.Error(), "E11000") { fmt.Printf("Database error. Err: %v", err) } } } }
fetch.go
160
Running MongoDB
161
❯ mongodMongoDB starting : pid=51054 port=27017 dbpath=/data/db
Running MongoDB
162
❯ mongo
Mongo Shell
163
❯ go run dagobah.go fetch --rsstimeout=1 15 new item(s) in http://spf13.com/index.xml 1 new channel(s) in http://spf13.com/index.xml Sleeping for 300 seconds on http://spf13.com/index.xml 25 new item(s) in http://www.goinggo.net/feeds/posts/default 1 new channel(s) in http://www.goinggo.net/feeds/posts/default Sleeping for 300 seconds on http://www.goinggo.net/feeds/posts/default 10 new item(s) in http://dave.cheney.net/feed 1 new channel(s) in http://dave.cheney.net/feed Sleeping for 299 seconds on http://dave.cheney.net/feed
Running planet
164
> use dagobah > db.items.findOne() { "_id" : ObjectId("53b4c08bddbc460a933cf3ed"), "date" : ISODate("2014-07-01T00:00:00Z"), "key" : "http://spf13.com/post/go-pointers-vs-references", "channelkey" : "Recent Content on Hacking Management", "title" : "Pointers vs References", "links" : [ { "href" : "http://spf13.com/post/go-pointers-vs-references", "rel" : "", "type" : "", "hreflang" : ""
Looking at the Data
165
_ID
• MongoDB’s primary key • Default is to use an “object_id” • Object_id guaranteed to be unique across your entire cluster
• MongoDB will create it for you on insert 166
https://github.com/spf13/
firstGoApp-Planet/tree/
step4168
Step 5
Building a web Server
169
Lots of choices• BeegoHigh-performance web framework
• GinMartini like with better performance
• GojiA minimalistic web framework
• Gorilla A web toolkit
• httprouter A high performance router
• Mango modular web-app framework like Rack
• Martini modular web apps & services
• net/httpStandard library web package
• pat Sinatra style, by the author of Sinatra.
• RevelA high-productivity web framework
• tigertonic framework for JSON web services
• traffic Sinatra inspired web framework for Go
• web.go A simple framework to write webapps in Go.
170
Gin
• Compatible with net/http • Uses Httprouter (really fast) • Familiar/Friendly Interface • Works well with JSON
• Had to pick something 171
go get -u github.com/gin-gonic/gin
Getting Gin
172
boilerplate stuff
173
// Copyright © 2014 Steve Francia <spf@spf13.com>. // // Use of this source code is governed by an Apache2 // license that can be found in the LICENSE file. !package commands !import ( "github.com/gin-gonic/gin" "github.com/spf13/cobra" "github.com/spf13/viper" )
Server.Go (1)
174
Defining the Server command
175
var serverCmd = &cobra.Command{ Use: "server", Short: "Server for feeds", Long: `Dagobah will serve all feeds listed in the config file.`, Run: serverRun, } !func init() { serverCmd.Flags().Int("port", 1138, "Port to run Dagobah server on") viper.BindPFlag("port", serverCmd.Flags().Lookup("port")) }
server.go (2)
176
func addCommands() { RootCmd.AddCommand(fetchCmd) RootCmd.AddCommand(serverCmd) }
planet.go
177
Defining our first route
178
func serverRun(cmd *cobra.Command, args []string) { r := gin.Default() !
r.GET("/ping", func(c *gin.Context) { c.String(200, "pong") }) !
port := viper.GetString("port") fmt.Println("Running on port:", port) r.Run(":" + port) }
server.go (3)
179
❯ go run dagobah.go server
Running it
180
Check it outhttp://localhost:1138 181
Tweet Break
I wrote a webserver in go using Gin with @spf13 at #OSCON http://j.mp/gin-go
https://github.com/spf13/
firstGoApp-Planet/tree/
step5183
Step 6
Building a Dynamic Server
184
Binary Problem?
• Go ships as a single binary • Need to load files from within the binary • Don’t want to embed by hand
185
go.rice
• Nicely allows local loads in dev, and embedded loads when executing binary
• Stable, but not very mature yet • Best option I know of
186
go get -u github.com/GeertJohan/go.rice
Getting Go.Rice
187
Serving Static Files
188
Get your static &
templates on
c.spf13.com/OSCON/step6-static.zip189
import ( "fmt" "html/template" "log" "net/http" ! "github.com/GeertJohan/go.rice" "github.com/gin-gonic/gin" "github.com/spf13/cobra" "github.com/spf13/viper" )
server.go
190
func serverRun(cmd *cobra.Command, args []string) { ... r.GET("/static/*filepath", staticServe) !
port := viper.GetString("port") fmt.Println("Running on port:", port) r.Run(":" + port) }
server.go
191
func staticServe(c *gin.Context) { static, err := rice.FindBox("static") if err != nil { log.Fatal(err) } ! original := c.Request.URL.Path c.Request.URL.Path = c.Params.ByName("filepath") fmt.Println(c.Params.ByName("filepath")) http.FileServer(static.HTTPBox()).ServeHTTP(c.Writer, c.Request) c.Request.URL.Path = original }
server.go
192
Loading & Serving Templates
193
func loadTemplates(list ...string) *template.Template { templateBox, err := rice.FindBox("templates") if err != nil { log.Fatal(err) } templates := template.New("") ! for _, x := range list { templateString, err := templateBox.String(x) if err != nil { log.Fatal(err) } ! // get file contents as string _, err = templates.New(x).Parse(templateString) if err != nil { log.Fatal(err) } } return templates }
server.go
194
func serverRun(cmd *cobra.Command, args []string) { r := gin.Default() templates := loadTemplates("full.html") r.SetHTMLTemplate(templates) ! r.GET("/ping", func(c *gin.Context) { c.String(200, "pong") }) ! r.GET("/", homeRoute) r.GET("/static/*filepath", staticServe)! port := viper.GetString("port") fmt.Println("Running on port:", port) r.Run(":" + port) }
server.go
195
func homeRoute(c *gin.Context) { obj := gin.H{"title": "Go Rules"} c.HTML(200, “full.html", obj) }
server.go
196
❯ go run dagobah.go server
Running it
197
Check it outhttp://localhost:1138 198
https://github.com/spf13/
firstGoApp-Planet/tree/
step6200
Step 7
Connecting the Data to The templates
201
Go Templates
• Really nice to work with • Very simple, yet very powerful • Safe • Go Template Primer
202
Understanding the data
203
> db.items.find().sort({ "date" : -1 }).limit(1).pretty() { "_id" : ObjectId("53b4c08bddbc460a933cf3ed"), "date" : ISODate("2014-07-01T00:00:00Z"), "key" : "http://spf13.com/post/go-pointers-vs-references", "channelkey" : "Recent Content on Hacking Management", "title" : "Pointers vs References", "links" : [ { "href" : "http://spf13.com/post/go-pointers-vs-references", "rel" : "", "type" : "", "hreflang" : "" }
Looking at the Schema
204
Listing POsts
205
passing data into the template
• Go’s templates accept a variety of data types • Gin’s default is ‘H’ : map[string]interface{} • Feels natural • All (exported) methods bound to values in the map are accessible inside the template
206
func homeRoute(c *gin.Context) { var posts []Itm results := Items().Find(bson.M{}).Sort("-date").Limit(20) results.All(&posts) ! obj := gin.H{"title": "Go Rules", "posts": posts} c.HTML(200, "full.html", obj) }
server.go
207
... <section id="latest-list" class="ui divided inbox selection list active tab" data-tab="recent"> {{ range $item := .posts }} {{ if $item.WorthShowing }} <a class="item" href=“#{{$item.Key | urlquery }}"> <div class="description">{{ $item.Title }}</div> <div class="right floated ui label small">{{ $item.Date.Format "Jan 2, 2006" }}</div> </a> {{ end }} {{ end }} </section> ...
Full.html
208
Adding Helpers to itm
209
func (i Itm) FirstLink() (link rss.Link) { if len(i.Links) == 0 || i.Links[0] == nil { return } return *i.Links[0] } !func (i Itm) WorthShowing() bool { if len(i.FullContent) > 100 { return true } return false }
fetch.go
210
displaying POsts
211
... <main> {{ with .message}}! <h1 class="ui header large"> {{.}} </h1>! {{ end }}!! {{ range $item := .posts }}! {{ if $item.WorthShowing }}! <article data-key="{{$item.Key}}">! <header>! <a name="{{$item.Key}}">! <h1 class="ui header">{{$item.Title}}</h1>! <section class="meta-tags">! <a class="ui label large blue author" href="/channel/{{$item.ChannelKey}}">{{$item.Author.Name}}</a>! <span class="large ui label date">{{ $item.Date.Format "Jan 2, 2006" }}</span>! </section>! </header>! <section class="main-content">! {{$item.FullContent | html }}! </section>! <footer>! {{with $item.FirstLink}}! <a class="ui basic button" href="{{.Href}}">Source</a>! {{end}}! <div class="ui divider"></div>! </footer>! </article>! {{ end }}! {{ else }}! No Articles! {{ end }}! </main> ...
full.html
212
Adding Template Functions
213
func loadTemplates(list ...string) *template.Template { templateBox, err := rice.FindBox("templates") if err != nil { log.Fatal(err) } ! templates := template.New("") ! for _, x := range list { templateString, err := templateBox.String(x) if err != nil { log.Fatal(err) } ! // get file contents as string _, err = templates.New(x).Parse(templateString) if err != nil { log.Fatal(err) } }
! funcMap := template.FuncMap{ "html": ProperHtml, "title": func(a string) string { return strings.Title(a) }, } ! templates.Funcs(funcMap) ! return templates }
server.go
214
func ProperHtml(text string) template.HTML { if strings.Contains(text, "content:encoded>") || strings.Contains(text, "content/:encoded>") { text = html.UnescapeString(text) } return template.HTML(html.UnescapeString(template.HTMLEscapeString(text))) }
server.go
215
Looking Goodhttp://localhost:1138 216
Tweet Break
Conquering Go Templates with @spf13 at #OSCON
https://github.com/spf13/
firstGoApp-Planet/tree/
step7218
Step 8
More Routes219
Adding Helpers
220
func (c Chnl) HomePage() string { if len(c.Links) == 0 { return "" } !
url, err := url.Parse(c.Links[0].Href) if err != nil { log.Println(err) } return url.Scheme + "://" + url.Host }
fetch.go
221
!func four04(c *gin.Context, message string) { c.HTML(404, "full.html", gin.H{"message": message, "title": message}) }
server.go
222
func AllChannels() []Chnl { var channels []Chnl r := Channels().Find(bson.M{}).Sort("-lastbuilddate") r.All(&channels) return channels }
MongoDB.go
223
Listing channels
224
func homeRoute(c *gin.Context) { var posts []Itm results := Items().Find(bson.M{}).Sort("-date").Limit(20) results.All(&posts) ! obj := gin.H{"title": "Go Rules", "posts": posts, "channels": AllChannels()} c.HTML(200, "full.html", obj) }
server.go
225
... <section id="channel-list" class="ui large inverted vertical menu tab" data-tab=“channels"> <a class="item js-navigate" href="/" >! <i class="external home icon"></i>! <span class="ui name" href=""> All </span>! </a>! {{ range $channel := .channels }}! <div class="item js-navigate-channel" data-href="/channel/{{$channel.Key}}" data-key="{{$channel.Key}}" style="cursor:pointer;">! <a href="{{$channel.HomePage}}" class="ui float right"><i class="external url sign icon"></i></a>! <span class="ui name" href=""> {{$channel.Title}} </span>! <span class="ui label">{{ len $channel.Links }}</span>! </div>! {{end}}! </section> ...
full.html
226
displaying channels
227
func serverRun(cmd *cobra.Command, args []string) { ... r.GET("/channel/*key", channelRoute) r.Run(":" + port) }
server.go
228
func channelRoute(c *gin.Context) { key := c.Params.ByName("key") if len(key) < 2 { four04(c, "Channel Not Found") return } ! key = key[1:] ! fmt.Println(key) ! var posts []Itm results := Items().Find(bson.M{"channelkey": key}).Sort("-date").Limit(20) results.All(&posts) ! if len(posts) == 0 { four04(c, "No Articles") return } ...
server.go - pt 1
229
... var currentChannel Chnl err := Channels().Find(bson.M{"key": key}).One(¤tChannel) if err != nil { if string(err.Error()) == "not found" { four04(c, "Channel not found") return } else { fmt.Println(err) } } ! obj := gin.H{"title": currentChannel.Title, "header": currentChannel.Title, "posts": posts, "channels": AllChannels()} ! c.HTML(200, "full.html", obj) }
server.go - pt 2
230
displaying A post
231
func serverRun(cmd *cobra.Command, args []string) { ... r.GET("/post/*key", channelRoute) r.Run(":" + port) }
server.go
232
func postRoute(c *gin.Context) { key := c.Params.ByName("key") ! if len(key) < 2 { four04(c, "Invalid Post") return } ! key = key[1:] ! var ps []Itm r := Items().Find(bson.M{"key": key}).Sort("-date").Limit(1) r.All(&ps) ! if len(ps) == 0 { four04(c, "Post not found") return } ...
server.go - Pt 1
233
...! var posts []Itm results := Items().Find(bson.M{"date": bson.M{"$lte": ps[0].Date}}).Sort("-date").Limit(20) results.All(&posts) ! obj := gin.H{"title": ps[0].Title, "posts": posts, "channels": AllChannels()} ! c.HTML(200, "full.html", obj) }
server.go - Pt 2
234
Looking Goodhttp://localhost:1138 235
Tweet Break
Making my own planet in #go with @spf13 at #OSCON http://j.mp/go-planet
https://github.com/spf13/
firstGoApp-Planet/tree/
step8237
Step 9
Advanced Routes238
Congrats
• You’ve made it really far • Now the training wheels are off • Let’s try to add pagination & search
240
Hints
•You will need a new index •We’re talking full text here
•You will also need a new route241
... r.GET("/", homeRoute) r.GET("/post/*key", postRoute) r.GET("/search/*query", searchRoute) r.GET("/static/*filepath", staticServe) r.GET("/channel/*key", channelRoute) ...
server.go
242
Step 10 !
Add Polish243
What’s next ?
• Add ‘truncate’ command to remove old posts and channels
• Change root command to run fetcher & server
• Add infinite scroll capabilities 244
What’s next ?
• Make it so users can provide their own static files & templates
• Documentation • Blog posts
245
In Conclusion
246Go mascot designed by Renée French and copyrighted under the Creative Commons Attribution 3.0 license.
Thank yOU247
What have we done ?
• Written our first lines of Go
• Written our first Go package
• Written our own web server
• Written our first Go application
• Learned a lot & had fun doing it 248
@spf13
• Author of Hugo, Cobra, Viper & More
• Chief Developer Advocate for MongoDB
•Gopher 249
Recommended