The Heroku Hacker’s GuideRelease 1.0
Randall Degges
September 08, 2012
CONTENTS
1 Preface 1
2 Why Heroku? 32.1 Simplicity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.2 Age . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.3 Popularity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.4 Polyglot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42.5 Solid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42.6 Best Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3 Getting Started with Heroku 73.1 Create an Account . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73.2 Install the Toolbelt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73.3 Install heroku-accounts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73.4 Project Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93.5 Create an Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93.6 Follow the Required Quickstart Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.7 Push Your Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.8 Check it Out . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
4 Dynosaurs 114.1 What Are Dynos? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114.2 Understanding Dynos . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114.3 Dynos Run Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114.4 One Procfile to Rule Them All . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124.5 Testing Your Procfile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124.6 Scaling Up and Scaling Down Dynos . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134.7 Handling Failure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144.8 Calculating Dyno Costs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
5 The Environment 155.1 Best Practices and You . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155.2 The Config Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155.3 Benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
6 Take the Pain Away (with Addons) 176.1 What are Addons? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176.2 How do Addons Work? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176.3 The Addon Catalogue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176.4 Adding, Removing, Upgrading, and Downgrading Addons . . . . . . . . . . . . . . . . . . . . . . . 18
i
6.5 Addon Cost . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
7 PostgreSQL Patterns 217.1 Why Heroku PostgreSQL? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217.2 Bootstrapping a Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217.3 Connecting to Your Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227.4 Destroying a Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237.5 Creating Read Slaves . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237.6 Creating a Duplicate Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247.7 Promoting a Slave Database to a Master Database . . . . . . . . . . . . . . . . . . . . . . . . . . . 247.8 View Slow Queries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257.9 Backing Up Your Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257.10 Downloading Your Backups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267.11 Restoring From a Backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277.12 Final Thoughts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
8 Caching with Memcached 298.1 Why Cache? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298.2 Using Memcached . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298.3 Memcached on Heroku . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
9 Scheduling Tasks with Cron 319.1 The Scheduler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319.2 Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329.3 Cost . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
10 Logging 3310.1 Log Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3310.2 Viewing Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3310.3 Viewing Select Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3410.4 When to Check Your Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3410.5 Log Storage Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
11 Monitoring with New Relic 3511.1 Why New Relic? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3511.2 Installing New Relic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3611.3 The Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3611.4 A Visual Map of Your Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3711.5 Web Transactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3811.6 Database Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3811.7 External Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3911.8 Dyno Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4011.9 Background Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4111.10 Final Thoughts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
12 Talking to the World 4312.1 Using Custom Domains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4312.2 Updating Your DNS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4312.3 Encrypt All the Things! (with SSL) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
13 Managing Releases 4513.1 Heroku’s Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4513.2 Viewing Your Releases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4513.3 Rolling Back . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4613.4 Final Thoughts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
ii
14 Working with Others 4714.1 Who is the Application Owner? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4714.2 Sharing Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4714.3 Sharing Permissions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4714.4 Managing Collaborators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4814.5 Transferring Ownership . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
15 Do Great Things 4915.1 Build Services, Not Apps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4915.2 Building Service Oriented Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5015.3 Heroku and Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5015.4 Be Dynamic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5015.5 Final Thoughts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
16 References and Further Reading 53
17 Special Thanks 55
iii
iv
CHAPTER
ONE
PREFACE
In the past several years, “the cloud” has really taken off. More and more companies (startups, side projects, and evenlarge enterprises) have decided to move their infrastructure to the cloud in order to reduce cost and complexity whilesimultaneously gaining availability, redundancy, and speed.
Unfortunately, while amazing cloud platforms like Amazon Web Services (http://aws.amazon.com/) have maderunning your company in the cloud affordable, they haven’t exactly made it simple. This is where Heroku(http://www.heroku.com/) comes in.
Heroku is a new type of cloud platform. Instead of renting virtual servers and installing your own software on them,Heroku handles the heavy lifting for you–all you do is push your code (using the Git (http://git-scm.com/) versioncontrol system) to Heroku, and they handle the hard stuff:
• Running your application across a specified number of virtual servers.
• Handling releases: rolling back, rolling forward, storing environment variables, etc.
• Ensuring your application automatically recovers from server failures.
• Handling load balancing across many application instances allowing you to instantly scale your application tosupport millions of users.
• Allowing you to instantly add and remove infrastructure components (databases, caching servers, etc.).
• And lots more.
In my six years of writing, deploying, and managing high performance web applications, I’ve spent thousands of hoursbuilding and maintaining the stuff that Heroku handles instantly out of the box.
Whenever I launch code, whether it be a hobby project, work project, or startup–I instantly turn to Heroku.
My goal with this book is to teach you how to use Heroku successfully. Using a new platform is always tricky at first,but as you read through this book you’ll learn all you need to know about Heroku’s platform.
This book is for you, the hackers.
Thanks for buying The Heroku Hacker’s Guide (http://www.theherokuhackersguide.com/), and I hope you enjoy it!
1
The Heroku Hacker’s Guide, Release 1.0
2 Chapter 1. Preface
CHAPTER
TWO
WHY HEROKU?
Before we get into the fun stuff, I thought I’d take a moment to explain why you should be using Heroku in the firstplace. After all–there are a ton of cloud hosting platforms available, why use Heroku?
2.1 Simplicity
Out of all the cloud hosting options, Heroku is the simplest. Heroku is built around the Unix philosophy(http://en.wikipedia.org/wiki/Unix_philosophy), which states that you should do one thing, and do it well.
To deploy your application on Heroku, all you need to do is push your Git repository to their servers and bam–yourcode is instantly deployed across as many virtual instances as you want.
Need specific infrastructure components? Databases? Caching servers? Key value stores? No problem, you caninstantly provision them and attach them to your application.
Because Heroku is a simple, modular platform–you’ll spend less time handling infrastructure, redundancy, deploy-ment, etc.–and more time hacking code.
2.2 Age
Heroku has been around since 2007. They were one of the first cloud platforms ever built.
Since cloud computing (as we now know it) is relatively new (Amazon only launched its extremely popular webservices product in 2006), this means that Heroku has been in the business for a long time–much longer than most ofthe other guys.
Furthermore, Heroku is owned by Salesforce.com (http://www.salesforce.com/), one of the oldest and largest cloudenterprise companies. Heroku won’t be going away anytime soon, and you can feel safe putting your business criticalapplications on Heroku, knowing they won’t be disappearing overnight.
2.3 Popularity
In addition to being around a long time–Heroku is also one of the most popular choices for cloud hosting. Chancesare that if you’re a web programmer, you’ve either heard of or used Herkou in the past.
Heroku’s popularity amongst programmers means that you can easily find help from experienced developers who arefamiliar with Heroku’s platform. This makes hiring help a lot easier, and ensures that regardless of whether you’re juststarting out, or growing to support millions of users–you’ll be able to find an abundance of quality talent.
3
The Heroku Hacker’s Guide, Release 1.0
2.4 Polyglot
One of the best reasons to use Heroku is that they probably support your programming language (and web framework)of choice. Heroku touts itself as a polyglot platform–and they truly are.
At the time of writing, Herkou currently supports the following programming languages:
• Clojure
• Java
• Python
• Scala
• Javascript
• Ruby
• Go
• PHP
• And many more.
This means that regardless of what technologies you choose, you can feel comfortable knowing that even if youlaunch projects in different programming languages, you’ll be able to use the same familiar platform to deploy allyour applications.
2.5 Solid
Heroku is built on top of Amazon Web Services–one of the largest, oldest, and most widely used cloud platforms.Amazon Web Services is known for its speed, availability, and flexibility.
While Heroku does abstract away all the nasty bits, knowing that your applications are hosted on one of the fastest andlargest platforms behind the scenes is a good thing to keep in mind.
2.6 Best Practices
The last thing I’d like to bring up is best practices. One of the greatest things about Heroku is that it is built around theconcept of 12 factor design (http://www.12factor.net/).
What this means is that the platform itself encourages (and enforces) a best practices approach to software develop-ment:
• Your application should be driven by version control.
• You should explicitly declare and isolate your dependencies.
• You should store your application configuration in environment variables.
• You should treat necessary infrastructure components as attachable resources.
• You should separate build and run stages.
• Your application should be stateless, so it can be scaled across many instances.
• You should export your services via port bindings.
• You should scale out your application via concurrency.
4 Chapter 2. Why Heroku?
The Heroku Hacker’s Guide, Release 1.0
• You should have fast start up and fast shut down.
• Development, staging, and production should be as similar as possible.
• You should treat your logs as event streams.
• You should run admin tasks as one-off processes.
This is a good thing for many reasons–but most importantly, since Heroku’s platform is built to encourage this sort ofdesign–your applications will inherently be much more robust, scalable, and simple.
2.6. Best Practices 5
The Heroku Hacker’s Guide, Release 1.0
6 Chapter 2. Why Heroku?
CHAPTER
THREE
GETTING STARTED WITH HEROKU
This chapter will walk you through all the Heroku basics. You’ll set up your local machine for working with Heroku,create your first application, and learn a bit about the platform along the way!
3.1 Create an Account
The first thing you need to do is create your very own Heroku account. Do this now: https://api.heroku.com/signup.
I’d recommend creating several separate Heroku accounts–one for yourself (a personal account), and one for eachcompany you plan to write applications for.
Having separate accounts will allow you to:
• Handle billing separately for each account.
• Group your applications together.
• Easily grant and revoke contributor access to applications on a per-application basis (more on this later).
• See which users are pushing code to which applications.
3.2 Install the Toolbelt
The preferred way to work with Heroku is through their command line utilities. The Heroku Toolbelt(https://toolbelt.heroku.com/) works with Debian / Ubuntu, Mac OS X, and Windows operating systems.
What this gets you is three command line utilities:
• The Heroku client, a CLI tool for creating and managing your Heroku applications.
• Foreman, a simple tool which emulates Heroku’s behavior locally.
• Git, the version control system that powers everything.
To install the Herkou Toolbelt, visit https://toolbelt.heroku.com/ and follow the instructions for your operating system.
3.3 Install heroku-accounts
The heroku-accounts (https://github.com/ddollar/heroku-accounts) plugin is a great utility that makes working withmultiple Heroku accounts a lot easier. It allows you to do things like:
• Create new Heroku applications on your work account.
7
The Heroku Hacker’s Guide, Release 1.0
• Build private applications on your personal account.
• Add new Heroku accounts in a flash.
• Easily switch back and fourth amongst your accounts.
• Handle multiple SSH keys for your accounts almost transparently.
To install heroku-accounts, do the following:
$ heroku plugins:install git://github.com/ddollar/heroku-accounts.githeroku-accounts installed
Once you’ve gotten heroku-accounts installed, you should then be able to run heroku help accounts to see usageinformation:
$ heroku help accountsUsage: heroku accounts
list all known accounts
Additional commands, type "heroku help COMMAND" for more details:
accounts:add # accounts:addaccounts:default # accounts:defaultaccounts:remove # accounts:removeaccounts:set # accounts:set
Now, let’s go ahead and configure your personal account that you created at the top of this chapter:
$ heroku accounts:add rdegges --autoEnter your Heroku credentials.Email: [email protected] (typing will be hidden):Generating new SSH keyGenerating public/private rsa key pair.Created directory ’/home/rdegges/.ssh’.Your identification has been saved in /home/rdegges/.ssh/identity.heroku.rdegges.Your public key has been saved in /home/rdegges/.ssh/identity.heroku.rdegges.pub.The key fingerprint is:c3:b5:59:7f:4b:b5:c7:fb:59:eb:c6:c8:af:ac:7f:da rdegges@baalThe key’s randomart image is:+--[ RSA 2048]----+| || || . . .|| . . + . .o|| S o .o+|| . ..+|| . oo.|| .o.+=|| .o=BE.|+-----------------+Adding entry to ~/.ssh/configAdding public key to Heroku account: [email protected]
The above command created a new local rdegges account, automatically generated an SSH key for it, and uploadedit to Heroku. Since rdegges is my personal account, I’ll now be able to create applications on my personal account,push code, etc.
8 Chapter 3. Getting Started with Heroku
The Heroku Hacker’s Guide, Release 1.0
If you have any other accounts (work accounts, whatever), create them now using the heroku accounts:add<account name> --auto command.
Once you’ve created all your accounts, you can list them at any time by running:
$ heroku accountsrdeggesmycompany1mycompany2
Furthermore, you’ll also probably want to set one of your accounts as the default on your system. To do this, just run:
$ heroku accounts:default rdegges$ heroku accounts
* rdeggesmycompany1mycompany2
As you can see, the default account has a star (*) next to it, to help you quickly identify it. The default account will beused whenever you run any local heroku commands, if no other account is specified.
3.4 Project Prerequisites
Now that we’ve gotten the basic tools installed, let’s talk about projects.
Through the rest of this book you’ll be learning how Heroku works, how to use it, and the best patterns for using iteffectively. While it isn’t required that you have a project ready to deploy, if you do, you’ll be able to follow alongwith many of the examples as we go and may get more out of this book.
If you do have a project you want to use through the duration of the book, now’s a good time to set it up. The onlyrequirement for your project is that it must be version controlled with Git.
3.5 Create an Application
Assuming you already have a project setup with Git, let’s prepare it for deployment on Heroku.
First, go into your project’s root directory:
$ cd ~/myproject
Next, specify which Heroku account you’d like to create your new application under:
$ heroku accounts:set rdegges
The above command tells the heroku command line tool that from now on, any Heroku commands you run whileinside your project directory should be run under your personal account (rdegges, in my case).
Now that you’ve specified your Heroku account, go ahead and create your Heroku application:
$ heroku create myproject$ heroku apps=== My Appsmyproject
=== Collaborated Apps
3.4. Project Prerequisites 9
The Heroku Hacker’s Guide, Release 1.0
Note: If you don’t specify a name when you run the heroku create <project> command, Heroku willautomatically assign your application a name.
Congratulations, you’ve created your first Heroku application!
3.6 Follow the Required Quickstart Guide
If you got this far, you’ll want to read through the official Heroku Quickstart(https://devcenter.heroku.com/articles/quickstart) guide before continuing. This quickstart guide will walk youthrough bootstrapping your application on Heroku. Since these instructions are different for each programminglanguage (and each framework), I’ll leave this as a reader exercise.
The official quickstart guide will walk you through installing langauge specific dependencies, discuss the details ofeach language, and show you exactly what to do in order to make sure everything works as expected.
The rest of this book assumes you’ve been through the quickstart.
3.7 Push Your Code
Now that you’ve created a Heroku application and been through the quickstart, the next step is to get your code ontoHeroku’s servers.
When you created your application in the previous section, Heroku automatically created a new Git remote(http://gitref.org/remotes/) named heroku. The way you push your code to Heroku (and subsequently deploy yourapplication live) is to do a simple Git push to your new private Heroku Git repository:
$ cd ~/myproject$ git push heroku master... output showing that Heorku received your code ...
If you see errors, you’ll need to return to the Heroku Quickstart (https://devcenter.heroku.com/articles/quickstart)guide and figure out what went wrong. Assuming everything worked, you’re now in business!
3.8 Check it Out
Once you’ve pushed your application to Heroku, you’ll most likely want to check it out and make sure the basics areworking.
Each Heroku application is assigned a unique URL, which automatically routes incoming HTTP requests to yourapplication. Your URL will be http://myproject.herokuapp.com, where myproject is your Heroku applicaton’sname. To see your application in action, you can visit this URL.
We’ll discuss using your own domain names, and encrypting your HTTP traffic (SSL!) in a later chapter.
10 Chapter 3. Getting Started with Heroku
CHAPTER
FOUR
DYNOSAURS
At the core of Heroku’s platform are dynos (https://devcenter.heroku.com/articles/dynos). This chapter will explaineverything you’ve ever wanted to know about dynos, and then some.
4.1 What Are Dynos?
Dynos are to Heroku what virtual servers are to every other cloud hosting provider you’ve probably heard of.
Essentially–a dyno is a single, secure instance of your application that runs across Heroku’s servers. Dynos are inreality just Ubuntu 10.04 virtual server instances that can be spun up and down on demand. Each dyno instance has(at the time of writing):
• 512MB of RAM, 1GB of swap.
• 4 CPU cores (Intel Xeon X5550 @ 2.67GHz).
• Isolated execution. Anything you store on your dyno will be isolated from all other dynos.
• A chroot jail (http://en.wikipedia.org/wiki/Jail_(computer_security)) environment. This means that you arecompletely locked down to one directory tree, with no write access to system files.
• Per-second billing. You get one dyno for free each month (per app). If you use a dyno for 2 seconds, you’ll payfor two seconds. If you use a dyno for an entire month solid, you’ll pay for the entire month.
4.2 Understanding Dynos
Since Heroku dynos are nothing more than virtualized Ubuntu server instances, this means that as long as your appli-cation can run on Ubuntu Linux, it can run on top of Heroku’s stack.
This also means that almost anything you can do on Linux, you can do on Heroku: use bash(http://en.wikipedia.org/wiki/Bash_(Unix_shell)), list files, whatever:
For almost all purposes, you can generally think of dynos as small virtual servers that you can scale up and downinstantly.
4.3 Dynos Run Services
When building modern web applications, you typically need multiple services running. In any given application, youmay have:
11
The Heroku Hacker’s Guide, Release 1.0
• A web instance running that serves your HTTP requests.
• A worker instance running that processes asynchronous jobs.
• A scheduler instance running that regularly schedules worker tasks to be ran.
Luckily, since dynos are really just virtual server instances, they can run any type of service for you easily.
For our example above, we’d need at least one:
• web dyno.
• worker dyno.
• scheduler dyno.
Which leads us to our next section...
4.4 One Procfile to Rule Them All
In order to instruct our Heroku dynos how to run our various services (web, worker, and scheduler), we needto tell Heroku what commands to run. The way we tell Heroku what commands to run is by defining a Procfile(https://devcenter.heroku.com/articles/procfile) in the root of our project directory.
Let’s say we’re trying to deploy a Flask (http://flask.pocoo.org/) web application and its web, worker, andscheduler services. We’d define a Procfile that looks like this:
# ~/myproject/Procfileweb: gunicorn -b ’0.0.0.0:$PORT’ myproject:appworker: python manage.py celerydscheduler: python manage.py celeryd -B
The format of the Procfile is simple. You define one type of dyno on each line. The part on the left is your dynoname (in this case, we have three types of dynos: web, worker, and scheduler), and the part on the right (afterthe colon) is the command that Heroku needs to run in order to start your service.
In the example above, I needed to run gunicorn myproject:app from the command line in order to start myweb service, so that’s why I defined my web dyno by writing:
web: gunicorn --bind ’0.0.0.0:$PORT’ myproject:app
If my project only needed a web dyno, I could just define a single line in my Procfile.
Note: In my web example I told my web server to bind itself to 0.0.0.0:$PORT. This is because Heroku auto-matically assigns a random port number to each dyno, so your application must use the special variable $PORT, whichHeroku will automatically substitute out when running your service.
4.5 Testing Your Procfile
To make sure your Procfile works as you expect, it’s really best to test it out locally. Luckily, the Heroku Toolbelt(https://toolbelt.heroku.com/) package you installed in the Getting Started with Heroku section shipped with a reallyhandy command line tool, foreman, which does exactly that!
Once you’ve defined your Procfile in the root of your project, you can test it works as expected by running:
12 Chapter 4. Dynosaurs
The Heroku Hacker’s Guide, Release 1.0
$ cd ~/myproject$ foreman start18:06:23 web.1 | started with pid 4721918:06:23 worker.1 | started with pid 4722018:06:23 scheduler.1 | started with pid 4722118:06:23 web.1 | => Output from your web process.18:06:23 worker.1 | => Output from your worker process.18:06:23 scheduler.1 | => Output from your scheduler process.
If your application doesn’t work, you’ll know you’ve messed something up. Since foreman outputs all standardoutput from each of your processes to the terminal, you should be able to easily debug any unexpected problems.
Using foreman is also a great way to run your application locally while in development. For complex applicationsthat require multiple services to be running at any given time, defining a simple Procfile and using foreman is agreat way to manage your application processes locally as you build your project. In comparison to the usual workflow(start multiple services in separate terminal windows with multiple commands), foreman is quite simple.
4.6 Scaling Up and Scaling Down Dynos
One of Heroku’s greatest strengths is the ability to instantly scale the amount of dynos you have running up or downat any time.
Let’s say that your application is able to serve a total of 5 requests per second. If you wanted to support more users,you’d need to do one of two things:
• Either optimize your application code, or
• Run another instance of your web dyno (and possibly other worker and scheduler instances as well).
You might be wondering how running multiple web dynos can help you here. After all–if Heroku is routing incomingHTTP requests to your web dyno, how can having two dynos running help?
The answer, of course, is that Heroku is magically load balancing all incoming HTTP requests for you automatically,regardless of how many dynos you have running!
What this means is that if you tell Heroku to run 10 dynos, some incoming HTTP requests will hit your web.1 dyno,some will hit your web.2 dyno, etc.
So now the only question is how can you actually add and remove dynos? Using the Heroku ps and scale com-mands:
$ cd ~/myproject$ heroku ps=== web: ‘gunicorn --bind ’0.0.0.0:$PORT’ myproject:app‘web.1 idle since 2012/08/27 10:12:22
$ herkou scale web=10 scheduler=1 worker=5Scaling web processes... done, now running 10Scaling scheduler processes... done, now running 1Scaling worker processes... done, now running 5
$ heroku ps=== web: ‘gunicorn --bind ’0.0.0.0:$PORT’ myproject:app‘web.1 up for 1mweb.2 up for 1mweb.3 up for 1mweb.4 up for 1mweb.5 up for 1m
4.6. Scaling Up and Scaling Down Dynos 13
The Heroku Hacker’s Guide, Release 1.0
web.6 up for 1mweb.7 up for 1mweb.8 up for 1mweb.9 up for 1mweb.10 up for 1m
=== scheduler: ‘python manage.py celeryd -B‘scheduler.1 up for 1m
=== worker: ‘python manage.py celeryd‘worker.1 up for 1mworker.2 up for 1mworker.3 up for 1mworker.4 up for 1mworker.5 up for 1m
Simple, right? As you can see above, the heroku ps command lists all your active Heroku processes (dynos), whilethe heroku scale command lets you pick and choose how many dynos you’d like to run.
Of course, you can also scale down your dynos just as easily:
$ heroku scale web=1 scheduler=0 worker=0Scaling web processes... done, now running 1Scaling scheduler processes... done, now running 0Scaling worker processes... done, now running 0
$ heroku ps=== web: ‘gunicorn --bind ’0.0.0.0:$PORT’ myproject:app‘web.1 up for 1m
Using Heroku’s scale command, you can instantly scale any application up from a single dyno to a hundred in amatter of seconds.
This is particularly useful if your software has lots of burst traffic, as you can easily scale up to support more users,and down afterwards to save on operational cost.
4.7 Handling Failure
In the event a Heroku server running your dyno(s) fails, you have nothing to worry about, as Heroku will automaticallymove your dynos to different hosts.
Furthermore, when your dynos are being moved around, restarted, or anything in between, Heroku automaticallybuffers any incoming HTTP requests at the load balancer level to ensure no requests are dropped.
4.8 Calculating Dyno Costs
Heroku has a really nifty pricing calculator on their website, which you can find here:http://www.heroku.com/pricing#0-0.
If you need to do capacity planning, cost projections, or anything of the sort, you should be using their pricingcalculator, as it will give you a really good idea of how much your application hosting will cost each month.
14 Chapter 4. Dynosaurs
CHAPTER
FIVE
THE ENVIRONMENT
All applications require some form of configuration, whether it be API credentials, passwords, or anything in between.This chapter explains the best way to keep your configuration information safe, and decoupled from your codebase asmuch as possible.
5.1 Best Practices and You
The third factor of the 12 factor app is configuration. To quote from the 12 factor app (http://www.12factor.net/config):
An app’s config is everything that is likely to vary between deploys (staging, production, developer envi-ronments, etc). This includes:
• Resource handles to the database, Memcached, and other backing services.
• Credentials to external services such as Amazon S3 or Twitter.
• Per-deploy values such as the canonical hostname for the deploy.
The main idea is that in order to maintain a flexible, simple, and reusable codebase as much as possible, you shouldalways strictly separate your code from configuration. This means no hardcoding constants in your code.
To encourage this practice, Heroku provides a simple config command that allows you to instantly add, edit, andremove environment variables across all your dynos.
5.2 The Config Command
Let’s say you have a web application on Heroku that requires a GitHub API key to function. Instead of hardcodingyour API key into your code, you can use the heroku config command to store your API key as an environmentvariable:
$ cd ~/myproject$ heroku config:set GITHUB_API_KEY=blahSetting config vars and restarting myproject... done, v21GITHUB_API_KEY: blah
$ heroku config=== myproject Config Vars...GITHUB_API_KEY: blah
15
The Heroku Hacker’s Guide, Release 1.0
As you can see above, running the heroku config:set GITHUB_API_KEY=blah command tells Heroku tostore a new environment variable, GITHUB_API_KEY, as well as restart your running dynos so they can start usingthe new variable.
In your application code, you can now reference your GITHUB_API_KEY value by grabbing it from the environment.If you’re writing python code, you can accomplish this using the os.environ function:
# test.pyfrom os import environprint environ.get(’GITHUB_API_KEY’)
Of course, you can also remove environment variables as well using the heroku config:unset command:
$ heroku config:unset GITHUB_API_KEYUnsetting GITHUB_API_KEY and restarting myproject.. done, v22
5.3 Benefits
In addition to making it easy to decouple your secret information from your codebase, Heroku’s environment variablehandling has other benefits as well.
For one, Heroku internally keeps a record of all your application releases (more on this in the Managing Releaseschapter). This means each time you push code to Heroku, you’ll be assigned a unique deploy number (something likev1, v2, etc.) that stores not only which version of code you’re currently running, but also the environment variablesyou currently have set (among other things).
In the event you need to look at past environment variable values, you can easily do so knowing Heroku saves a copyof them each time you push code, effectively version controlling your environment variables in a safe manner.
Secondly, using environment variables allows you to easily swap out problematic application components. Since manyservices your application relies on (database servers, caching servers, etc.) are configured via simple URLs, pullingconfiguration information from the environment gives you a great amount of flexibility.
For example, to fully configure a PostgreSQL database server (in most cases) all you need to do is point your ap-plication at the correct URL string (postgres://user:password@host:port/db). If you wanted to haveyour application use a new PostgreSQL server, you’d only need to update your environment variable containing thePostgreSQL URL and your application would instantly start using the new database server (without changing any ofyour code).
No matter what sort of applications you’re building, always store configuration information in the environment tomake your application simpler to maintain and manage in the future.
16 Chapter 5. The Environment
CHAPTER
SIX
TAKE THE PAIN AWAY (WITH ADDONS)
One of the most painful parts of software development and deployment is often managing infrastructure services:databases, caching servers, etc.
Luckily, this process doesn’t have to be painful thanks to Heroku’s amazing collection of addons(http://addons.heroku.com/).
6.1 What are Addons?
While Heroku’s core offering is a scalable on-demand application hosting platform–they don’t stop there. Heroku hasan ever increasing amount of third party (and self created) addons that any Heroku user can instantly provision asattachable resources for their applications.
What this means is that if your application needs a PostgreSQL (http://addons.heroku.com://addons.heroku.com/heroku-postgresql) database, you can easily add one. Need a Redis (https://addons.heroku.com/openredis) server? Noproblem.
Addons are attachable resources that you can spin up and down (on demand) just like dynos. There are tons of differentaddons, everything from database servers to monitoring software to web APIs.
6.2 How do Addons Work?
Addons work through Heroku’s environment variables. Each time you ‘add’ an addon to one of your applications,your application will automatically be assigned one or more environment variables which specify any credentials youmay need to interact with your new tool.
For instance, if you were to provision a Heroku PostgreSQL (http://addons.heroku.com://addons.heroku.com/heroku-postgresql) database, you’d have a new HEROKU_POSTGRESQL_BLAH_URL variable added, which your applicationcould then interact with.
Since addons interact with your application through environment variables, this makes it extremely easy to add, up-grade, downgrade, and remove addons without downtime for your applications.
6.3 The Addon Catalogue
To find which addons are currently available, visit the Heroku addon catalogue: https://addons.heroku.com/.
This page contains a listing of all available addons, as well as each addon’s documentation, pricing information, andmore. At the time of writing, Heroku has a total of 89 addons available, and the number is always growing.
17
The Heroku Hacker’s Guide, Release 1.0
6.4 Adding, Removing, Upgrading, and Downgrading Addons
Working with addons is extremely simple. Heroku comes with an addons command that allows you to do anything(and everything) you need to do.
To list the addons currently provisioned to your application, you can use the heroku addons command directly:
$ cd ~/myproject$ heroku addons=== myproject Configured Add-onsheroku-postgresql:dev HEROKU_POSTGRESQL_BROWN
As you can see above, my example application has a single heroku-postgresql:dev addon provisioned to itcurrently. All addons follow a naming pattern, which is the addon name, followed by the plan. In this case, the addonname is heroku-postgresql and the plan is dev.
To add a new addon to my project, I can simply run the heroku addons:add command:
$ heroku addons:add openredis:microAdding openredis:micro on myproject... done, v23 ($8/mo)Use ‘heroku addons:docs openredis:micro‘ to view documentation.
$ heroku config=== myproject Config Vars...OPENREDIS_URL: redis://:somepassword@somehost:someport
In the example above I provisioned the smallest openredis (https://addons.heroku.com/openredis) plan. As youcan see, once I added the addon I can immediately see a new environment variable available in my application:OPENREDIS_URL.
Now that I’ve provisioned my new Redis server, I can easily use the OPENREDIS_URL environment variable toconnect to it and start doing things.
If I now want to upgrade my Redis server to a higher plan level (for more memory, in this case), I can do so using theheroku addons:upgrade command:
$ heroku addons:upgrade openredis:mediumUpgrading to openredis:medium on myproject... done, v24 ($69/mo)Use ‘heroku addons:docs openredis:micro‘ to view documentation.
If I want to downgrade my Redis server, I can do that as well:
$ heroku addons:upgrade openredis:microUpgrading to openredis:micro on myproject... done, v25 ($8/mo)
And just incase I want to remove my Redis server completely, I can do so instantly:
$ heroku addons:remove openredis:micro! WARNING: Destructive Action! This command will affect the app: myproject! To proceed, type "myproject" or re-run this command with --confirm myproject
> myprojectRemoving openredis:micro on myproject... done, v26 ($8/mo)
18 Chapter 6. Take the Pain Away (with Addons)
The Heroku Hacker’s Guide, Release 1.0
6.5 Addon Cost
Each addon has their own cost and billing structure, and is prorated to the second. That means if you use an addon for5 seconds, you’ll be charged for 5 seconds worth of service.
All addons are billed directly through your Heroku account, which means you don’t have to do any extra work–nosetting up multiple accounts, or anything like that. If you use an addon, your usage fees are simply added to yourHeroku monthly bill.
While there isn’t a cost calculator for most addons, their pricing is always clear and straightforward.
To figure out how much you’ll be paying for those extra infrastructure services, you can navigate your way to theaddons page (see: https://addons.heroku.com/) and look at the costs listed.
6.5. Addon Cost 19
The Heroku Hacker’s Guide, Release 1.0
20 Chapter 6. Take the Pain Away (with Addons)
CHAPTER
SEVEN
POSTGRESQL PATTERNS
One of the oldest and most popular addons Heroku offers is their own PostgreSQL (https://addons.heroku.com/heroku-postgresql) addon. Since this is such a commonly used service, this section describes some best practices for workingwith heroku-postgresql to minimize your effort.
You can view more information (including database plans and pricing) on the Heroku PostgreSQL page:https://addons.heroku.com/heroku-postgresql.
7.1 Why Heroku PostgreSQL?
Not sure which database to use? Heroku’s PostgreSQL service is an excellent choice for you if:
• Your application requires a relational database for storing information.
• You don’t want to worry about downtime or data loss (since Heroku fully manages your PostgreSQL instances).
• You want the ability to scale your PostgreSQL database by:
– Creating read slaves on demand.
– Creating database snapshots for experimental testing (eg: duplicate your production database and run testsagainst the clone).
– Easily upgrading (or downgrading) the size of your database.
– Paying only for what you use (by the second) so you can instantly scale up to support burst traffic–anddown again to keep costs low.
• You want the ability to instantly backup and restore your data to any PostgreSQL database (no vendor lock-in).
• Your application is running anywhere on the internet; Heroku’s PostgreSQL service is a standalone service andworks just like any normal PostgreSQL server, so you it’s useful even if you’re not a Heroku user, but still wanta good hosted PostgreSQL solution.
Heroku PostgreSQL is the largest hosted PostgreSQL service in the world, and is an extremely safe bet for poweringapplications of any size.
7.2 Bootstrapping a Database
To get started, let’s bootstrap a new PostgreSQL server instance. In the example below, we’ll create a free (shared)PostgreSQL database:
21
The Heroku Hacker’s Guide, Release 1.0
$ cd ~/myproject$ heroku addons:add heroku-postgresql:devAdding heroku-postgresql:dev to myproject... done, v27 (free)Attached as HEROKU_POSTGRESQL_BLACK
If we wanted to create a larger (paid) database, we could run the following command:
$ heroku addons:add heroku-postgresql:craneAdding heroku-postgresql:crane to myproject... done, v28 ($50/mo)Attached as HEROKU_POSTGRESQL_IVORYThe database should be available in 3-5 minutesUse ‘heroku pg:wait‘ to track statusUse ‘heroku addons:docs heroku-postgresql:crane‘ to view documentation
To verify that our database(s) exist, we can now run the pg:info command:
$ heroku pg:info=== HEROKU_POSTGRESQL_BLACKPlan: DevStatus: availableConnections: 1PG Version: 9.1.5Created: 2012-08-29 17:46 UTCData Size: 6.0 MBTables: 1Rows: 0/10000 (In compliance)Fork/Follow: Unavailable
=== HEROKU_POSTGRESQL_IVORYPlan: CraneStatus: preparingData Size: -1 BTables: -1PG Version: ?Fork/Follow: Temporarily UnavailableCreated: 2012-08-29 17:46 UTCMaintenance: not required
As you can see from the output above, our Heroku application now has two databases defined, a free (dev) database,and a paid (crane) database.
Note: There are some differences between the shared database we created, and the paid database:
• The shared database is created instantly (there is no delay), while the paid database is provisioned on the fly,and takes a minute or so to become available.
• The shared database has some restrictions on it (such as the amount of data you can store), while the paiddatabase has no restrictions (other than the amount of RAM and CPU you’re paying for).
7.3 Connecting to Your Database
Now that you’ve got a database or two provisioned to your application, let’s connect to it:
$ heroku pg:psql HEROKU_POSTGRESQL_BLACKpsql (9.1.5)SSL connection (cipher: DHE-RSA-AES256-SHA, bits: 256)
22 Chapter 7. PostgreSQL Patterns
The Heroku Hacker’s Guide, Release 1.0
Type "help" for help.
dbname=>
Using the psql (http://www.postgresql.org/docs/9.1/static/app-psql.html) tool, we can connect directly to our new Post-greSQL database to ensure it’s working.
Note: This command will only work if you have the psql command available locally on your machine. For mostusers, this means installing the official postgresql-client software.
You’ll also notice that if you look at your environment variables via heroku config, you’ll see your new databasecredentials added:
$ heroku config=== myproject Config Vars...HEROKU_POSTGRESQL_BLACK_URL: postgres://user:password@host:port/dbHEROKU_POSTGRESQL_IVORY_URL: postgres://user:password@host:port/db
This means you can easily instruct your application to read these environment variables and automatically connect toyour new database.
Another thing you can do to simplify your application code if you’re using a single database is set yourDATABASE_URL environment variable on Heroku. This way, you can tell your application to always connect tothe database at DATABASE_URL, instead of hardcoding in the specified database you’d like to connect to each time(HEROKU_POSTGRESQL_BLACK_URL, for instance).
To do this, you can simply run:
$ heroku pg:promote HEROKU_POSTGRESQL_BLACKPromoting HEROKU_POSTGRESQL_BLACK to DATABASE_URL... done
Now if you look at your environment variables, you’ll notice that both DATABASE_URL andHEROKU_POSTGRESQL_BLACK_URL are identical.
7.4 Destroying a Database
If you’d like to remove a database that you’ve already provisioned, you can do so via the addons:remove command:
$ heroku addons:remove HEROKU_POSTGRESQL_BLACK! WARNING: Potentially Destructive Action! This command will affect the app: myproject! To proceed, type "myproject" or re-run this command with --confirm myproject
> myproject----> Removing HEROKU_POSTGRESQL_BLACK from myproject... done, v30 (free)
Heroku bills for database usage by the second, so as soon as your database has been removed, you’ll stop beingcharged.
7.5 Creating Read Slaves
Creating read slaves (http://en.wikipedia.org/wiki/Master/slave_(technology)) is a popular way to help scale read re-quests across a cluster of database servers. Luckily, Heroku makes this process extremely simple with their follow
7.4. Destroying a Database 23
The Heroku Hacker’s Guide, Release 1.0
(https://devcenter.heroku.com/articles/heroku-postgresql#follow-beta) feature.
Let’s assume your application currently has a single database, HEROKU_POSTGRESQL_GREEN. In order to create anew read slave database, you can use the --follow option when creating your new slave database:
$ heroku addons:add heroku-postgresql:ronin --follow HEROKU_POSTGRESQL_GREENAdding heroku-postgresql:ronin to myproject... done, v7 ($200/mo)Attached as HEROKU_POSTGRESQL_AMBERFollower will become available for read-only queries when up-to-dateUse ‘heroku pg:wait‘ to track statusUse ‘heroku addons:docs heroku-postgresql:ronin‘ to view documentation
The newly created database, HEROKU_POSTGRESQL_AMBER, will now automatically stay up-to-date with its masterdatabase, HEROKU_POSTGRESQL_GREEN.
Using Heroku’s follow feature, you can create as many read slaves as you like.
Note: While Heroku ensures your new slave database will follow its master, there will be some replication delay.This means that newly written data to the master database may take several seconds to reach your slave database(s).
7.6 Creating a Duplicate Database
In many situations, the ability to create a duplicate database can be extremely useful:
• You want to run tests against your production data, but you don’t want to expose your production database toyour testing code.
• You want to test database migrations before applying them in production.
• You want a copy of a production database just in case something bad happens.
Duplicating a database using Heroku is really simple thanks to Heroku’s fork(https://devcenter.heroku.com/articles/heroku-postgresql#fork_beta) feature. To duplicate (fork) a copy of yourproduction database, HEROKU_POSTGRESQL_GREEN, you can create a new database using the --fork option:
$ heroku addons:add heroku-postgresql:ronin --fork HEROKU_POSTGRESQL_GREENAdding heroku-postgresql:ronin to myproject... done, v7 ($200/mo)Attached as HEROKU_POSTGRESQL_AMBERDatabase will become available after it completes forkingUse ‘heroku pg:wait‘ to track statusUse ‘heroku addons:docs heroku-postgresql:ronin‘ to view documentation
Once the new database is up and running, you’ll be able to use it like any other master database–you can perform writequeries, give it a read slave, whatever you want.
Note: Forked databases do NOT stay up-to-date with the database they were forked from.
7.7 Promoting a Slave Database to a Master Database
Let’s say you’re in a situation where you need to make one of your read slaves writable (as a master). Heroku makes thisprocess extremely simple using their unfollow (https://devcenter.heroku.com/articles/heroku-postgresql#unfollow)command.
24 Chapter 7. PostgreSQL Patterns
The Heroku Hacker’s Guide, Release 1.0
Let’s assume you’ve got a read slave named HEROKU_POSTGRESQL_AMBER. To make it writable, all we do is runthe pg:unfollow command, like so:
$ heroku pg:unfollow HEROKU_POSTGRESQL_AMBER! HEROKU_POSTGRESQL_AMBER will become writable and no longer! follow HEROKU_POSTGRESQL_BRONZE. This cannot be undone.
! WARNING: Potentially Destructive Action! This command will affect the app: myproject! To proceed, type "myproject" or re-run this command with --confirm myproject
> myprojectUnfollowing... done
7.8 View Slow Queries
A big part of writing good code is knowing when you do things wrong. Slow database queries, in particular, areprobably the greatest cause of poor site performance.
Luckily for us, Heroku’s logging system allows you to easily view a stream of slow query logs directly from yourconsole. Any query that takes longer than 50ms to execute will be dumped into the log output.
To view the streaming logs, you can simply run: heroku logs --tail --ps postgres. If you’d like to justview the most recent logs, you can run the same command without the optional --tail argument: heroku logs--ps postgres.
For more advanced database troubleshooting, see Monitoring with New Relic.
7.9 Backing Up Your Database
Backing up your database is incredibly important, but not always easy to do. To combat the complexity of backing upyour database, Heroku created their extremely useful pgbackups addon (https://addons.heroku.com/pgbackups).
At the time of writing, all of the pgbackups addon plans are completely free! This means you can store backups ofyour database for no cost at all.
To get started, we’ll use the largest available backup plan: auto-month. This plan will:
• Automatically backup your DATABASE_URL (default database) every night.
• Retain 7 daily backups.
• Retain 5 weekly backups.
• Retain 10 manual backups.
To get it going, install the addon:
$ heroku addons:add pgbackups:auto-monthAdding pgbackups:auto-month to myproject... done, v14 (free)You can now use "pgbackups" to backup your databases or import an external backup.
Once you’ve got the addon installed, Heroku will start automatically backing up your primary database each day.
To view a list of your available backups, you can run the pgbackups command. Since we just installed theauto-month backup plan, however, we have no existing backups:
7.8. View Slow Queries 25
The Heroku Hacker’s Guide, Release 1.0
$ heroku pgbackups! No backups. Capture one with ‘heroku pgbackups:capture‘.
Let’s fix that right now by forcing a manual backup:
$ heroku pgbackups:capture
HEROKU_POSTGRESQL_GREEN (DATABASE_URL) ----backup---> b001
Capturing... doneStoring... done
Now, if we run the pgbackups command again, we should see:
$ heroku pgbackupsID | Backup Time | Size | Database-----+---------------------+------------+------------------------b001 | 2012/08/29 23:09.50 | 918.0bytes | HEROKU_POSTGRESQL_GREEN
As time progresses, and we gradually get more backups, they’ll show up in the pgbackups listing.
Note: In the backup example above, we backed up our default database (DATABASE_URL). If you’d like tobackup another database, you can do so by specifying its name, for example: heroku pgbackups:captureHEROKU_POSTGRESQL_BLACK.
As a quick note: all manually captured backups display with a b prefix in the heroku pgbackups listing, whileall automatically captured backups display with an a prefix. This allows you to easily distinguish between manual andautomatic backups when viewing your backup listing.
7.10 Downloading Your Backups
So you’ve got a few database backups, and you’d like to download them–no problem! Assuming you’ve got thefollowing backup available:
$ heroku pgbackupsID | Backup Time | Size | Database-----+---------------------+------------+------------------------b001 | 2012/08/29 23:09.50 | 918.0bytes | HEROKU_POSTGRESQL_GREEN
You can create a publicly available download URL from Amazon S3 (http://aws.amazon.com/s3/) (which is good for10 minutes) by running:
$ heroku pgbackups:url b001"https://s3.amazonaws.com/hkpgbackups/[email protected]/b001.dump?AWSAccessKeyId=blah&Expires=blah&Signature=blah"
Which you can then download directly to your computer using wget, curl, or any other standard tool. These backupsare standard PostgreSQL database dumps, which means you can restore them to any PostgreSQL database–regardlessof whether or not its hosted on Heroku.
Note: Using these database dumps is a great way to import your production data into local databases for testingpurposes.
26 Chapter 7. PostgreSQL Patterns
The Heroku Hacker’s Guide, Release 1.0
7.11 Restoring From a Backup
While backing up your database is always a good idea, your backups will do you no good unless you know how torestore from the backup when things go wrong.
Assuming you have the following backup available:
$ heroku pgbackupsID | Backup Time | Size | Database-----+---------------------+------------+------------------------b001 | 2012/08/29 23:09.50 | 918.0bytes | HEROKU_POSTGRESQL_GREEN
You can restore your backup (b001 in this case) to any of your available databases using the pgbackups:restorecommand:
$ heroku pgbackups:restore HEROKU_POSTGRESQL_GREEN b001
HEROKU_POSTGRESQL_GREEN (DATABASE_URL) <---restore--- b001HEROKU_POSTGRESQL_GREEN2012/09/20 23:09.50918.0bytes
! WARNING: Potentially Destructive Action! This command will affect the app: myproject! To proceed, type "myproject" or re-run this command with --confirm myproject
> myproject
Retrieving... doneRestoring... done
Note: The restoration process make take a few minutes if your backups are large.
7.12 Final Thoughts
No matter how big (or small) your application needs, Heroku’s PostgreSQL service is a great choice.
If you end up using Heroku PostgreSQL, be sure to familiarize yourself with the information above (particularly howto setup automatic backups) so you’ll always feel comfortable knowing your data is available at any time.
Having backups ready and available is crucial, as even a small coding issue can wipe out lots of production data inseconds.
7.11. Restoring From a Backup 27
The Heroku Hacker’s Guide, Release 1.0
28 Chapter 7. PostgreSQL Patterns
CHAPTER
EIGHT
CACHING WITH MEMCACHED
Caching is a popular topic, as caching is quite possibly the single best way you can speed up modern web applications.
This chapter is short, but discusses caching and briefly explores the available caching addons on Heroku.
8.1 Why Cache?
A user makes a web request. The web request hits Heroku’s load balancer, which then hits one of your web dynos.Your web server responds, runs the appropriate functions, and eventually hit your database server.
After getting a result back from the database server and doing things with the result, your web server replies to theuser’s request, which is then routed back to the user to display.
For most web applications, talking to the database is the single slowest part of processing user requests (in Monitoringwith New Relic I’ll show you how to profile this).
While databases can be very fast, complex database queries can be slow and use a lot of resources on your databaseserver. This in turn leads to less database throughput, slower responses, and unhappy users.
Luckily, most database queries never have to happen.
If you were to analyze your application code–how many of your database queries are simply grabbing static or almoststatic data? Is it possible to store these results elsewhere and avoid hitting the database?
This is what caching is all about.
8.2 Using Memcached
Memcached (http://memcached.org/) is an extremely popular, stable, and high performance in-memory caching sys-tem. It allows you to store and retrieve key-value information at extremely high rates, without ever hitting the disk.
For most web applications, using memcached is without a doubt the best way to increase application performance, asyou can simply store results of your time consuming database queries in memcached, and retrieve them quickly so youcan avoid hitting your database any more than necessary.
To make things even easier–almost all web frameworks have some sort of built in memcached support, which makesintegrating caching logic into your application a simple process.
29
The Heroku Hacker’s Guide, Release 1.0
8.3 Memcached on Heroku
Using memcached is very easy on Heroku, as there are multiple memcached addons available which allow you toinstantly add, remove, and upgrade memcached servers in a single command.
At the time of writing, there are three available memcached addons:
• Memcache (https://addons.heroku.com/memcache) - The oldest (and most expensive).
• Memcachier (https://addons.heroku.com/memcachier) - A newer but much less expensive option.
• IronCache (https://addons.heroku.com/iron_cache) - A new elastic memcached service currently in beta.
All of which work amazingly well, and are easy to work with.
Regardless of which addon you choose to host memcached for you, you should without question be using memcachedto speed up your applications and better handle user traffic. As a general rule: the more stuff you cache, the better.
30 Chapter 8. Caching with Memcached
CHAPTER
NINE
SCHEDULING TASKS WITH CRON
Most web applications require periodic scripts to be ran in the background. Heroku supports cron(http://en.wikipedia.org/wiki/Cron) like tasks via their (free) Heroku Scheduler (https://addons.heroku.com/scheduler)addon.
9.1 The Scheduler
Getting started with the Heroku’s scheduler is a simple process.
First, install the scheduler addon:
$ cd ~/myproject$ heroku addons:add scheduler:standard
Next, run the heroku addons:open scheduler command which will open your web browser and take youto the scheduler dashboard, where you can add new jobs, specify how often you’d like them to run, and even forcemanual runs yourself. At the time of writing, the scheduler dashboard looks like this:
Assuming you have a program named generate_reports.py in your project’s root directory, you could simplycreate a new job and enter generate_reports.py as the action.
31
The Heroku Hacker’s Guide, Release 1.0
9.2 Debugging
To make sure your task works as expected, you should try running it manually first through Heroku.
If your task, generate_reports.py is supposed to run once an hour, you could test this manually by running:
$ heroku run generate_reports.py
As that is precisely what the scheduler addon will do for you at the desired time intervals.
Some things to check if you’re having problems are:
• Is your script executable? (chmod +x)
• Can you run your script using heroku run?
• Is your Heroku application configured properly with any required configuration settings?
9.3 Cost
While the Heroku scheduler addon is free, running tasks are billed by the second. What this means is that if you’vedefined a task to run once an hour, and this task takes 1 minute to run each time–then you’d be paying for approximately24 dyno minutes per day to run your task.
This is something to consider if your cron tasks take a considerable amount of time to run.
32 Chapter 9. Scheduling Tasks with Cron
CHAPTER
TEN
LOGGING
Knowing how and when to read your logs is a good way to find problems and avoid trouble. Heroku’s logs commandallows you to easily view your application logs and get direct (real time) access to your aggregated log data.
10.1 Log Types
Heroku has three primay types of logs:
• Application logs which display the standard output and error streams from your application.
• System logs which display all messages related to the Heroku infrastructure–router logs, error pages, dynorestarts, etc.
• API logs, which contain metadata about your Heroku application: code deploys, dyno scaling, etc.
10.2 Viewing Logs
The simplest way to view your logs is by viewing a total list of all the recent log messags for your application. Youcan do this by running:
$ heroku logs2012-08-30T06:49:09+00:00 heroku[web.1]: State changed from starting to up2012-08-30T06:49:10+00:00 app[web.1]: 2012-08-30 06:49:10 [2] [INFO] Handling signal: term... lots of output...
All logs have the same components:
• A timestamp in UTC.
• The type of log (usually something like app[web.1], heroku[router], etc.).
• The message.
To view all your logs in real time, you can do:
$ heroku logs --tail...
The --tail option tells Heroku to continuously stream the latest logs as they come in, outputting them directly toyour terminal. While this is a great way to view logs–outputting all of your logs at once can be overwhelming, sinceyou’ll be seeing so much data scroll by. It is therefore a lot more effective to restrict your logs to specific messages.
33
The Heroku Hacker’s Guide, Release 1.0
10.3 Viewing Select Logs
To view only select logs, you can specify the optional --source option. For instance, if you want to only view yourheroku log messages (router requests, dyno restarts, etc.), you could run:
$ heroku logs --tail --source heroku2012-08-30T06:49:05+00:00 heroku[api]: Add scheduler:standard add-on by [email protected]:49:05+00:00 heroku[api]: Release v54 created by [email protected]...
You can also restrict your output to any of the following three sources (mentioned in the previous section): app,heroku, and api.
To narrow things down even more, you can specify a select process to view. This way you can run log queries like:
$ heroku logs --ps web.1 # only view logs from your first web dyno$ heroku logs --ps scheduler.1 # only view logs from your first scheduler dyno$ heroku logs --ps postgres # only view your postgresql logs
This is an excellent way to view critical log information without being blasted by thousands of messages all at once.
10.4 When to Check Your Logs
If you’re having issues with your application, the first place to look will usually be your log output.
When I’m deploying new code to Heroku, I’ll often keep a streaming log open on my web dynos, just to make sureeverything goes smoothly:
$ heroku logs --tail --ps web
As a general rule, you should always check your logs before and after deploying code. This will help you quicklyfind problems (did my recent code changes cause my application to crash?) and give you the information necessary tosolve them quickly (or allow you to rollback to a previous working release).
10.5 Log Storage Options
For some users, viewing logs via Heroku’s command line tool is simply not suitable. Currently, there are two availableoptions for users who’d like to store their logs for long periods of time, search them, etc.
The first (and simplest) option is to use one of the excellent logging addons. At the time of writing, there are threecompeting logging addons on Heroku, all of which are excellent:
• Papertrail (https://addons.heroku.com/papertrail)
• Logentries (https://addons.heroku.com/logentries)
• Loggly (https://addons.heroku.com/loggly)
Each of these addons will store your logs and let you search them (in various ways).
The second option is to manually drain your logs to a remote server of your choosing using Heroku’s syslogdrain (https://devcenter.heroku.com/articles/logging#creating-your-own-syslog-drain) support. This option is likelycheaper, but requires you to setup, manage, and maintain your own logging server. If you’d like to go this route, readthrough the official configuration instructions here: https://devcenter.heroku.com/articles/logging#creating-your-own-syslog-drain.
34 Chapter 10. Logging
CHAPTER
ELEVEN
MONITORING WITH NEW RELIC
Monitoring is one of the most critical tools available to every developer running production applications. Withoutgood monitoring software, knowing when problems arise, what problems you have, and how to fix your problems isnearly impossible.
This chapter is dedicated to explaining what I consider to be one of the most important aspects of successful develop-ment on Heroku: using New Relic (https://addons.heroku.com/newrelic) to monitor your application.
11.1 Why New Relic?
There are currently lots of application monitoring services available–but New Relic is the absolute best.
Among other things, New Relic allows you to:
• Setup full stack monitoring without modifying your application code at all.
• Surf through your data, metrics, and reporting dashboards using a beautiful web interface.
• View an abundance of real time data, including:
– Application response times.
– Errors (404s, 500s, etc.).
– Downtime alerts.
– Slow web and database transactions.
– Average throughput.
– Average web user response time.
– Location of your users.
– Dyno memory usage.
– Dyno CPU usage.
– Dyno backlog.
– And lots more.
I guarantee that if you give New Relic a try, you’ll be extremely happy with your decision.
35
The Heroku Hacker’s Guide, Release 1.0
11.2 Installing New Relic
To get started with New Relic, you’ll need to install the necessary libraries and configure your project as required.Since this process is different depending on which programming language and web framework you’re using, I’ll leavethis as a reader exercise.
To continue, you’ll need to follow through the New Relic getting started guide, which can be found here:https://devcenter.heroku.com/articles/newrelic.
I’ll also assume for the rest of this chapter that you’re using New Relic’s professional plan, as it contains the moreadvanced reporting features I’ll be talking about below. If you’re planning on running New Relic’s standard (free)plan, you can see which features are available by checking out the official New Relic feature and pricing page(http://newrelic.com/pricing/details) on their website.
Regardless of what New Relic plan you’re using, you’ll be gaining an invaluable amount of insight into your applica-tion. At the very least, you should be running the New Relic free plan so you can see some basic application metricsand make informed decisions about your application when you need to.
Note: New Relic makes frequent updates to their platform and features, and as such, is continuously evolving.Depending on which version of the New Relic agent you’re running, you’ll likely have different features available.
11.3 The Overview
In my opinion, New Relic’s most useful (and simple) reporting page is their overview dashboard page. To view thispage in your browser, you can run the following command:
$ cd ~/myproject$ heroku addons:open newrelic
Once you’re at the overview page, you should see something like the following:
The overview page shows you:
• Recent slow transactions (you can click them to see more detail).
36 Chapter 11. Monitoring with New Relic
The Heroku Hacker’s Guide, Release 1.0
• Recent application errors and tracebacks.
• Application response time (in ms).
• End user response time (in ms).
• Recent events (code deploys, downtime, etc.).
• Throughput (requests per minute).
• Apdex score (how satisfied are your users?) The apdex score is calculated based on a number of factors fromwithin New Relic.
• What parts of your application are taking up how much time per request. EG: Your database takes 50ms of timeper request, your caching takes 25ms of time per request, and your python application takes 400ms of time perrequest.
If you take nothing else away from this section, take this: watch this page!
By looking at just this page, you’ll be able to:
• Immediately notice if a new code deploy negatively (or positively) affects your application response time.
• Notice downtime as soon as it happens (is Heroku experiencing an outage?).
• Figure out if your infrastructure components are suffering from overusage (is your database absorbing a lot ofyour request time?).
• See how many users are hitting your application right now.
11.4 A Visual Map of Your Application
Another very useful page in the New Relic dashboard is the ‘Map’ page. To access it, click the Map menu item at thetop of the page:
This page gives you a quick visual mapping of your application. It shows you:
• The average response time for each of your infrastructure components (your web service, your database, cachingserver, external web services, etc.).
11.4. A Visual Map of Your Application 37
The Heroku Hacker’s Guide, Release 1.0
• Graphs which show how many calls per minute each infrastructure component is handling, and at which speeds.
• Colors (green for OK, yellow for DECENT, and red for BAD).
This page allows you to quickly eyeball your entire infrastructure, and immediately tell if any infrastructure compo-nents are failing, responding slowly, or handling an unexpected amount of traffic.
When collaborating with other developers, showing them this page can help them immediately understand your appli-cation’s architecture and configuration from a high level perspective.
11.5 Web Transactions
The New Relic Web Transactions page shows you a great deal of information about each of your web views:
It shows you each of your web transactions, how much time they take to run, and what each transaction is actuallydoing (hitting the database, cache, etc.).
If you’re looking for bottlenecks and ways to optimize your code, look no further. By clicking around this page, you’llfind out everything you ever wanted to know (and more!) about your application’s performance, and you’ll have clearindicators of how things can be improved.
By looking at this page before and after you deploy code, you can see exactly how your code changes affect yourapplication performance. This is a very useful trick for quick optimization, and allows you to immediately see howyour code changes are affecting your users and overall application performance.
11.6 Database Monitoring
The New Relic Database page is a great tool for hunting down performance issues in the database:
38 Chapter 11. Monitoring with New Relic
The Heroku Hacker’s Guide, Release 1.0
Assuming you’re using PostgreSQL, you can see all sorts of stats:
• Which queries are ran most frequently.
• How long each query takes to run.
• What the throughput is for each query.
• Which code calls the queries.
• And more.
Using this page, you can easily narrow down and eliminate database issues by drilling your search down on a queryby query basis, finding problematic code, and optimizing it.
11.7 External Services
New Relic can even show you stats about external services your application uses. The External Services tablets you easily track down problems with third party API and providers, and see exactly how performant they are:
11.7. External Services 39
The Heroku Hacker’s Guide, Release 1.0
For applications that rely on lots of third party APIs and mashups, this is a great way to debug issues, find bottlenecks,and optimize requests.
11.8 Dyno Monitoring
In terms of planning your Heroku capacity, nothing beats New Relic’s Dynos page:
It tells you:
• How much memory your application is using.
• How many dynos you have running.
• How many dyno restarts you’ve had (high numbers could indicate application crashes).
40 Chapter 11. Monitoring with New Relic
The Heroku Hacker’s Guide, Release 1.0
• How many HTTP requests are currently backlogged (if you have too many incoming HTTP requests and yourdynos can’t serve reponses fast enough, you will begin to build a backlog). If this number is above 0, you shouldprobably run more dynos.
Ensuring you always have adequate dynos available is critical to maintaining a high level of application availabilityand throughput.
11.9 Background Tasks
New Relic even lets you monitor your background tasks via their Background Tasks dashboard page:
On this page, you can:
• View all your background tasks.
• See how long they take to run.
• View your CPU usage.
• View your memory usage.
• View slow tasks.
• View a performance breakdown for each task (how much time was spent in the database, cache, application,etc.).
Larger applications that rely heavily on background tasks can be easily debugged through this page. Since backgroundtasks are typically asynchronous in nature, and don’t raise user facing exceptions, this page gives you an incrediblyamount of insight into how well your tasks work, how they can be optimized, and how they can be fixed (when broken).
11.10 Final Thoughts
In addition to all the New Relic dashboard features I mentioned above–there are tons more. New Relic is constantlyadding new dashboards, metrics, and reporting tools to give you complete insight into the various components of yourapplications.
11.9. Background Tasks 41
The Heroku Hacker’s Guide, Release 1.0
If you spend a bit of time poking around menus, you’ll be able to find almost any metric you can think of.
Out of all the things you can do to proactively make your application better, monitoring your applications with NewRelic and spending time analyzing your metrics is quite possibly the most effective way to write faster, more error freecode and ensure your users have a good experience on your site.
42 Chapter 11. Monitoring with New Relic
CHAPTER
TWELVE
TALKING TO THE WORLD
At some point, you’ll need to make your new Heroku application publicly accessible. This usually requires setting upyour DNS and purchasing SSL certificates.
In this chapter, I’m going to walk you through setting up your DNS, and using SSL for your application.
12.1 Using Custom Domains
When you first created your Heroku application, you were assigned a unique URL for your application so that youcould test it directly.
Each Heroku app is assigned a unique URL of the form http://myproject.herokuapp.com, wheremyproject is your Heroku application’s name.
In order to make your Heroku application accept requests from your custom domain name(s), you’ll need to use theHeroku domains command:
$ cd ~/myproject$ heroku domains:add www.myproject.comAdding www.myproject.com to myproject... done
$ heroku domains=== myproject Domain Nameswww.myproject.com
In the example above, we assigned a custom domain name (www.myproject.com) to our Heroku application. Ofcourse, you can assign as many custom domains names as you’d like to your application in this way.
Note: You should always use a CNAME when defining Heroku domains (www, api, blog, etc.). To find out why, readthe Naked Domains (https://devcenter.heroku.com/articles/custom-domains#naked-domains-mydomaincom) Herokuarticle.
If you’d like to remove domain names, you can also use the domains:remove command, or the domains:clearcommand (which will remove all domains).
12.2 Updating Your DNS
Now that you’ve added your domains to your Heroku application, you need to configure your DNS with your DNSprovider. To do this, log into your DNS provider’s control panel and create a new CNAME record for your domain.
43
The Heroku Hacker’s Guide, Release 1.0
In my case, I’ll add a new CNAME for www.myproject.com (the CNAME here is www), and I’ll point it to myHeroku application: myproject.herokuapp.com:
CNAME www -> myproject.herokuapp.com
After a few hours (DNS can be slow to update), you should now be able to visit www.myproject.com and see yourHeroku application!
12.3 Encrypt All the Things! (with SSL)
To handle SSL for your application, Heroku has an excellent SSL Endpoint (https://addons.heroku.com/ssl) addonavailable.
To get started using SSL, we’ll first install the SSL Endpoint addon:
$ heroku addons:add ssl:endpointAdding ssl:endpoint on myproject... done, v52 ($20/mo)Next add your certificate with ‘heroku certs:add PEM KEY‘.Use ‘heroku addons:docs ssl:endpoint‘ to view documentation.
Next, you need to actually generate and download your SSL certificate from your SSL provider. For most people, thismeans going to your domain registrar (DNSimple (https://dnsimple.com/) is a good choice), and purchasing an SSLcertificate.
Once you’ve got your SSL certificate files, download them to your local machine. You should have two files available:a PEM file (server.crt), and a KEY file (server.key).
Now all you need to do is upload these files to Heroku using the certs:add command:
$ heroku certs:add server.crt server.keyAdding certificate to myproject... done.myapp now served by blah-blah.herokussl.com.Certificate details:
subject: C=US; ST=CA; L=SF; O=Heroku; CN=www.myproject.comstart date: 2011-11-01 21:53:18 GMTexpire date: 2012-10-31 21:53:18 GMTcommon name(s): www.myproject.comissuer: C=US; ST=CA; L=SF; O=Heroku; CN=www.myproject.comSSL certificate is self signed.
At any time, you can display a listing of your active SSL certificates using the certs command:
$ heroku certsEndpoint Common Name Expires Trusted----------------------- ----------------- ----------------------- -------blah-blah.herokussl.com www.myproject.com 2013-08-30 21:20:18 GMT True
Notice that when you added your SSL cert to Heroku, you were assigned a unique endpoint name,blah-blah.herokussl.com in this case. To test your application over SSL, you can now try to hithttps://blah-blah.herokussl.com in your browser.
Assuming your SSL endpoint test went well, all you need to do now is update your DNS one last time. To do this, loginto your DNS provider’s site, and update your CNAME record to point to your new SSL endpoint server:
CNAME www -> blah-blah.herokussl.com
After your DNS has finished updating, your site should be serving up encrypted traffic!
44 Chapter 12. Talking to the World
CHAPTER
THIRTEEN
MANAGING RELEASES
Being able to quickly rollback to previous application versions is critical. Accidents can and will happen, and noamount of unit testing, integration testing, and even manual testing will be 100% successful at preventing bad softwarefrom getting into the wild.
This chapter discusses an important part of the deployment lifecycle: managing application releases.
13.1 Heroku’s Model
Heroku takes application releases very seriously. Each time you push code, modify an addon, or even set an environ-ment variable–Heroku generates a new internal ‘release’ for your application.
Each release contains important metadata about your application, allowing you to rollback to previous points in timeinstantly, with complete confidence.
13.2 Viewing Your Releases
To view a list of your application releases, use the releases command:
$ cd ~/myproject$ heroku releases=== myproject Releasesv52 Add scheduler:standard add-on [email protected] 13h agov51 Detach HEROKU_POSTGRESQL_CRIMSON resou.. heroku-postgresql 13h agov50 Deploy c5817c2 [email protected] 2012/08/28 10:12:20...
Each release has a version number (v52, v51, etc.), a description of what happened, and a time.
To view details about a specific release, you can run the releases:info command:
$ heroku releases:info v52=== Release v52Addons: scheduler:standardBy: [email protected]: Add scheduler:standard add-onWhen: 13h ago
=== v52 Config VarsDATABASE_URL: postgres://user:password@host:port/dbDEBUG: True...
45
The Heroku Hacker’s Guide, Release 1.0
Using the releases:info command, you can see that Heroku stores not only the change that happened–but alsothe environment variables set at the time. This way, in the event that you need to rollback to a previous release, youcan easy inspect your previous release to make sure it has the right environment variables, etc.
13.3 Rolling Back
Once you’ve determined that you need to rollback to a previous release, doing so is simple:
$ heroku releases:rollback v52Rolling back myproject... done, v52
Once you’ve rolled back to a previous release, your dynos will instantly be running your old release, and things shouldbe working as normal again.
Note: You’ll notice that after performing a rollback have a new release number that shows your rollback! This meansif you make a mistake, you can easily swap back and fourth between versions.
13.4 Final Thoughts
Knowing how and when to rollback your code is critical to maintaining highly available applications. By understandinghow Heroku’s release system works, you can quickly recover from errors without significantly affecting your users.
One thing to keep in mind, however, is that while Heroku’s rollback allows you to easily revert your application versioninstantly, it will not bring back addons which have been removed or modified.
For instance, if you remove your Heroku PostgreSQL addon, then try to rollback, you will notice that the rollbackwill fail. This is to be expected, as Heroku cannot safely rollback to your previous release as your database no longerexists.
Before performing potentially destructive actions like removing addons, you should carefully think about whether ornot losing the ability to rollback to any point in time before the action is acceptable or not.
46 Chapter 13. Managing Releases
CHAPTER
FOURTEEN
WORKING WITH OTHERS
Knowing how to best work with other developers on your Heroku project is a good thing to keep in mind. At the timeof writing, Heroku has a very simple collaboration model that makes it easy to share your Heroku project with otherdevelopers, and takes only a moment to understand.
14.1 Who is the Application Owner?
The way Heroku currently works (I’ve heard this is in the process of changing) is that there are no ‘organizationaccounts’. What this means is that if you’d like to host a new application for your new company, “Hacker Holdings,LLC”, you need to create a new Heroku account for your company ([email protected] ([email protected])), andcreate the application there.
One you’ve created your new application under your new Heroku account ([email protected] ([email protected])),you can then ‘share’ access to this application with your personal account, allowing yourself access to the project.
14.2 Sharing Access
To share access to your Heroku project with another developer, you can use the sharing:add command:
$ cd ~/myproject$ heroku sharing:add [email protected] [email protected] to myproject collaborators... done$ heroku sharing:add [email protected] [email protected] to myproject collaborators... done
The above example invites two developers to collaborate on myproject. All you do is add the person via thesharing:add command, and that person will then receive an email invitation to collaborate on the project.
14.3 Sharing Permissions
Once a project has been shared with another person, that person will have the ability to:
• Push code.
• Resize existing addons.
• Rollback to previous releases.
• Make database backups, etc.
47
The Heroku Hacker’s Guide, Release 1.0
Collaborators will not have the ability to:
• Scale dynos up or down.
• Add or remove paid addons.
• Delete or rename the application.
• View invoices.
This means that if you need to add new paid addons to your application, scale your dynos, or rename your applicationyou’ll need to have the application owner do so on your behalf.
14.4 Managing Collaborators
Managing application collaborators is quite simple. To view a list of all the current collaborators on a given application,you can use the sharing command directly:
$ heroku sharing=== myproject [email protected]@[email protected]
To remove a collaborator from an application, use the sharing:remove command:
$ heroku sharing:remove [email protected] [email protected] from myproject collaborators... done
When working with large teams, this becomes essential as team members frequently leave / join new projects, andmaintaining an accurate permissions system becomes very important.
14.5 Transferring Ownership
Lastly, you may occasionally want to transfer ownership of an application from your account to someone else’s. Thisis possible by use of the sharing:transfer command:
$ heroku sharing:transfer [email protected] myproject to [email protected]... done
Note: Heroku won’t let you transfer ownership of an application to a user that isn’t already an application collaborator.This means you must add the new owner as an application collaborator before initiating the transfer.
I find that this is most useful when you accidentally create work applications under your personal accounts. Instead ofrecreating the application under your company account, you can simply transfer application ownership.
48 Chapter 14. Working with Others
CHAPTER
FIFTEEN
DO GREAT THINGS
Using Heroku is a great way to save yourself time, money, and headaches when developing web software.
To really take advantage of Heroku’s excellent platform, there are certain practices that you can and should use whiledeveloping on Heroku to not only make your life easier, but to help you write better, faster, and more scalable systems.
This chapter is dedicated to explaining some of these best practices in order to help you do great things and prosper.
15.1 Build Services, Not Apps
In any given web application, there are various moving parts. Typical web applications require:
• A frontend that renders HTML output.
• User registration and account management.
• Backend functionality that calls various web APIs, does processing, etc.
• Worker processes which handle one off tasks.
While it’s easy to think of a project as a single unit, in reality, projects are often better expressed when broken downinto sub-components (services).
Let’s say you’re building a database hosting service. It’s quite easy to think of this as a single application that:
• Allows users to sign up for accounts.
• Allows users to choose what sort of database they’d like.
• Takes this user input, and runs some code which configures a database, sets it up, and grants the user access.
• Has a background task which monitors these databases and ensures they don’t break.
And while you can most certainly build your database hosting service as a single application using your favorite webframework, you will run into problems:
• You will have a lot of project code.
• You will have a lot of complexity in your codebase.
• You will need to handle multiple layers of complex caching to avoid performance problems.
• You will have trouble scaling your application as some parts will need to be used much more than others, leavingyou with complex scaling formulas.
Instead of building an application, you should really be building multiple small services.
49
The Heroku Hacker’s Guide, Release 1.0
15.2 Building Service Oriented Applications
Service oriented applications are an ideal solution to the problem of complexity in software development.
Using the example above, we’d break our application into multiple purpose-driven services:
• A project-www service which renders the public facing website, logs users in and out of their account, andacts as a dumb client.
• A project-auth service which exposes an HTTP API that allows you to create, manage, and remove useraccounts.
• A project-monitor service which continuously monitors all your customer databases and ensures they areworking at any given time–and if not, handles failover.
• A project-api service which handles the initial provisioning of the customer databases, updates to thesedatabases, and other required bits of functionality.
By separating your application into these independent services, you can now:
• Individually scale the various parts of your project as needed.
• Keep each codebase small and maintainable.
• Avoid complexity.
• Reduce cost.
15.3 Heroku and Services
Heroku’s platform encourages you to build service oriented applications.
Since Heroku applications are easy to create, and each one comes with a single free dyno each month–scaling yourproject across multiple Heroku apps by breaking it up into services can save you money.
In our example above, if we were to build our database hosting service by creating four separate Heroku applicationswe’d have 4 free dynos total each month, a much simpler codebase, and the ability to scale our application exactly aswe need.
Furthermore, by exposing an HTTP API for each of our internal services, we’d be able to easily modularize our project,making it easier to maintain over time, grow, and upgrade.
Instead of having to sift through a single large codebase updating the name of our internal functions, we’d be able tochange our function in a single place, so long as it still exports the HTTP API our other services expect.
For further information on building service oriented applications, check out the Wikipedia page on Service OrientedArchitecture (http://en.wikipedia.org/wiki/Service-oriented_architecture).
15.4 Be Dynamic
One of the great benefits of using a cloud provider like Heroku in the first place is that you pay only for what you use.This is in stark contrast to purchasing your own equipment, running things yourself, etc.
Unfortunately, most people don’t take advantage of the truly dynamic nature of Heroku when building their applica-tions!
Heroku’s built in logging service as well as instant scaling power allows you to save money by scaling up to supportdemand from your users, and scaling down to save money when the burst traffic has stopped.
50 Chapter 15. Do Great Things
The Heroku Hacker’s Guide, Release 1.0
Luckily, there are several services available which use Heroku’s API to monitor and scale your applications for you–myfavorite of which is HireFireApp (http://hirefireapp.com/).
HireFireApp will analyze your Heroku applications, automatically scaling your web and worker dynos as needed,saving you money and ensuring your application is always as fast as possible for your users.
Don’t blindly estimate your capacity planning–that’s the old way of doing things; instead, be dynamic!
15.5 Final Thoughts
I really hope you’ve enjoyed reading this book–I had a lot of fun writing it. Heroku is an excellent platform, and it hasbeen my pleasure to both use and write about it over the past couple years.
I’ve had excellent experiences building, maintaining, and growing products on Heroku, and want to make sure you doas well.
If you have any questions about Heroku, this book, or anything else–feel free to send me an email: [email protected]([email protected]). I’d be happy to talk with you, give you some guidance, or even just say “hi”.
Now take what you’ve learned here and go build amazing things! Happy hacking!
Randall Degges (http://rdegges.com/)
15.5. Final Thoughts 51
The Heroku Hacker’s Guide, Release 1.0
52 Chapter 15. Do Great Things
CHAPTER
SIXTEEN
REFERENCES AND FURTHERREADING
Although this book contains a majority of the things you need to know to both understand Heroku and use it success-fully in the wild, there are tons of other resources out there which you may find useful.
Below is a listing (in no particular order) of places you may want to check out if you’re interested in furthering yourknowledge of Heroku and best practices.
• Heroku’s Dev Center (https://devcenter.heroku.com/) is Heroku’s public documentation store that holds all oftheir topical guides and short articles. This is the first resource you should turn to if you know what you’relooking for, and need quick reference information.
• Heroku’s Stack Overflow (http://stackoverflow.com/questions/tagged/heroku) page is an excellent place to getyour questions answered, and even answer questions from other Herokai.
• My Personal Blog (http://rdegges.com/) is a good reference to anyone interested in Heroku, web development,and best practices. I frequently publish articles about a variety of topics, many of which are deployment related.
• Heroku’s Addon Provider Site (https://addons.heroku.com/provider) is the only place to go if you’d like to learnabout building your own Heroku addons. If you’re at all interested in becoming a Heroku addon provider, thispage will tell you everything you need to know about the process, including technical and business requirements.
• Heroku’s GitHub Account (https://github.com/heroku) is a great place to go if you’re looking for code samples,examples, relevant libraries, or anything in between. Heroku has tons of engineers on staff, and publishes lotsof high quality open source projects to their company GitHub account.
53
The Heroku Hacker’s Guide, Release 1.0
54 Chapter 16. References and Further Reading
CHAPTER
SEVENTEEN
SPECIAL THANKS
I’d like to personally thank the technical editors who helped review this book. Without their help, you would now bereading a much lesser book.
• Kenneth Reitz (http://kennethreitz.com/) - Python Overlord @ Heroku
• Graham Dumpleton (http://blog.dscpl.com.au/) - Python Hacker @ New Relic
• Craig Kerstiens (http://www.craigkerstiens.com/) - Product Guy @ Heroku
• Andrew Austin (http://andrew-austin.com/) - Django Developer
I’d also like to thank my wife, Samantha (http://hardlyfunny.com/), who provided lots of encouragement along theway and helped with photoshop. Thanks, babe.
Last (but not least) I’d like to personally thank everyone at Heroku. You guys (and gals) have built an amazing product,and you should be proud. Your work is tremendous. Thank you.
55
Recommended
