Git: The Lean, Mean, Distributed Machine

Chris Wanstrath

http://defunkt.github.com

hi everyone, i’m chris wanstrath.

how many people here use git?

i play guitar

i have a schecter classic similar to this. mine is prettier.

i’m from cincinnati ohio

but live in san francisco

i started as a lowly paid consultant

Then I worked at CNET

then worked at CNET for a few years

Then I worked at CNET

(which is now owned by CBS)

Then I was a highly paid consultant

after that i became a highly paid consultant

before co-founding github

gitThe Lean Mean Distributed Machine

By Chris Wanstrathanyway, i want to talk a bit about git today

if we’re going to talk about git, we need to start by talking about source control management(SCM)

or version control

basically version control is like wikipedia

for your code

you use it to see what changes others made

inspect those changes

and contribute your own

gitwho uses ?token promo slide

these companies use git

and so do these open source projects

let’s briefly go through the history of open source SCMs

(at least the ones we care about)

Revision Control System (RCS) was written in the early 80s and used to store history on a file by file basis. A directory of source code could contain many RCS repositories, each concerning itself with a single file.

it’s like a hut - very basic, primitive even, but works great when all you need is shelter

Later that decade, a professor began working with two grad students on a C compiler (in the name of scholarly pursuits). As they began using RCS, the professor noted a number of limitations. It was difficult to share files, and even more difficult to share entire projects.

So, they wrote CVS - the Concurrent Versioned System - and released it as open source in the early 1990s. Concurrent because it allowed multiple individuals to collaborate on a project together, without stepping on each other's toes, and versioned system because it was initially a collection of RCS repositories with network awareness.

CVS was like a cabin. better than a hut, but still pretty crappy

it worked well for a while, but there were limitations. Dealing with directories was difficult, and much different than the way one would normally deal with directories in Unix.

Ten years later (see a pattern?) a new revision control system was released, called Subversion (or SVN). Subversion was intended to replace CVS by improving on CVS. History, directories, deletions, and other CVS warts were fixed. mod_dav integration was included, as well as anonymous checkout. (Anonymous checkout in CVS was literally a hack added by the OpenBSD.)

Subversion was not subversive, but it did work well enough. Many felt it a welcome relief and hurriedly switched. Big, lumbering organizations spent years converting their repositories to SVN. IDEs and editors included Subversion integration.

it was like a house, same idea but much better than a cabin

Committer Committer Committer

Server

this is the rcs / cvs / svn model.

Server

someone commits to the server

Server

and everyone else pulls down the changes

this is bad

Server

first off, the server is the babysitter

SVN’s down!

“The Subversion server’s down”

you can’t do anything without the server’s permission

second, low visibility into your coworkers and subordinates’ activity.

at cnet, we used bugzilla. it was awesome (as you can see)

we’d also get diffs emailed to us after each commit

because our group was large, and fluid, i’d often get commits emailed to me i didnt care about

or understand

this meant i spent extra time throwing away junk

emails that come to me should be for me

if you’re lucky, you use trac to watch what everyone is doing.

it has rss and is a bit smarter.

usually you have to set it up yourself

another problem: your subversion workflow is single threaded

it’s hard to stop working on a feature to quickly fix a bug without losing your feature’s changes

changed files are either committed or discarded

well, not entirely true...

you could always use a branch

but that sucks, and big changes are usually disasters

i once worked on a project for 3 months where we worked on a massive new feature

as it neared completion, we had a big meeting

a 2 hour meeting

there we decided how to merge in the changes from our branch to trunk

it was one of the worst meetings ever

and i’ve been in some bad meetings

one person was assigned with the task of merging the branch and trunk

he was merging and fixing bugs in code he did not write

or understand

it did not go well

another problem with the ‘babysitter’ model is that experimentation is difficult

all your experiments are public

everyone sees everything you commit

solution? don’t commit

at least, that was my solution

and that’s just work stuff

the centralized model, when applied to open source, is a huge pain

it’s difficult to maintain patched versions of open source independently

if a project dies on the internet, does anyone care?

what’s the answer?

dvcs! git! happy!

git was started by linus torvalds

a DVCS built as an efficient content addressable file system

the guy who started the linux kernel

if rcs is a hut

cvs is a cabin

and subversion is a house

git is a castle

or a ninja

or shaq

Server

the thing that makes git, and all distributed version control systems like it different, is the idea that it’s “distributed”

take this centralized model

Server

Committer CommitterCommitter

and make every copy of the code its own, full fledged repository

any copy can accept or create commits. anyone can pull commits from any copy.

Server

now you can push to the server

Server

or committers can push and pull from each other

Committer

Server

instead of “checking out” code

Server

Committer

you “clone” or copy a repository

if github explodes, you don’t lose any code

Server

Committer

ServerServer

in fact, because you have a full copy of your repository at all times, you don’t need to tie yourself to a single remote repository

you can push to as many servers as you want

another word for ‘clone’ or ‘copy’ is ‘fork’

it may seem like anarchy at first

but sane and useful workflows have evolved

in fact, the first workflow i want to talk about is called Anarchy

Server

you remove the server

then make everyone a peer

Committer

CommitterCommitter

then take away commit access from each other

CoderCoder

and you end up with repositories floating in the void

this is how the internet works, or how git works by default

CoderCoder

everyone pushes and pulls from each other, managing their own version of the code

for small projects with very few contributors, it works fine

and it would work great on small, experimental projects inside of any organization

(you just need a place to publish your changes)

an example of this, let’s say i was on github and i wanted to add a patch to schacon’s ticgit

i’d click the fork button

i now have a copy of schacon’s ticgit called defunkt’s ticgit

now i can make changes and ask scott to check them out. if he likes them, he’ll merge them in

if he doesn’t like them, oh well. i can still use them in my project and keep up to date with his changes.

someone can come along and fork from me if they want, too

#2blessed

but anarchy doesn’t scale

the second workflow is called Blessed

CoderCoder

the blessed workflow has the same basic idea as Anarchy

Blessed

CoderCoder

but one of the repositories is the Blessed repo

Blessed

CoderCoder

everyone takes their cues from the blessed repository

its development is considered the mainline, or trunk

deploys and packages are pushed from the blessed repo

Blessed

CoderCoder

others can still push and pull from each other, remember

in the business world this works great for dealing with contractors

Blessed

CoderCoder

don’t give them push access, just pull access

Blessed

they pull down your code, make their changes, then tell you when the changes are ready

Blessed

if you like what you see, you merge in the contractor’s changes

the contractor never has direct write access to your company’s code

if there were a bunch of us working on ticgit, scott’s may be the Blessed repository

he started the project and is in charge of merging in all changes. we all watch his changes

this is how rails works

rails/rails is the Blessed repository, from which the gems are built and david controls.

we all follow this repo’s development and treat it as “official” by convention only

this is also how rentzsch’s click to flash works

click to flash is an amazing safari plugin that disables flash, similar to the firefox extension

it was forked from google code and has been given a life of its own on github, under rentzsch’s guidance. contributors fork his repository and he merges in good changes.

the plugin’s development has been a perfect example of how distributed version control puts the power in the hands of the developer, not the server

#3lieutenant

the next workflow is called ‘lieutenant’ - great for massive projects, like the kernel

Blessed

LieutenantLieutenant

CoderCoder

in this model, there is a blessed repository and a few designated lieutenants

the lieutenants are people trusted by the blessed repository

Blessed

CoderCoder

coders will pull from a lieutenant, make their changes, then request the lieutenant merge in their changes

Blessed

CoderCoder

lieutenants are usually in charge of a specific subsystem or part of the large system

if they like the change, they will pull it in

Blessed

CoderCoder

they’ll then inform the blessed repository that they have changes which need to be merged in

the blessed repo, trusting the lieutenant, pulls in the changes

Blessed

CoderCoder

this is typically coordinated over a mailing list

this is the kernel’s model

#4centralized

finally the centralized model, one repository acts as the ‘server’

Server

this mimics the traditional babysitter model

but you only need your babysitter to pull and push changes - branches and commits can still be created locally whenever

Server

the server being down does not dramatically hamper your work

Server

yet the flow stays mostly the same

DeployServer

the central server, as in the old model, can also be used to deploy

Production

Staging

and staging servers can easily be setup

git isn’t all about distributed servers

in fact, one of the best parts about git is its branching support

branches are local, incredibly lightweight, and easy to switch between

it’s easy to devote each branch to a single feature or bug

we call these ‘topic branches’

$ git checkout -b bug_2342

from your working directory

you just made a new branch

buckets of different things

because branches are so cheap, you can keep around buckets filled with experiments, new ideas, or new features

no one will ever see them unless you want them to be seen

Production

Staging

this changes the staging server idea

Production

Feature A Feature B

you may start to have topic staging servers where you boot up staging for a single branch and test out a new feature,

no more generic ‘staging’ branch - each person may even have their own staging server

because every copy is its own repository, we are given the freedom to structure our workflows socially rather than technically

Team A Team B

it’s easy to have multiple small teams, move people between projects, and monitor multiple projects

no need for one monolithic subversion server - git repositories are a breeze to setup

with something like github, watching your team’s development is trivial.

do it with rss...

or with a service you’re already comfortable using

integration with campfire, email, fogbugz, lighthouse, friendfeed, twitter, etc

the site also lets you comment on commits, providing dead simple and effective code review

git and github are what we use in our private client work and on our own websites, as well as for our open source

as far as git IDE support, the textmate ProjectPlus extension shows you the status of tracked files right in the drawer

there’s also a git textmate bundle available on github

if you’re an eclipse user, the egit plugin lets you commit to, manage, and track git repositories from within eclipse

it’s written using jgit, a pure-java implementation of git

emacs people can use DVC which aims to provide a common interface for all distributed version control systems

there’s also a git mode

or my person favorite, magit

if you use os x, an open source program called GitX is under active development

which is based on the cross platform Git-GUI

for OS X there’s also GitNub which isn’t as actively developed

as far as libraries go, a search for ‘git’ on github returns almost 3000 unique repositories

darcs or hg to git converters, git vim projects, git in .NET, even blogs and wikis based on git

remember when i said git was a content addressable file system? well, it’s true

this is gist. it’s a git powered pastie

you paste in code and share it with coworkers or friends

but the best part are these clone URLs

i can check out a pastie i made, make changes, then push a new version

these are the revisions

you can’t tell the difference between changes i made on the web and changes i made locally then pushed

this kind of stuff is the future

imagine a distributed, versioned wiki or documentation project or book

a distributed, versioned bug tracker

a distributed, versioned chat application

in fact, a number of books are already being written on github

scott’s book is being translated right now

a distributed, versioned everything

http://git-scm.comthis has been a fairly basic overview of git

for more information, check out git-scm.com

thanks questions?

http://flickr.com/photos/dooleymtv/2660521051/http://flickr.com/photos/shankrad/219945665/http://flickr.com/photos/thomashawk/268524287/http://flickr.com/photos/drp/41370809/http://flickr.com/photos/keithmarshall/432924465/http://flickr.com/photos/ivviphotography/2555026221/http://flickr.com/photos/raybyrne/43604789/http://flickr.com/photos/robertlz/481261885/http://flickr.com/photos/petervanallen/774522321/http://flickr.com/photos/seantubridy/389310649/http://flickr.com/photos/millzero/705902956/http://flickr.com/photos/larskflem/95757299/http://flickr.com/photos/dc5dugg/2330760034/http://flickr.com/photos/will-lion/2629598996/http://flickr.com/photos/probablykat/377911954/http://flickr.com/photos/pixelbuffer/5375185/http://flickr.com/photos/gordonhamilton/2096653379/http://flickr.com/photos/poolie/2250698836/http://flickr.com/photos/doergn/425407884/http://flickr.com/photos/miscellaneous/531864821/http://flickr.com/photos/hchalkley/30724738/http://flickr.com/photos/absence-is-steel/419515663/http://flickr.com/photos/samiksha/408007916/http://flickr.com/photos/keithallison/3230928864/

http://flickr.com/photos/baonguyen/2557970434/http://flickr.com/photos/bocavermelha/66759796/http://flickr.com/photos/dark-o/408532292/http://flickr.com/photos/vividbreeze/480057824/http://flickr.com/photos/12693492@N04/1338136415/http://flickr.com/photos/theducks/2236111019/http://flickr.com/photos/pfenwick/2229585929/http://flickr.com/photos/41232123@N00/457119628/http://flickr.com/photos/pedacitosdemi/2346980097/http://flickr.com/photos/aleks/2759584309/http://flickr.com/photos/bigfrank/71691236/http://flickr.com/photos/chinapix/2757393280/http://flickr.com/photos/28960190@N03/2766239909/http://flickr.com/photos/68226666@N00/444253460/http://flickr.com/photos/chilledsalad/2335688523/http://flickr.com/photos/thefuzz90/444183705/http://flickr.com/photos/domk/291808114/http://flickr.com/photos/csb13/66558459/http://flickr.com/photos/kalhusoru/1346688882/http://flickr.com/photos/heather_shade/254728862/http://flickr.com/photos/ostromentsky/479147675/http://flickr.com/photos/flopper/1336613530/http://flickr.com/photos/jagelado/16631508/http://flickr.com/photos/ansy/2533871446/http://flickr.com/photos/cdm/54246114/http://flickr.com/photos/chibnall/2278573838/

flickr

http://www.flickr.com/photos/andrer69/246847221/http://www.flickr.com/photos/willstotler/758104340/

Git: The Lean, Mean, Distributed Machine

Business

Cookham Wood Lean & Mean Jane Foulkes

Lean mean it machine

The Lean Hospital: What does it mean? - University of ... · PDF fileThe Lean Hospital: What does it mean? Kristin Furfari, MD. Outline • Waste in the US Health Care System • Lean

Lean and mean MongoDB

Lean &Mean Borland C++

Lean, Mean Fighting Machine

CSN From Lean & Mean to Lean & Meaningful CSN From Lean & Mean to Lean & Meaningful Frans van der Reep London, May 15th 2009 Somesso

Lean mean php machine

Lean & Mean Tokyo Cabinet Recipes (with Lua) - FutureRuby '09

Lean, Mean Design Machine!

Lean and Mean Blueprint Manual

From Lean & Mean to Lean & Meaningful

What Does Lean Six Sigma Really Mean?

THE LEAN MEAN FAT REDUCING GRILLING MACHINE TM

From Lean & Mean to Lean & Meaningful CSN 2011 Amsterdam

Lean Mean Programming Machine

Building A Lean Mean Design Team

Time Management: Become a Lean, Mean Productive Machine

Mark robinson what does lean mean for software testing

LEAN & MEAN - GO MICROSERVICES WITH DOCKER SWARM MODE …callistaenterprise.se/assets/presentationer/golang-microservices... · lean & mean - go microservices with docker swarm mode