81
Lessons Learned Building Distributed Systems At Bitly by @theSeanOC

Lessons Learned with Distributed Systems at Bitly

Embed Size (px)

Citation preview

Page 1: Lessons Learned with Distributed Systems at Bitly

Lessons Learned Building Distributed Systems At Bitlyby @theSeanOC

Page 2: Lessons Learned with Distributed Systems at Bitly

shortlinks?

Page 3: Lessons Learned with Distributed Systems at Bitly
Page 4: Lessons Learned with Distributed Systems at Bitly

flickr.com/photos/lobo235/75085378

Page 5: Lessons Learned with Distributed Systems at Bitly
Page 6: Lessons Learned with Distributed Systems at Bitly
Page 7: Lessons Learned with Distributed Systems at Bitly
Page 8: Lessons Learned with Distributed Systems at Bitly

How hard can it be?

230,000,000 redirects20,000,000 shortens

4,000,000 page crawls

400 servers2 data centers

each day we handle

and we do it with

20+ deploys20 engineers

EVERY DAY

Page 9: Lessons Learned with Distributed Systems at Bitly

seanoc.co/1o1DLWP

Page 10: Lessons Learned with Distributed Systems at Bitly

Distributed Systems

Page 11: Lessons Learned with Distributed Systems at Bitly
Page 12: Lessons Learned with Distributed Systems at Bitly

A distributed system is a software system in which components located on networked computers communicate and coordinate their actions by passing messages. The components interact with each other in order to achieve a common goal. Three significant characteristics of distributed systems are: concurrency of components, lack of a global clock, and independent failure of components.

- Distributed Systems: Concepts and Design via seanoc.co/ds-cite

Page 13: Lessons Learned with Distributed Systems at Bitly

A distributed system is a software system in which components located on networked computers communicate and coordinate their actions by passing messages. The components interact with each other in order to achieve a common goal. Three significant characteristics of distributed systems are: concurrency of components, lack of a global clock, and independent failure of components.

- Distributed Systems: Concepts and Design via seanoc.co/ds-cite

Page 14: Lessons Learned with Distributed Systems at Bitly

A distributed system is a software system in which components located on networked computers communicate and coordinate their actions by passing messages. The components interact with each other in order to achieve a common goal. Three significant characteristics of distributed systems are: concurrency of components, lack of a global clock, and independent failure of components.

- Distributed Systems: Concepts and Design via seanoc.co/ds-cite

Page 15: Lessons Learned with Distributed Systems at Bitly

A distributed system is a software system in which components located on networked computers communicate and coordinate their actions by passing messages. The components interact with each other in order to achieve a common goal. Three significant characteristics of distributed systems are: concurrency of components, lack of a global clock, and independent failure of components.

- Distributed Systems: Concepts and Design via seanoc.co/ds-cite

Page 16: Lessons Learned with Distributed Systems at Bitly

A distributed system is a software system in which components located on networked computers communicate and coordinate their actions by passing messages. The components interact with each other in order to achieve a common goal. Three significant characteristics of distributed systems are: concurrency of components, lack of a global clock, and independent failure of components.

- Distributed Systems: Concepts and Design via seanoc.co/ds-cite

Page 17: Lessons Learned with Distributed Systems at Bitly

A distributed system is a software system in which components located on networked computers communicate and coordinate their actions by passing messages. The components interact with each other in order to achieve a common goal. Three significant characteristics of distributed systems are: concurrency of components, lack of a global clock, and independent failure of components.

- Distributed Systems: Concepts and Design via seanoc.co/ds-cite

Page 18: Lessons Learned with Distributed Systems at Bitly

A distributed system is a software system in which components located on networked computers communicate and coordinate their actions by passing messages. The components interact with each other in order to achieve a common goal. Three significant characteristics of distributed systems are: concurrency of components, lack of a global clock, and independent failure of components.

- Distributed Systems: Concepts and Design via seanoc.co/ds-cite

Page 19: Lessons Learned with Distributed Systems at Bitly
Page 20: Lessons Learned with Distributed Systems at Bitly

=

Page 21: Lessons Learned with Distributed Systems at Bitly

seanoc.co/1tSEw7d

Page 22: Lessons Learned with Distributed Systems at Bitly

seanoc.co/1tSGpAB

Page 23: Lessons Learned with Distributed Systems at Bitly

seanoc.co/1tSHxo7

Page 24: Lessons Learned with Distributed Systems at Bitly

seanoc.co/1tSHZCC

Page 25: Lessons Learned with Distributed Systems at Bitly

seanoc.co/1ko4e0F

Page 26: Lessons Learned with Distributed Systems at Bitly

seanoc.co/1ko5QHG

Page 27: Lessons Learned with Distributed Systems at Bitly

seanoc.co/1m8WhYQ

Page 28: Lessons Learned with Distributed Systems at Bitly

http://seanoc.co/1ko6hlm

Page 29: Lessons Learned with Distributed Systems at Bitly

seanoc.co/1ko73Pp

Page 30: Lessons Learned with Distributed Systems at Bitly

http://seanoc.co/1ko73Pp

Page 31: Lessons Learned with Distributed Systems at Bitly

seanoc.co/1ko8oG0

Page 32: Lessons Learned with Distributed Systems at Bitly

seanoc.co/1ko91iL

Page 33: Lessons Learned with Distributed Systems at Bitly

seanoc.co/1ko9gdu

Page 34: Lessons Learned with Distributed Systems at Bitly
Page 35: Lessons Learned with Distributed Systems at Bitly

seanoc.co/1koaiXb

Page 36: Lessons Learned with Distributed Systems at Bitly

seanoc.co/1koaKEQ

Page 37: Lessons Learned with Distributed Systems at Bitly

seanoc.co/1kobGJk

Page 38: Lessons Learned with Distributed Systems at Bitly
Page 39: Lessons Learned with Distributed Systems at Bitly

seanoc.co/1kodphC

Page 40: Lessons Learned with Distributed Systems at Bitly

Service Oriented Architecture

Page 41: Lessons Learned with Distributed Systems at Bitly

http://gph.is/15uC6Rf

Page 42: Lessons Learned with Distributed Systems at Bitly

seanoc.co/1koeq9n

Page 43: Lessons Learned with Distributed Systems at Bitly

seanoc.co/1koeMgs

Page 44: Lessons Learned with Distributed Systems at Bitly

seanoc.co/1kofgD4

Page 45: Lessons Learned with Distributed Systems at Bitly

flickr.com/photos/75905404@N00/7126146307

Page 46: Lessons Learned with Distributed Systems at Bitly

flickr.com/photos/miguel77/5541356248

Page 47: Lessons Learned with Distributed Systems at Bitly

flickr.com/photos/3336/155407777

Page 48: Lessons Learned with Distributed Systems at Bitly

Async > Syncexcept when it isn’t

Page 49: Lessons Learned with Distributed Systems at Bitly

flickr.com/photos/beautyredefined/2389559961

Page 50: Lessons Learned with Distributed Systems at Bitly

flickr.com/photos/dtanist/10651668875

Page 51: Lessons Learned with Distributed Systems at Bitly

flickr.com/photos/onkel_wart/3601611572

Page 52: Lessons Learned with Distributed Systems at Bitly

flickr.com/photos/mikemcilveen/5187192661

Page 53: Lessons Learned with Distributed Systems at Bitly

flickr.com/photos/pauldineen/4049601480

Page 54: Lessons Learned with Distributed Systems at Bitly

flickr.com/photos/andreasnilsson1976/287956009

Page 55: Lessons Learned with Distributed Systems at Bitly

flickr.com/photos/a2community/1387603666

Page 56: Lessons Learned with Distributed Systems at Bitly
Page 57: Lessons Learned with Distributed Systems at Bitly
Page 58: Lessons Learned with Distributed Systems at Bitly

Events > Commands

Page 59: Lessons Learned with Distributed Systems at Bitly

seanoc.co/1kogw9x

Page 60: Lessons Learned with Distributed Systems at Bitly

seanoc.co/1kohCSx

Page 61: Lessons Learned with Distributed Systems at Bitly

seanoc.co/1koi7fk

Page 62: Lessons Learned with Distributed Systems at Bitly

seanoc.co/1koijLN

Page 63: Lessons Learned with Distributed Systems at Bitly

seanoc.co/1koiTsU

Page 64: Lessons Learned with Distributed Systems at Bitly
Page 65: Lessons Learned with Distributed Systems at Bitly

seanoc.co/1kok4IG

Page 66: Lessons Learned with Distributed Systems at Bitly

Dealing With Failure

Page 67: Lessons Learned with Distributed Systems at Bitly

seanoc.co/1kokvmk

Page 68: Lessons Learned with Distributed Systems at Bitly
Page 69: Lessons Learned with Distributed Systems at Bitly

seanoc.co/SrO6iK

Page 70: Lessons Learned with Distributed Systems at Bitly

seanoc.co/1kolSBx

Page 71: Lessons Learned with Distributed Systems at Bitly

seanoc.co/1komS8F

Page 72: Lessons Learned with Distributed Systems at Bitly

Monitoring

Page 73: Lessons Learned with Distributed Systems at Bitly

seanoc.co/1iMl7QC

Page 74: Lessons Learned with Distributed Systems at Bitly

A distributed system is one in which the failure of a computer you didn't even know existed can render your own computer unusable.

- Leslie Lamport

Page 75: Lessons Learned with Distributed Systems at Bitly
Page 76: Lessons Learned with Distributed Systems at Bitly

seanoc.co/1kopRxX

Page 77: Lessons Learned with Distributed Systems at Bitly
Page 78: Lessons Learned with Distributed Systems at Bitly
Page 79: Lessons Learned with Distributed Systems at Bitly

=

Page 80: Lessons Learned with Distributed Systems at Bitly

bit.ly/jobs

Page 81: Lessons Learned with Distributed Systems at Bitly

Thanks!