Embed Size (px)
Citation preview
Engineering: Weekly Tech Talk
Startup Survival Lessons
Martin CozziMay 2014
Developer
Development environment
● Install all the things
Development environment
● Install all the things● Manually
Development environment
● Install all the things● Manually● Don’t document any of the steps
Development environment
● Install all the things● Manually● Don’t document any of the steps● Or the versions
Development environment
● Install all the things● Manually● Don’t document any of the steps● Or the versions
Developersdevelopers developers
Development environment(s)
● Install all the things | What things?
Development environment(s)
● Install all the things | What things?● Manually | Mac? Linux? Windows?
Development environment(s)
● Install all the things | What things?● Manually | Mac? Linux? Windows?● Don’t document any of the steps● Or the versions
○ Postgres 9.1, 9.2, 9.3, 9.4a○ ES 0.90.4, 0.90.11, master
Development environment(s)
● Install all the things | What things?● Manually | Mac? Linux? Windows?● Don’t document any of the steps● Or the versions
○ Postgres 9.1, 9.2, 9.3, 9.4a○ ES 0.90.4, 0.90.11, master○ Ruby 1.9.3, 2.0.0, 2.1.0, master
I have an idea
Development environment
● Install all the things● Manually● Don’t document any of the steps● Or the versions
Development environment
● Install all the things● Manually● Don’t document any of the steps● Or the versions
IN ONE PLACE
Development environment
Development environment
Development environment
Development environment
SPoF
Availability
Expectations
I have an idea
Development environment
Lesson 1
Configuration management
still by developers developers
Chef
chef-solo
Development environment
Development environment
● Install all the things● Manually● Don’t document any of the steps● Or the versions
code >
Configuration management
● Consistent workstation○ developers○ sandbox○ production
● Shared● Code
Lesson 2
Deployments
Versionsmatter.
dev -> sandbox -> prod
dev -> sandbox -> prod
dev -> sandbox -> prod
dev -> sandbox -> prod
dev -> sandbox -> prod
dev -> sandbox -> prod
dev -> sandbox -> prod
dev -> sandbox -> prod
dev -> sandbox -> prod
Blue Green
Blue Green deploys
Blue Green deploys
Blue Green deploys
Blue Green deploys
Blue Green deploys
Canary
Canary deploys
Lesson 3
OwnershipIf you wrote it support it.
Test it
Measure it
Log it
Monitor it
Externally too
Externally too
Watch it
bulletins over time
Look for the signs
Lesson 4
PrioritizationPick your battles
Prioritization
● Priority○ p0, p1, p2, p3○ Critical, High, Medium, Low
Prioritization
● Priority○ p0, p1, p2, p3○ Critical, High, Medium, Low
● Channels○ Email○ Hipchat○ Text○ Phone call
Get some sleep!
Lesson 5
CommunicationEscalate, but don’t panic.
NotifySupport, Bizdev, Engineering
Fixbut don’t take shortcuts
UpdateSupport, Bizdev, Engineering
Post MortemsHelp you learn from your mistakes.
TestsCatch it early on next time
Automate
Lesson 5
Cattle not Cats(don’t be sad)
If someone knows an IPnuke the box
(clouds don’t have feelings)it’s OK
Lesson 6
Time > MoneyYMMV
Blackboxes are great
Instant value
EBS
EBS
● Stop your instance
EBS
● Stop your instance● No data loss on deletion (instance)
EBS
● Stop your instance● No data loss on deletion (instance)● Resize
EBS
● Stop your instance● No data loss on deletion (instance)● Resize● Snapshot
EBS
● Stop your instance● No data loss on deletion (instance)● Resize● Snapshot● IOPS
Blackboxes are greatuntil they break
Monitor Cost
High Level
AWS Breakdown
AWS / service breakdown
Conclusion
● Code > Wiki
Conclusion
● Code > Wiki● Systems fail
Conclusion
● Code > Wiki● Systems fail● Own your stuff
Conclusion
● Code > Wiki● Systems fail● Own your stuff● Communicate
Conclusion
● Code > Wiki● Systems fail● Own your stuff● Communicate● Track your costs
Conclusion
● Code > Wiki● Systems fail● Own your stuff● Communicate● Track your costs● Be proud
Questions?
