Wil Stuckey@quiiver
25 Million Members
25 Million Members
900,000 Shops
`
25 Million Members
900,000 Shops
1.4B Page Views / Month
Managing Experimentation
Managing Experimentation
ManagingLaunches
1. Experiments2. Rampups3. Communications
Life Before Continuous Deployments
Initial Idea &Planning
Geocities-ize
Initial Idea &Planning
Design & Development
Initial Idea &Planning
Deploy &Launch!
Design & Development
Pain?
Testing?
Communication?
How can we do better?
Living in a Continuous
Deployment World
Total Deploys: 4,040Total Commits: 45,447Total Unique Authors: 220
2013 ... So far
Total Deploys: 4,040Total Commits: 45,447Total Unique Authors: 220
Average Deploys per day: 35.75Average Authors per deploy: 7.01Average Commits per deploy: 11.25
2013 ... So far
That’s a lot of people
We are only halfway through the year!
CHAOS
Config System
$server_config['my_awesome_feature'] = [ 'enabled' => 10, 'admin' => 'on', 'groups' => [1234], 'users' => ['wstuckey', 'dmckinley'], ];
Branching in Code
if (Feature::isEnabled('my_awesome_feature')) { // display my_awesome_feature...} else { // continue on}
Typical LaunchCycle
Initial Idea &Planning
Initial Idea &Planning
Design & Development Launch!
Initial Idea &Planning
Design & Development Launch!
Internal Admin Launch
Initial Idea &Planning
Design & Development Launch!
Internal Admin LaunchPublic Prototype
Initial Idea &Planning
Design & Development Launch!
Internal Admin LaunchPublic Prototype50% Experiment
• Small iterative changes
• Small iterative changes
• Product exists in prod as soon as possible
• Small iterative changes
• Product exists in prod as soon as possible
• Can be available for small groups for testing and feedback.
• Small iterative changes
• Product exists in prod as soon as possible
• Can be available for small groups for testing and feedback.
• Easy to communicate milestones because they happen as part of the development process.
Because of the constant change you
have to constantly communicate.
Start Simple
Wiki + Email
• Works great in the beginning
Wiki
• Works great in the beginning
• Quickly becomes overwhelming as you increase your concurrent launches.
Wiki
• Works great in the beginning
• Quickly becomes overwhelming as you increase your concurrent launches.
• Requires manual labor to keep stakeholders informed
Wiki
FRUSTRATION
We can do better!
Frank Harris@hirefrank
Launch Calendar
Launch Calendar
• Simple web app
• Collect structured meta-data about the launch
• Central location for upcoming, current and past launch information
Launch CalendarCollect launch specific meta-data
Launch Calendar
• Owner
• Launch Date
• Internal Launch Date
• Team
• Expected Reaction
Collect launch specific meta-data
Launch Calendar
✉Make your email work for you
✉
Launch Calendar
• Automated emails to product owners around deadlines
• Daily emails to product groups containing upcoming and recently launched launches
Make your email work for you
Launch CalendarMake your email work for you
✉
Launch Calendar
• Highly utilized across the organization
• During the year+ that it was running we recorded and tracked over 275 launches
How did it go?
“This beats the crap out of the wiki page in every conceivable way.” - real
feedback
What could we do better?
What would a more integrated product look
like?
Can we automate more?
We can do better!
CatapultA unified launch management tool.
“I believe that by changing the ‘add to cart’ button to
blue we will increase conversion rate.”
Catapult
$server_config['my_awesome_button'] = [ 'enabled' => 50, 'admin' => 'on',];
Remember this?
Hack, Deploy, Hack, Hack Deploy!
Integration is awesome.
Communication is awesome.
Start Simple.
Build process that enable you to Ship.
Never stop deploying!
Thank You!
Wil Stuckey@quiiver
Recommended
