Upload
perforce
View
641
Download
0
Embed Size (px)
DESCRIPTION
Splitting a large depot can be a daunting task, and a hidden difficulty is the impact to the bug tracking system. This session will explain how Job Content Synchronization allows multiple depots to share a single link with the bug tracking system, eliminating needed changes and significantly reducing split preparation time. Adding new depots and supporting further splits is dramatically simplified.
Citation preview
Extending Perforce Scalability Using Job Content Synchronization Shannon Mann Software Developer Research In Motion
Outline
• Problem Definition • Brief History • Investigation
• Solution: Job Content Synchronization • JCS Concept • JCS Details
• Launch • Implementation • Benefits • Concerns
• Wrap-up • Conclusions • Questions
Problem Definition
Brief History
• Early 2000 – first Perforce use • Mid 2005 – large scale automation introduced
• Automated parallel builds • Dramatic growth of continuous integration • Performance issues began
• Late 2006 – performance severe • Decision to split
• Mid 2007 – split not possible at this time • Code inter-dependencies too high
Brief History
• Mid 2007 - Alternatives sought • Monitoring introduced • Hardware replaced - Sparc to Opteron • OS replaced – Solaris to Linux • Performance improved 10X
• Continued accumulated improvements by 100X • Initial 2007 samples at 22 seconds, now .2 seconds
• Userbase activities increased by 6X • Early 2010 - Decision to split revisited
Investigation
• Decision to split by duplicating db and depot data • separate each depot’s trees with protections • best done with explicit protections
• Reduce cost with post-split cleanup • clientspec cleanup reduces db.have size • Offline obliterates completes split
• leaves depot files as-is – expensive • Depot cleanup needed – future research topic
Investigation
• Split Challenges • large number of tool integrations • multiple groups with conflicting needs • moving product delivery dates
• Bug-tracking integration complex – show-stopper • Bug-tracker itself integrated with many tools • Integration complexity dramatically increased • Alternative desperately needed
• Goal - avoid changing bug-tracker integration
• Job Content Synchronization concept saves project
Solution: Job Content Synchronization
JCS Concept
• Middleman depot allows existing bug-tracking integration unchanged
• JCS copies job data from middleman to production
• Perforce Support gave enthusiastic support • Produced reference solution in two weeks
JCS Concept
Bug Tracker
Original Depot
P4DTG
Bug Tracker
P4Jobs Depot
P4DTG New Depot
JCS
New Depot
JCS Details
• Stand-alone depot has P4DTG integration with bug-tracker • Needs P4Broker to redirect ‘p4 jobs’ • Needs P4Auth to make redirect transparent • Needs P4Change to make connections unique • Needs triggers to request initial job data • Uses P4DTG in a depot-to-depot mirror of job data
JCS Details
Bug Tracker
P4Jobs Depot
New Depot
P4Auth P4Change
P4Broker
Users
P4DTG
JCS
Launch
Implementation
• Tested using full test implementation • All parts must be present • Users must test their tool integrations
• Staged production release • P4Broker, P4Auth and P4Change implemented first • Split done at later date – reduces risk • Additional depots added as needed
Benefits
• Greatly increased scalability
• Depot-side and bug-tracker side connections increase linearly and independently
• Near-trivial low-impact scalability
• Non-split depots benefit – bug-tracking connect for free
Concerns
• Significant increase in deployed complexity
• Expert knowledge of subsystems now required
• Complexity extends investigations and corrections
Concerns
Bug Tracker
Original Depot
Users Original Setup
Concerns
Bug Tracker
P4Jobs Depot
Users
P4Auth P4Change
New Depot
One Depot w/JCS
Concerns
Bug Tracker
P4Jobs Depot
Users
P4Auth P4Change
New Depot
New Depot
Two Depots w/JCS
Wrap-up
Conclusions
• JCS allows near-linear scaling of bug-tracking interconnects to multiple depots
• JCS allows a reduction in performance impact
• JCS eliminates split impact to bug-tracking system integrations
Conclusions (cont.)
• JCS also allows multiple non-split depots to share bug-tracking integration
• JCS can be used just for bug-tracking integration sharing • Splits aren’t required to consider using
• P4Auth, P4Change and P4Broker can be used alone
• Perforce Support are your friends. Seek their advice
Questions?