Upload
perforce
View
1.363
Download
2
Tags:
Embed Size (px)
DESCRIPTION
Jeppesen uses a combination of their own data centers and a variety of 3rd party cloud providers to support their distribution of navigation data to most of the world’s pilots. But before any data leaves the building it goes through Perforce. Learn why Jeppesen chose to put Perforce at the heart of their global data distribution system, and how you too could leverage Perforce to help provide atomic changesets and true history tracking to your cloud-scale object stores.
Citation preview
#
Daniel AnolikSolutions Architect, Jeppesen
Version Your Cloud
#
Daniel AnolikSolutions Architect, Airborne Navigation SystemsJeppesen (A Boeing Company)• 8 years with Jeppesen
• 15 years of direct SCM experience, 8 with Perforce
• Software developer, solutions architect, Scrum master, bluegrass mandolin picker, wilderness survival enthusiast…
#
Daniel AnolikSolutions Architect, Airborne Navigation SystemsJeppesen (A Boeing Company)• 8 years with Jeppesen
• 15 years of direct SCM experience, 8 with Perforce
• Software developer, solutions architect, Scrum master, bluegrass mandolin picker, wilderness survival enthusiast,
and lucky father of this baby girl:
#
Jeppesen at a Glance• 80-year strong company founded in aviation safety• Wide array of software supporting:
– airborne navigation systems– flight planning services– pilot training – airline operations management
#
Supplier of Critical Aviation Data
#
• Distribute many data types to many applications running on many platforms
• Support frequent data updates for hundreds of thousands of customers around the world
• Transfer hundreds of terabytes of data each month• Don’t be a bottleneck!
The Challenge
#
Initial Plan
Data Brokering Magic
Jeppesen DataProduction
Magic
JeppesenInfrastructure CDM
Cloud Object
Storage
Workflow Magic
Data PublishingInterface
Boring File Server
#
The system must support:• Fine read/write access controls within storage tree
• Atomic change-sets for publishing files
• Detailed history tracking
• Individual file restores
• Data lifecycle management
Hmm…It’s starting to sound like we need a real CMS system in there
Pesky Implementation Constraints
#
Improved Plan
Data Brokering Magic
Jeppesen DataProduction
Magic
JeppesenInfrastructure CDM
Cloud Object
Storage
Workflow Magic
Data PublishingInterface
#
• Supported all our implementation constraints with minimal customization
• Leverage existing in-house expertise
• P4D had very stable track record at Jeppesen
• Efficient, scalable blob storage
Why Perforce vs Other CMS?
#
• Keeping changes atomic:– File sets are submitted to P4 as single changelist– Data is processed by the changelist– Data is published by the changelist
• Files are never ‘available’ to our applications until all files in a given changelist are available
Leveraging Perforce: Changelists
#
• Changelists tell us:– What files were published,– when,– and by whom
• Spec depots!• Soft deletes of files minimize ‘clutter’ while still
supporting data retention requirements
Leveraging Perforce: History
#
• Custom counters– are defined for each data type
– track last successful changelist processed
– provide a central, durable mechanism for state tracking
across different work nodes in the system
Leveraging Perforce: Counters
#
• Job queue are simple files in p4
• In process jobs are ‘locked’
• Completed jobs are ‘deleted’
• Full job history is retained
Leveraging Perforce: Locks
#
Leveraging Perforce: Branching
Private “Staged” Data
Public “Promoted” Data
p4 integrate
Depot paths represent the promotional model of the data
#
• P4 protections restrict access by:– depot path/data library– pattern matching/file extensions – IP addresses
• Service accounts granted read and write access to only what they need
Leveraging Perforce: Security
#
• 100% of interaction with Perforce fully automated through the P4Java API
• Custom utilities used to:– Read and write data in depot
– Manage branch integrations
– Read and update counters
– Read changelist details
– ‘Expire’ content
Leveraging Perforce: Java API
#
All the great capabilities of cloud based object storage, plus true configuration management:
– Detailed history tracking and extreme auditability
– Reproducibility: roll back any part of your ‘cloud’ to any point in time
– Atomic changelist management of file sets
The Result: Object Storage++
#
Any Questions?
#