Upload
imogen-haynes
View
218
Download
1
Embed Size (px)
Citation preview
Copyright Feature Creep 2008Copyright Feature Creep 2008
Product X Release Y:Product X Release Y:Release Gate V1Release Gate V1
Josephine SoapFreda BloggsHugh Jarse
Copyright Feature Creep 2008
Agenda
Refresh on Feature Creep Delivery Framework
Product Management status
Product Quality
Marketing Status
Support & Maintenance Status
Services Status
Sales Status
Recommendation
Approvals
Copyright Feature Creep 2008
OverviewOverviewPhase 1Concept
Product Delivery Team Formed Post Release
Review
Release Requirements
Integration Test
Phase 2Definition & Planning
Phase 3Development
Phase 4Readiness
Phase 5Release
Integration Complete
DesignComplete
Development
Test
Maintenance
Integrated Project Plan
Secure Field Trial Customer
Product Architecture Document Implementation and Unit Test
ArchitectureAnalysis
Optional Phase Review
Draft Product DescriptionProduct
Management
Project Plan Baselined
Test Plan and Test Cycle Design
Field Trial Plan
Test Strategy and Planning
Final Product Description
Product Delivery Team Leader and Project Mgmt
Design Specification
DevelopmentCommitted
Concept Committed
On the Radar
Documentation
Concept Complete(RPG Review)
Definition & PlanningComplete(RPG Review)
DevelopmentComplete(RPG Review)
ReadinessComplete(RPG Review)
Re-work Software
Update Documentation
Feature Complete
Add S&M Requirements to PRD
Field Trial
Requirements Baselined
Product Documentation Development
Training Assessment Training ExecutionTraining Materiel Development
Test Execution
Phase Review
Feature Status
Functional Specification
Install and Deployment Plan
Supportability Review
ReleaseComplete(RPG Review)
TC
Product Delivery Team Dissolved
GACR
TechnologyEvaluation
Documentation Plan
Design Documentation Review
Maintenance Acceptance Criteria
Project Task
Deliverable
Optional Optional
TC = Test CompleteCR = Controlled ReleaseGA = General AvailabilityKey
Servicesinc. Training
Install & Deploy + Partner support
Maintainability Review
PRD (Product RequirementsDocument)
Role
Support Operational Impact Statement
Support
Unit Test Plan
Updated Roadmap
Release Notes
Support Training
Alpha TrialServices Training
Certification
Test Preparation
Sales &Marketing
Business Case
Sales Validation(with customers)
Go-To-MarketStrategy(inc Channel Strategy)
(Updated) Go-To-Market Plan
Go-To-Market Execution
Sales Planning and Account TargetingMRD (Market Requirements Document) Sales Collateral
Sales Training
Add Services Requirements
Performance Reqs
Benchmarks
Prescriptive Architecturesand Activity Profiles
Developer Performance Guidelines
PerformanceDesign Changes
Perf designanalysis
Tuning & Benchmarking
Tuning/sizingguidelines
Custom Benchmarking
Iterative performance adjustments, tuning and testing
Iterative performance adjustments, tuning and testing
Platform Defn
UpdatedPlatform Defn
Copyright Feature Creep 2008
Milestone Reviews – what is presentedMilestone Reviews – what is presented
Phase 1: Concept
Phase 2: Definition and
Planning
Phase 3: Development
Phase 4: Readiness
Phase 5: Release
• Product Concept
• Fit with strategy and roadmap
• Marketing Requirements
• Target market and size
• Value proposition to customer
• Product Requirements
• Business case• Project
schedule• Resources
required• Project costs
• Gap between product requirements and actual product
• Updated business case
• Updated project schedule and costs
• Test results• Field trial
feedback• Launch plan• Updated
business case• Updated
project schedule and costs
• Initial bookings and revenue
• Review of development effort
• Retirement plan
Copyright Feature Creep 2008
Release Review ObjectivesVerify that: Product is at sufficient level of quality for trial customers
Software release media/mechanisms are prepared
Trial customers have been secured and plans in place
Pricing/packaging defined and communicated
Marketing/Sales materials are in place
Customer/Partner engagement plan is in place
Support organisation is ready
Services organisation is ready
Sales organisation is ready
Copyright Feature Creep 2008
Phase 1Concept
Product Delivery Team Formed
Post ReleaseReview
Release Requirements
Integration Test
Phase 2Definition & Planning
Phase 3Development
Phase 4Readiness
Phase 5Release
Integration Complete
DesignComplete
Development
Test
Maintenance
Integrated Project Plan
Secure Field Trial Customer
Product Architecture Document Implementation and Unit Test
ArchitectureAnalysis
Optional Phase Review
Draft Product DescriptionProduct
Management
Project Plan Baselined
Test Plan and Test Cycle Design
Field Trial Plan
Test Strategy and Planning
Final Product Description
Product Delivery Team Leader and Project Mgmt
Design Specification
DevelopmentCommitted
Concept Committed
On the Radar
Documentation
Concept Complete(RPG Review)
Definition & PlanningComplete(RPG Review)
DevelopmentComplete(RPG Review)
ReadinessComplete(RPG Review)
Re-work Software
Update Documentation
Feature Complete
Add S&M Requirements to PRD
Field Trial
Requirements Baselined
Product Documentation Development
Training Assessment Training ExecutionTraining Materiel Development
Test Execution
Phase Review
Feature Status
Functional Specification
Install and Deployment Plan
Supportability Review
ReleaseComplete(RPG Review)
TC
Product Delivery Team Dissolved
GACR
TechnologyEvaluation
Documentation Plan
Design Documentation Review
Maintenance Acceptance Criteria
Project Task
Deliverable
Optional Optional
TC = Test CompleteCR = Controlled ReleaseGA = General Availability
Key
Servicesinc. Training
Install & Deploy + Partner support
Maintainability Review
PRD (Product RequirementsDocument)
Role
Support Operational Impact Statement
Support
Unit Test Plan
Updated Roadmap
Release Notes
Support Training
Alpha TrialServices Training
Certification
Test Preparation
Sales &Marketing
Business Case
Sales Validation(with customers)
Go-To-MarketStrategy(inc Channel Strategy)
(Updated) Go-To-Market Plan
Go-To-Market Execution
Sales Planning and Account TargetingMRD (Market Requirements Document) Sales Collateral
Sales Training
Add Services Requirements
Performance Reqs
Benchmarks
Prescriptive Architecturesand Activity Profiles
Developer Performance Guidelines
PerformanceDesign Changes
Perf designanalysis
Tuning & BenchmarkingTuning/sizingguidelines
Custom Benchmarking
Iterative performance adjustments, tuning and testing
Iterative performance adjustments, tuning and testing
Platform Defn
UpdatedPlatform Defn
Delivery Process: Checklist done will do Past due Will not doKey Will do later or delay
Copyright Feature Creep 2008
Agenda
Refresh on Feature Creep Delivery Framework
Product Management status
Product Quality
Marketing Status
Support & Maintenance Status
Services Status
Sales Status
Recommendation
Approvals
Copyright Feature Creep 2008
Product Management Status
• Summary of release content
• Gaps or variance from Development Committed plan
• Issues, risks, opportunities
Copyright Feature Creep 2008
Trial Customers
Customer Region Focus Start Date Probability
Customer A NAM Voice input 5/6/2008 100%
Customer B EMEA Integration with SharePoint 7/6/2008 100%
Customer C EMEA Clean Energy from ear wax 13/7/2008 60%
Customer D APAC Faster than light travel tbd 100%
Copyright Feature Creep 2008
Trial Plans – Customer A
• Business Problem: what will they do with the release?
• Environment: where/how will it be deployed?
• Trial size : how many users or systems?
• Users: what kind of users?
• Future: If successful, what deployment will follow the trial?
• Issues: What concerns, limitations, knowledge or resource gaps threaten a successful trial?
• Resource Plan: What resources are required and are they in place?
Copyright Feature Creep 2008
Agenda
Refresh on Feature Creep Delivery Framework
Product Management status
Product Quality
Marketing Status
Support & Maintenance Status
Services Status
Sales Status
Recommendation
Approvals
Copyright Feature Creep 2008
Product Quality Summary e.g. duration of testing, number of test
cases added, executed, new things tried, lessons, observations etc
Copyright Feature Creep 2008
Severity DefinitionsSeverity A (Issue Priority = 1 Critical)
A problem which will prevent the accomplishment of an essential capability and NO known work-around:
Show-stopper, production down Critical functionality does not work Data loss Causes irreversible (or reversible but with a lot of effort) database damage Repeated, service-affecting core dumps Security violations Unbounded memory growth Severe degradation (> 50%) in throughput performance below that specified in the PRD
Severity B (Issue Priority = 2 High) A problem will adversely affect the accomplishment of an operational capability and HAS
known work-around: Critical functionality of the product does not work but known work-around solution is
available Non-Critical functional issue that a customer is LIKELY to come across but known work-
around solution is availableSeverity C (Issue Priority = 3 Medium)
A problem will result in user/operator inconvenience or annoyance, but does not affect a required operational or mission-essential capability:
Customer is NOT likely to come across this with known work-around Non-Critical functionality issue with known work-around
Severity D (Issue Priority = 4 Low) Very minor problem resulting in negligible inconvenience or annoyance
Copyright Feature Creep 2008
Exit Criteria set at Development Commit
100% new feature test case coverage and pass rate 100% regression test case coverage Zero severity A’s and B’s across entire product 90% of severity C defects raised against new features are
closed 80% of severity D defects raised against new features are
closed No incidents in a status of Further Evidence Required No incidents in a status of Awaiting Test Evidence that defect cumulative count is levelling off
Copyright Feature Creep 2008
Test Case Coverage
100% new feature test case coverage and pass rate 100% regression test case coverage
Copyright Feature Creep 2008
Test Case Coverage
100% new feature test case coverage and pass rate 100% regression test case coverage
Copyright Feature Creep 2008
Incidents
Zero severity A’s and B’s across entire product
Severity Open Defects
A 0
B 2
Copyright Feature Creep 2008
Incidents
Incident Description Severity Justification
93866 Office – when saving a new version clicking cancel on version comment dialog loses all changes to document.
B – infrequent loss or corruption of user-added data.
Defect pre-exists X.YLimitation of Vista.
94237 Lookups – large number of lookups causes client API and memory errors.
B – frequent failure of important component of the system, no acceptable workaround.
Defect pre-exists X.YSignificant change to product, deemed too risky.Uncommon scenario.
Zero severity A’s and B’s across entire product
Copyright Feature Creep 2008
Incidents
Zero severity A’s and B’s across entire product
Severity Open Defects
A 0
B 8
Copyright Feature Creep 2008
Incidents
90% of severity C defects raised against new features are closed 80% of severity D defects raised against new features are closed
Severity New Feature Open Defects
Expected Fix Count
Actual Fix Count
% Achieved
C 967 870 825 85
D 772 610 543 70
All Priority 1, 2 and 3 defects (as assigned by the Defect Council)
have been cleared
Greater number of pre-existing incidents closed than expected
Copyright Feature Creep 2008
Incidents FER or RAT
No incidents in a status of Further Evidence Required No incidents in a status of Awaiting Test
Copyright Feature Creep 2008
Defect Cumulative Count
Evidence that defect cumulative count is levelling off
Copyright Feature Creep 2008
Other Areas Installations, Upgrades and Backward Compatibility. Successful execution of Test Cycle on a New Installation. Successful execution of Upgrade Testing for a defined set of
supported releases. Successful execution of Backward Compatibility Testing for a
defined set of supported releases.
Product Performance Successful execution of Performance Type Testing. No Degradation in performance that can not be attributed to the
introduction of new functionality. Stress Testing.
Copyright Feature Creep 2008
Areas of Concern Identify any areas where quality is suspect and/or further testing is required
Copyright Feature Creep 2008
Summary
General Statement – Test ManagerThe X.Y Test Cycle has been completed and the product is deemed fit for release to customers.
The X.Y Test Cycle has achieved a higher level of testing than has been attained in previous releases, and by consequence is assumed to be of higher quality.
All defects which have been Suspended or De-scoped from the release have been with the agreement of the defect council members and are deemed lower risk.
Copyright Feature Creep 2008
Agenda
Refresh on Feature Creep Delivery Framework
Product Management status
Product Quality
Marketing Status
Support & Maintenance Status
Services Status
Sales Status
Recommendation
Approvals
Copyright Feature Creep 2008
Marketing Status
Update on pricing/packaging changesInventory of materiels and availabilityUpdate on sales collateralSales, customer and partner engagement/training plansAnything else relevant e.g. Web site updates, press releases etc
Copyright Feature Creep 2008
Agenda
Refresh on Feature Creep Delivery Framework
Product Management status
Product Quality
Marketing Status
Support & Maintenance Status
Services Status
Sales Status
Recommendation
Approvals
Copyright Feature Creep 2008
Support & Maintenance Status
Release mechanisms e.g. CDs, download etc?Readiness of support teams?Readiness of Maintenance teams?Timing of first service pack?
Copyright Feature Creep 2008
Agenda
Refresh on Feature Creep Delivery Framework
Product Management status
Product Quality
Marketing Status
Support & Maintenance Status
Services Status
Sales Status
Recommendation
Approvals
Copyright Feature Creep 2008
Services Status
Readiness of Professional Services?Readiness of training team (inc Training materiels)?
Copyright Feature Creep 2008
Agenda
Refresh on Feature Creep Delivery Framework
Product Management status
Product Quality
Marketing Status
Support & Maintenance Status
Services Status
Sales Status
Recommendation
Approvals
Copyright Feature Creep 2008
Sales Status
Update from each region on X.Y sales forecasts – what opportunities, dependencies etc. exist
Copyright Feature Creep 2008
Agenda
Refresh on MDF
Product Management status
Product Quality
Marketing Status
Support & Maintenance Status
Services Status
Sales Status
Recommendation
Approvals
Copyright Feature Creep 2008
Recommendation• Make release as planned• Allocate X resources to support trial at Customer B
• Release on Date• Cost to achieve Release: £XXX
Copyright Feature Creep 2008
Major Risks and Planned Response
Area Nature of Risk Probability Impact Planned Response (Owner)
Technical Install process is lengthy and complex on Vista
Low Medium Assign additional resources to Vista trials
Schedule/ Resource
4 customer trials in parallel is demanding and may overwhelm support and product management
Med Med Prepare 2 of Services team to provide additional support
Commercial Customer Y expected feature Z to be free but we’re treating as upsell
High Med Be prepared to discount, or trade additional licenses to keep them happy.
High/High precludes “Go” decision
Copyright Feature Creep 2008
Agenda
Refresh on Feature Creep Delivery Framework
Product Management status
Product Quality
Marketing Status
Support & Maintenance Status
Services Status
Sales Status
Recommendation
Approvals
Copyright Feature Creep 2008
ApprovalsDept Approval: Go/No Go
EMEA
NAM
CALA
APAC
CEO
CTO
Finance
Marketing
Support Maintenance & Test
Research
Development
Product Management
Copyright Feature Creep 2008
Questions?