http://cheyat.com/qa/hp-alm-online-training-tutorials
HP ALM Online Training
ALM Work Flow – Requirement through Deployment
2
Predictability
Heightened repeatability
Improved quality
Ready accommodation of change
Why Testing
3
Why Test management
4
How do I know my release readiness
How do I manage my requirements
How do I track my test coverage
How do I monitor my Release progress
How do I assess the severity and priority of my defects
How do I base line my test artifacts
5
Test management is the practice of organizing and controlling the process and artifacts required for the testing effort.
• Pen and paper• Word processors• Spreadsheets • Sharing in Drives (J:/)• Emails
Traditional tools used for test management
include:
Why Test Management Tools
6
• Start test management activities early • Test iteratively • Reuse test artifacts • Utilize requirements-based testing • Leverage remote testing resources • End to End Traceability of Release requirements• Defining and enforcing a flexible testing process • Coordinate and integrate with the rest of development • Communicate status • Focus on goals and results • Automate to save time
Test Management Recommendations
Benefits
7
88
ALM Stakeholders
Project Managers Business Analysts
Development Team Testing Team
• Dashboard and Analysis View• Release• Requirement
• Dashboard and Analysis View• Release• Requirement
• Dashboard and Analysis View• Release• Requirement• Testing• Defect
• Dashboard• Defect
• Track release progress• Visibility into key project milestones• Full trending analysis and insight into application
projects• Clearly communicate to all stakeholders
Define and track multiple requirement typesBi-directional traceability from requirements to requirements, tests and defectsLeverage existing assets in MS Word
Create test cases to adequately test the requirements
Manage and control execution of manual and automated tests
Create defects from manually or directly from the execution of manual and automated tests
Improve developer efficiency
Consistent defect management process across all projects and initiatives
Integrated into developers IDE
Domain Projects
AFG_Projects_ All
AFG_ RFC_ All
AFG_Bug Fix &Routine_ All
<LOB>_<Project ID>_<Project
Name>
Root Folder
Project Name
Sub Folder
Req 1
Req N
Root Folder
Project A
Test Case
Test case 1
Test case N
Defects
Defects
Reporting
Reporting
<LOB>_<RFCs>_<Year>
RFC_LOB_2015 RFC 1
RFC N
RFC_LOB_2015Test case 1
Test case N
Regression Test case 1
Regression Test case N
<LOB>_<Bug Fix&
Routine>_<Year>
Bug Fix_LOB_015
Bug Fix_LOB_015
Test case 1
Test case N
Regression Test case 1
Regression Test case N
HP ALM Login Entities Requirements Module Test Plan/Test Lab module Defects module Reporting module
Defects
Defects
Reporting
Reporting
Jan
DecRFC 1
RFC N
Bug Fix 1
Bug Fix NJan
DecBug Fix 1
Bug Fix N
Testing Process – An Overview (SOLMAN – HP ALM)
SOLMAN HP ALM 12SAP TAO
Component based Automation
HP SprinterManual Testing
HP UFTTest Automation Tool
Requirements
Defects to process in SOLMAN
Business Blueprints Test Planning Test Execution
• Business Processes• Test Objects
• Test Requirements
• Test Cases• Test sets to run• Test Results
Test Results to SOLMAN
• Defects• Defects Module
Manual Feed or Import from Excel
Req/Test cases
Automation Testing
Testing Process – Details (SOLMAN – HP ALM)
Requirements Module1. The Requirements can be loaded to HP ALM in any of the following three ways
• Import are Export the business blueprint and test objects from SOLMAN ‘SOLAR02’Note: Only the Business Processes/scenarios that has test object or transactions assigned to it can be export or import to HP ALM
• Capture the functional requirements in the defined excel spreadsheet and import the same to HP ALM• Create the required folder tree and enter requirements into that based on the functional requirements document
2. Once the Requirements are imported to the HP ALM then we can update/delete the requirement on accessing to the respective requirement
Test Plan Module1. The test cases can be loaded to HP ALM in any of the following two ways
• Directly enter the test cases and define the test steps in the Test plan module• Write the test cases in the defined excel spreadsheet and import the same to HP ALM test plan module
2. Each test case will be mapped to the corresponding requirements by editing the test case
Test Lab Module3. The corresponding test cases need to be mapped pull to the Test lab module as a Test set for the test execution 4. These test case need to be executed either Manually or Automated run for the automated cases.5. The test results must be transferred to SOLMAN – SOLAR01/SOLAR02
Testing Process – Details (SOLMAN – HP ALM) contd.
Defects Module1. During the test execution the defects can be logged when the testing scenario is fail to meet the expected result2. Defects falling under two types:
1. Defect: It is a regular defect that is been identified as to be processed in the HP ALM only by both Testing an Development team and it goes though the defect lifecycle process in ALM
2. SAP related Defect: It is a defect that is been identified as to be processed in the SOLMAN by the development team and it goes through the defect lifecycle process in SOLMAN (Defect Fix) and ALM (Defect logging, resetting and Closing )
• These can be viewed in the CRM_DNO_MONITOR or SM_WORKCENTER or CRM_ORDER transactions codes in SOLMAN
Testing Process – Details (SOLMAN – HP ALM) contd.
HP ALM SOLMAN
New
New
Fixed
Closed
Forwarded
Received from HPQC
Proposed Solution
Confirmed
Assign responsibility
to SAP
Defect Workflow
New
Clarification on Defect?
Valid Defect?
Deferred
Pending Rejected
Req For Info
Fixed
Retest
Retest Pass?
Successfully Tested Closed
Reopen
Assigned
Test LeadTest Lead
Test Lead
Func/Dev
Test Lead
Tester
Tester
Func/Dev
Test lead
Decision to Reject?
Rejected
Test Lead
Decision to Defer?
Test Lead
Test Lead
NO
NO
YES
NO
NO
YES
YES
YESNO
NO
Tester
Test Lead Test Lead
HP ALM Implementation Case Study
Client is one of the largest company that offers a comprehensive array of financial services to retail and institutional clients, which includes annuities, retirement plans, life insurance, mutual funds, managed accounts, alternative investments, direct banking, institutional investment management, employee benefits, and financial planning.
Client Profile
Client Requirements
The client was looking to implement HP ALM to address the following: Manage and create traceability between
requirements, tests artifacts and defects Standardize process across application lifecycle Facilitate collaboration and communication
between multiple, distributed teams Manage and execute automated testing Make informed go/no go decisions Provide a scalable architecture
Project Background
Cognizant focused on deploying integrated Quality management across the
application lifecycle to focus and prioritize testing resources at all phases and
find and manage defects earlier in the lifecycle when they are less costly to fix.
By taking a systematic, risk-based, managed approach to quality throughout
the lifecycle, the result would lead to a fewer issues in production and faster
time to delivery.
The implementation allowed the Clients to prioritize testing based on risk. And
with quality metrics and centralized reporting, they were able to make informed
decisions about application releases.
Cognizant Solution Approach
The Client wanted to assure quality with systematic automation testing
conducted over the course of one year before rolling out. This massive project
carried with it significant risk, as errors and downtime could erode customer
trust or even lead to business losses.
Benefits achieved using HP ALM Reduced cost through centralized management and enforcement of consistent
workflows and processes
Increased efficiency by reducing duplication of effort and sharing best practices
Increased visibility to quality status, requirement coverage and defect trends in
aggregate across projects and initiatives
Achieved significant savings by tracking key metrics against project milestones within
Project Planning and Tracking of HP Application Management
Business Benefits achieved
The Client was able to minimize business risk in
this important project.
A quality assurance process has been created for
building next generation systems.
Quantitative measures of IT quality and
performance have been established to support
increased business efficiency
Top New Features in ALMBenefits gain from ALM
Project Planning &
Tracking
Requirements
Templates
Project/Template
Reports
Traceability Matrix
Embedded Web
Scorecards &
Graphs
Business Process
Modeling
Integration
Test Configurations
Reduced cost through centralized management and enforcement of consistent workflows and
processes
Reduced cost and increased efficiency by reducing duplication of effort and sharing best
practices
Increased visibility to quality status, requirement coverage and defect trends in aggregate across
projects and initiatives
Others
HP ALM Implementation Case Study
Category Description
Defect When a defect is logged it’s categorized as “Defect” by default. It states that the defect is considered to be analyzed and fixed for the current release
Issue When a defect is identified and after reviewing ,it has few dependencies to be verified like test data, test steps to be carried etc. hence the defect will be categorized as Issue
New Req When a defect is logged in which the test carried to related to that defect is not mention in the requirement. Then the defect is considered as a “New Req” and it goes through the Change request process as AFG we following today.
Defect Categories
Status Responsibility Target Owner Description
New Tester Test Lead Tester/Business tester creates a defect during testing; the initial status of the defect will be in “New” status. The New status defect will be assigned to Test Lea d for the Defect triage or analysis.
Pending Rejected Test Lead Test Lead During defect meeting when the defect analysis results in to that the defect is not a valid one then test lead changes the defect to “Pending Reject” status for the further discussion with the Functional team or Business team.
Rejected Test Lead Test Lead
Test lead will change the status of the defect to “Rejected” in following conditions: Defect analysis results in to the decision that the system works as expected and it’s an invalid defect Tester's misunderstanding of requirements or test scripts. Tester mistakenly logged a duplicate defect that was already raised, in case of duplicate defect; the new one is marked as “Rejected” and
if any useful information from the new defect is copied over to the old defect.
Deferred Test Lead Test Lead
Test Lead will ‘Defer’ a defect when it is analyzed and • Not In-scope on functionality • Not in current scope but in future releases.
When these defects are need to be picked up and assign in future release then the test lead can change the defect status to “Assigned”
Assigned Test Lead Functional/ DEV team When a New defect is created by tester and it is considered as a valid defect, then the Test lead will assign to the concern team
(developer/Functional SME) and then the status will be changed to “Assigned”. New defects will be picked first during the defect triage to decide the assignment.
Req. for info Functional/DEVteam Test Lead
Once the Defect is assigned to the Functional SME or the developer then they will look the defect and if any clarification required on the defect then the clarification can be mentioned in defect comments and assign back to the tester/test lead as "Req. for info".
Test Lead will consult tester and provide the necessary details for the defect and assign back to the same Functional or Developer
Fixed Functional/ DEV team Test Lead After the defect is fixed, the status is changed to “Fixed” by Functional SME/Developer and assigned to Test lead for the verification to proceed
further for the retest
Retest Test Lead Tester Once the test lead confirms that fixes are ready to test then changes the defect status to “Retest” and assigned to the tester who is responsible for retest.
Reopen Tester Functional/ DEV team When the tester retest the defect and confirms that the fix does not resolve the defect completely, then changes the status to “Reopen” and assign back the functional or development team.
Successfully Tested Tester Test Lead When the tester retest the defect and confirms that the fix has worked fine, then defect the changes status to “Successfully Tested” which will assign to test lead
Closed Test Lead NA Test lead will verify the Successfully tested defects and changes the status to “Closed”Any new issues identified during retest should be created as a new defect rather than reopening the current defect.
Defect Statuses & Descriptions
Priority Level Business Impact Testing Impact
P1 This has severe impact to the customer and continuing to operate may not be possible. This must be fixed immediately.
Defect is a blocking issue. The defect would impact the completion of testing iteration if not resolved, and any testing cannot proceed until this defect is resolved
P2This is major impact to the customer but does not halt operations. This must be fixed as soon as possible and before the release of the current version in development.
Defect is severe. This impacts the general behavior of the application. A workaround may exist but its use is unsatisfactory. Test cases will be blocked due to this defect.
P3This has major impact to the customer. The problem should be addressed and fixed with the current version release if time permits, or a patch to the release issued, if possible.
The failure impacts non-critical aspects of the system, behavior of functionality under specific conditions. During testing this will result in test case failure, but will not block the execution of the other test cases.
P4 This has minor impact to the customer. The flaw is usually cosmetic to the customer in nature and only needs to be fixed when there is sufficient time.
Cosmetic problem. During testing this will result in test case failure, but will not block the execution of the other test cases.
Defect Priority levelsPriority is an outcome of subjective evaluation of how important an issue to the business. Other tasks in the queue and schedule are factors that may impact the decision of prioritizing the Defect. It is relative, it shifts, and it is always a business decision.
• Initial value for priority field will be set by the tester who creates the defects based on testing impact. During defect triage meeting it business re-evaluates and updates the value, if needed.
• DEV/business will participate in the defect triage meeting during the Testing (Functional/SIT/Regression) done by QA and will assign / change the priority to new defects based on business impact.
Reports and Graphs
UT SIT UAT0
20
40
60
80
100
120
23 215
128
10
21
10
18
823
6
10 12
21
1021
8
10
10
10
Defect Summary Report <as on date>
RejectedDeferredClosedRetestFixedAssignedNew
UT SIT UAT0
2
4
6
8
10
12
14
16
18
20
24
2
2
4
42
46
2
4 4
2
4 4
Test Summary Report <as on date>
No RunBlockedNot CompletedFailPass