Upload
sadams22
View
939
Download
0
Embed Size (px)
DESCRIPTION
How to develop a successful Selenium Automation Program
Citation preview
How to develop a successful Selenium
Automation Program
Dave Rader - CTO
Sreekanth Singaraju – VP of Testing Services
DATE: 6/28/2012
2
About Alliance
Alliance Global Services is a software development firm that partners with software, technology and information-intensive businesses on their mission-critical work. We plan, design, architect, build, test and implement software applications, platforms and products that become a primary driver of innovation and revenue growth for our clients’ businesses. We are recognized for driving quality and speed-to-market when business success depends on the software inside.
We operate agile development centers in the United States and Hyderabad, India and have over 650 employees
We have partnered with more than 400 innovative clients in the Information Services, Software and Technology, Insurance and Healthcare industries
3
Agenda
• Definition of success
• Impediments in Achieving Best in Class
Automation
• Challenges & Solutions for Selenium Automation
• Key Takeaways
• Titanium xAFT Overview
4
Definition of Success for an Automation Program
Effective
Maintainable
Robust
Re-Usable
Test Independ
ence
ROI
Identifies defects of significance
and adopted by Developers &
Testers
The framework should be of low
maintenance and should not take
significant effort to handle routinue
changes to AUT
Can be leveraged on different
releases, configurations &
environments.
Automated test execution produces
consistent and provable results.
Ability to run Automated test scripts
cases in any order.
Delivers measurable value in
terms of decreased cycles of
execution, decreased effort and
improved quality
5
Impediments to Achieving Best in Class Automation
Objectives Non-existent or Unrealistic automation objectives
Focused around technical goals rather than business goals
Automation Strategy
Highest automation ROI areas not identified
Subscale initiatives or insufficient commitment
Start-Up Costs
Tool licenses and framework creation costs
Up front Labor cost for implementing test cases
Expertise & Assets
Manual test teams lack development expertise
Knowledge of test case lifecycle mgmt. missing
Measuring Effectiveness
Lack of a metrics program to measure alignment with overall objectives
Evaluate effectiveness of testing programs and fine-tune strategy
6
Challenges with Selenium Automation
Object References
Ability to identify Objects and manage them to limit the expense and effort to maintain updates to UI
Coding Standards
Develop scripts in a consistent and maintainable format for easy maintenance and effective execution
Reporting Provide commercial quality reporting that enables testers to rapidly identify risks inherent in software and provide information to developers towards remediation
Framework Implement a framework and process that will streamline management of common elements and resources in automation development
Expertise & Skillset
The ability to develop and maintain effectively automation and related assets to deliver ROI on automation investments
7
UI Object Reference
//html/body/div/div/div[2]/ui/div/div[2]/div/h2/a
• Object reference created using
XPATH, ID, Name, Class or CSS.
Common mistakes in constructing
Object references
• Improper selection of Object
selection property type
• Duplicate references to same objects
• Full length XPATH from root used for
object identification
• Leading to brittle references that
break when minor changes to UI are
implemented
• Significant maintenance effort when
UI is redesigned
Most of the Commercial automation tools have the provision to create and manage UI objects in a centralized repository. Selenium has limited tools to address this leading to high-maintenance scripts
8
UI Object Reference – Recommendations
• Utilize common stored location
for objects using logical names
• Use Key-Value-Pair combination
for Logical name and Object
identification
• Logical hierarchies to group
objects for same features at the
same place
• Recommend UI developers use
frameworks that provide unique
identifiers for controls
• Use the smart identification
mechanism and use the
optimized XPATH
//div[@id=’icon-edit’/h2/a
9
Coding Practices
selenium.type("name=j_username",
"user1");
selenium.type("name=j_password",
"user123);
selenium.click("\\div\\div[1]\span\login
");
Thread.sleep(30000);
selenium.click("class=Compose_Email");
• Not using a design pattern or
development strategies to
develop framework and scripts
• No exception catching block in
the coding
• Use of Objects reference such as
path/xPATH in the code
• Use of hard coded values for the
data for inputs and validation
Test engineers that perform script development on Selenium may not have formal development training leading to common coding errors. These lead to expensive maintenance when the scripts need to be updated or unexplained behavior for execution
10
Coding Practices – Recommendations
• Using a design pattern and
development strategies like Page
Object Model OR Domain Driven
Design
• Utilize code style/quality checking
tools to address:
• Exception handling
• Commenting
• Hard coding
• Use reusable coding practices for
handling input data, validations,
object references and common
library functions
• Utilize reference based object
synchronization for script to
synchronize with the app
try {
//Login to the system and Verify Email
selenium.type(userNameLocator,
userNamevalue);
selenium.type(passwordLocator,passwordvalu
e);
selenium.click(loginLocator);
selenium.isTextPresent(welcomeText);
} catch (Exception e) {
Logger.Debug(("Element :["+ welcomeText+
"not found" +
"continuing text execution"));
selenium.click("Compose_Email");
}
11
Reporting & Metrics
• Poring over logs to identify
exceptions is time consuming
and error prone
• Execution metrics such as
coverage, script effectiveness etc
help fine-tune framework
• Test Execution meta data such
as environments, browsers,
pass/fail etc need to be extracted
from the basic logs
• Mapping of pass/fail data to
validation points needs to be
created
• Fatures such as screenshots are
completely missing
Most of the Commercial automation tools provide extensive reporting functionality is not available in Selenium and needs to be developed
12
Reporting & Metrics
• Multiple levels of logging for
easy debugging and analysis
• Drill down Test Result
reports
• Automatic Emailing the Test
Result Reports
• Test Result Report
Integration with Test Case
Management Tools
• Screenshot capture on every
step or on exception
• Defect identification
effectiveness, designed
coverage and traceability to
features
13
Automation Framework
Automation Framework: Target
Browsers, Test Data, Object References,
Reusable Functions, Exception Handling,
Reporting
An example of a poorly designed
framework:
• Poor architecture design limiting
flexibility of choosing target
browsers, target environments
and test data suites
• Lack of standards and guidelines
for reusable functions,
components and object references
• Reporting features limited to logs
from Selenium
• Weak governance leading to
diluted framework effectiveness
Poorly designed automation frameworks only transfer inefficiencies from scripts to the framework while providing limited benefits
14
Automation Framework
Application Specific Business
Functions
Configuration Files
Test Data
Object Repository
Generic Functions
Logging and Reporting
API Support
• Framework should be application-independent.
• Easy to expand, maintain, and perpetuate as the underlying application changes
• Generic functionality like navigation, link checking, login/logout and reports should be developed as reusable functions
• Business functionality will be addressed in test scripts/suites
• Test Data treated as a separate input into scripts
• Implement support for Windows File and Authentication dialogs
• Scripts should have minimal external dependencies from:
– Browser versions
– Target environments
15
Expertise & Skillset
Developers
Automation Engineer
• Facilitate the creation of
automation testers by
creating a cross
functional/technical
training
• Support them with
robust automation
frameworks and
reusable assets
• Ultimately Expensive to
hire, maintain and retain
resources
Currently there are a lot of good developers and good testers – but it is very hard to find good automation Engineers that can develop effective automation
Tester
16
Key Take Aways
Existing Automation Program
Run scripts on more than one browser
Run Sonar and identify code quality issues
Metrics and reports from framework
Validate scripts can run multiple data conditions with no code changes
Change IDs for a few UI controls and evaluate impact on scripts
Governance and process on managing framework
New Automation Program
Review off-the-shelf automation frameworks first before building from scratch
17
Titanium xAFT Overview
Titanium xAFT (eXcel Automated Functional Tester) is a robust enterprise automation framework that enables technical and non-technical users to manage and leverage automation suites
Across browsers and web services
Flexible & Maintainable
framework
Based on Open and proven
technologies
xAFT is a Selenium based Automation tool to test web based applications from an
Excel worksheet using English like language.
30% productivity improvements over Selenium automation
Execute scripts from Excel
Web Mobile WebServices
allianceglobalservices.com/innovation
18
Thank You