Embed Size (px)
Citation preview
T3 Test Automation 2019-‐05-‐02 09:45
Well, That's Random: Automated Fuzzy Browser Clicking
Presented by:
Paul Grizzaffi Magenic
‘
Brought to you by:
888-‐-‐-‐268-‐-‐-‐8770 ·∙·∙ 904-‐-‐-‐278-‐-‐-‐0524 -‐ [email protected] -‐ http://www.stareast.techwell.com/
Paul Grizzaffi As a Principal Automation Architect at Magenic, Paul Grizzaffi is following his passion of providing technology solutions to testing and QA organizations, including automation assessments, implementations, and through activities benefiting the broader testing community. An accomplished keynote speaker and writer, Paul has spoken at both local and national conferences and meetings. He is an advisor to Software Test Professionals and STPCon, as well as a member of the Industry Advisory Board of the Advanced Research Center for Software Testing and Quality Assurance (STQA) at UT Dallas where he is a frequent guest lecturer. Paul enjoys sharing his experiences and learning from other testing professionals; his mostly cogent thoughts can be read on his blog.
4/23/19
1
@pgrizzaffi #STAREast
Well THAT’S Random
@pgrizzaffi #STAREast
» Paul Grizzaffi » Principal Automation Architect at Magenic » Career focused on automation » “Software Pediatrician” » Advisor
Who Is This Guy?
4/23/19
2
@pgrizzaffi #STAREast
Where Is This Guy?
http://www.linkedin.com/in/paulgrizzaffi
@pgrizzaffi
http://responsibleautomation.wordpress.com
@pgrizzaffi #STAREast
Once Upon A Time
4/23/19
3
@pgrizzaffi #STAREast
Regression Testing
Smoke Testing
Acceptance Testing
Exploratory Testing
Ad Hoc Testing
What Do We Do Today?
@pgrizzaffi #STAREast
Pesticide Paradox
4/23/19
4
@pgrizzaffi #STAREast
What Do We Miss Today?
@pgrizzaffi #STAREast
High-Volume Automated Testing (HiVAT)
4/23/19
5
@pgrizzaffi #STAREast
HiVAT
“…a family of testing techniques that enable the tester to create, run and evaluate the results of arbitrarily many tests” *
* http://context-driven-testing.com/?p=69
@pgrizzaffi #STAREast
Exploit Existing Tests or Tools
Long-sequence regression testing
Load-enhanced functional testing
Exploit Available Oracles
Functional equivalence testing
Constraint checks
Focused On Input
High-Volume Parametric Variation
Fuzzing
Areas of HiVAT
4/23/19
6
@pgrizzaffi #STAREast
Testing without an
oracle
Know what shouldn’t happen
Vary inputs
Textboxes
Fuzzing
@pgrizzaffi #STAREast
Randomly Vary Input
4/23/19
7
@pgrizzaffi #STAREast
Breadcrumbs Inputs
Screenshots
HTML
Message Dumps
Reproducibility
@pgrizzaffi #STAREast
Why should textboxes have all the fun? |
4/23/19
8
@pgrizzaffi #STAREast
Random Clicker Clicks random clickables
Looks for “weirdnesses”
Logs steps
Saves HTML and screenshots
@pgrizzaffi #STAREast
Why?
Valid but unintuitive
Cheap to Build
It works
4/23/19
9
@pgrizzaffi #STAREast
“Design”
@pgrizzaffi #STAREast
Implementation/Algorithm while (keepGoing) {
clickables = itemFinder.FindAllClickables(currentPage); chosenClickable = itemChooser.ChooseRandomItem(clickables); infoLogger.LogInterestingInfo(chosenClickable); pageAfterClick = chosenClickable.Click();
infoLogger.LogInterestingInfo(pageAfterClick); if (weirdnessDetector.IsPageWeird(pageAfterClick)) { infoLogger.LogWeirdnessInfo(pageAfterClick); currentPage = START_PAGE; }
else { currentPage = pageAfterClick; } keepGoing = ShouldWeKeepGoing();
}
4/23/19
10
@pgrizzaffi #STAREast
“Scud”
Random Link Clicker
First bug in four weeks
Random Menu Clicker
Four issues in first week
Out In The Wild
@pgrizzaffi #STAREast
But Wait…Sounds Like You Have An Oracle To Me
OK
Fine
Ya Got Me
Providing Value
4/23/19
11
@pgrizzaffi #STAREast
Stay in domain
Too many clicks on same page
Restart algorithm/tool
Logins
Stopping criteria
Additional Algorithm Considerations
Photo: http://www.wocintechchat.com
@pgrizzaffi #STAREast
Where Does This Fit In?
Ad Hoc Testing
HiVAT &
Random
Acceptance Testing
Regression Testing
Smoke Testing
Exploratory Testing
4/23/19
12
@pgrizzaffi #STAREast
Other Input Types
Thick Clients
Mobile
URL Fuzzing
API Calls
Textbox Fuzzing
Possible Next Steps
@pgrizzaffi #STAREast
Why Bother With Textboxes?
Out of scope
Means to an end
Additional page visits
4/23/19
13
@pgrizzaffi #STAREast
Takeaways
Randomization
Considerations
Non-Conventional HiVAT?
@pgrizzaffi #STAREast
tnsoQusie
http://www.linkedin.com/in/paulgrizzaffi
@pgrizzaffi
http://responsibleautomation.wordpress.com
