Upload
others
View
8
Download
0
Embed Size (px)
Citation preview
AI and Machine Learning for Testers
Jason Arbon, CEO @Appdiff
Ai for Test Automation
2
Relevant Context
Testing Neural Net Ranker
Personalized Web Search and Chrome Test Automation
AI for Mobile Test Automation
Ai for Test Automation
Agenda
AI For Testing
Testing AI
Future
Appdiff Presentation4
The Real Data Scientists
Joanne TsengData Scientist @appdiff.com
Appdiff mission:
Transform app
development with
automation & insightsFrancis Iannacci
Lead Data Scientist @appdiff.com
Testing is much like AI5
Testing: Apply Inputs, checking outputs.
https://www.extremetech.com/extreme/215170-artificial-neural-networks-are-changing-the-world-what-are-they
6
AI: Apply Inputs, check outputs
AI for App Testing
Features
Complexity increases
exponentially as new features and
states interact with existing
featuresTests
Test coverage grows linearly
because tests can only be added
one at a time
Time
Co
mp
lexi
ty /
Co
vera
ge
COVERAGE GAP
7
Testing Needs AI
Our Story8
Definition
When will AI Start Testing?
Our Story10
AI for Testing
What We Do11
ML : Subjective or complex labels via Humans
Our Story12
AI for Testing: Input
Reduce input space
150 actions per page
35 steps -> 150^35 = 10^78 paths
#atoms in universe10^78...
Our Story13
AI for Testing
Our Story15
AI for Testing
Our Story16
AI Driven Test Flows
Our Story17
AI Driven Test Flows
Our Story18
AI Driven Test Flows
Our Story19
Coverage
Coverage
What We Do21
ML : App Changes Don’t Break Bot Testing!
Our Story22
AI bots can test almost any app. 10,000+ apps tested.
Our Story23
Testing the Conference App
Our Story24
Benchmarked Performance
portfolio
AI for Testing26
Current AI Coverage: ~2/3rds Existing Tests
Long Sequences of Dependant Actions and Verifications
Basic Tasks (Login, Search, Create Account, Add items to Card, etc.)
Specific Sequences of Events with Specific Input (search for ‘beanie babies’, etc.)
AI for Testing27
Automation Coverage: Bots soon 100%
AI for Testing28
Automation Coverage: Bots... soon after, 10X
Canonical Tests
Learning Tests Cross-app
Our Story29
Testing AI: Test Graph Generation and Traversal
What We Do30
Testing Flow with AI
What We Do31
Abstract Intent Language: AIT
Focus on Intent, not “how”
Human Readable
App-Independant
Our Story32
AI for Testing
Jason Arbon, [email protected]
Appendix
What We Do35
ML: Page Label Training
Our Story36
Appdiff Represents the Evolution of Software Quality
1
AD HOC TESTINGReactively test
2
MANUALTESTINGProactively test
3
TEST AUTOMATION
Automate repetition
4
AI-DRIVENAPPROACHAccelerate coverage
App
QA
App
Automation
QA
App