Upload
bernadette-cross
View
223
Download
2
Embed Size (px)
Citation preview
The Future of Testing
James A. WhittakerPrincipal ArchitectVisual Studio Team TestMicrosoft
blogs.msdn.com/james_whittaker
toerris
HUMAN
nothing
is
PERFECT
WEall
make
MISTAKES
ITcouldhave
happenedto
ANYONE
We have many expressions to capture the fact
that humans are fallible
Our creations and inventions echo this unnerving tendency
toFAIL
if there was an Olympics for FAILURE
Or a World Cup for that which sucks
surelySOFTWARE
wouldreign
supreme
It’s enough to make one wonder …
… what we’re doing wrong
Software … … stocks supermarket shelves… delivers electricity and water… stores our personal information… runs nuclear power plants… controls doomsday weapons…
Are these failures typical?
How many of you have seen software fail?
How many users share this same experience?
Any reason to think things will improve?
Let’s take a look at the future and find out
How will we test the applications of tomorrow?
Virtualization
Information
Visualization
Virtualization
Information
Visualization
Virtualization
This prediction is 20 years out, but the technology to achieve it is here today
Virtualization
• Users are effective testers• Let’s leverage their power– They have the environments we seek– They have the data we lack– They run the tests we don’t think of
• “Crowd-sourcing” is one answer I like• Virtualization is the next step
Virtualization
• The story today:– A user (or tester) finds a bug, developer can’t repro it– A tester wants to reuse test cases, but can’t get them to run
• Virtualization solves this– User creates a VM at the point of failure
• These VMs have a market value• Developers could have access to the sum total of all the
environments in which their application can run
– A tester creates VMs for every test suite • We now have reusable, fully transferable testing assets • An automated historical record of all testing on our app
• If we can store every video in the world, why not every test case/test data/user environment?– It’s not been possible before but it is now
Virtualization
• What does the virtualized testing world look like?– Think crowd-sourcing on steroids– Test VM’s have great value, businesses emerge owning
and managing these environments– Testers analyze an app, license the appropriate Test VM’s,
conduct the test and report results
• Companies no longer need QA departments• Why test a subset of possible usage when (in the
future) we can test them all?• Software is important, it deserves this kind of
coverage
Virtualization
Information
Visualization
Virtualization
Information
Visualization
Information
This prediction is 1-3 years out, people are working on it today
Information
Information
Information about me (Status, Health, etc.)
Quick Access to oft-used abilities.
Information about my target. Information about my
location in the world.
Information
• What is the information we need?– Architects see Visio diagrams and flow
charts– Program Managers see PowerPoint decks
and Word storyboards– Developers see Visual Studio– Testers see binaries and interfaces
• In this model, testers see only what the user sees … I object!
Information
What the user sees
What the tester sees
Information
• Testers need more information– The architecture, design, code … they
are all useful to testers in the right context
– I want to hover over a UI element and see code, data types, value ranges, previous bugs, test history …
– ‘Cheating’ is not wrong when you are trying to build a great product!
Virtualization
Information
Visualization
Virtualization
Information
Visualization
Visualization
This prediction is happening today among the best test teams, soon it will be pervasive
Visualization
What does software look like?
Visualization
• Well, there’s:– Input– Output– Data flow– Control flow– Modules– Dependencies– Environment variables– Files– Interfaces– Bugs
Visualization
Conclusion
• Software will continue to run our economy, our businesses and our lives
• Software will be a critical part of the solutions that will grow food, cure disease and produce sustainable energy
• Remove software from the equation and humankind’s problems get a lot harder
• It is imperative that we get software right!
Parting thoughts
• 20 years from now– Software testing will look fundamentally
different– The tools will be more powerful and
allow us to be more impactful– Software testers will contribute more
fundamentally to the development process
Parting thoughts
• 20 years from now–Will the quality of software be taken for
granted? Will users be genuinely surprised when it fails?
–Will researchers look back in wonder that there was ever even a need for dedicated bug finders?
–What hard problems will simply cease to be hard because the software industry of this time did what needed to be done?
James A. Whittaker, PhDBlog: blogs.msdn.com/james_whittakerEmail: [email protected]: msdn.microsoft.com/testercenter