Upload
ghodhbane-mohamed-amine
View
47
Download
0
Tags:
Embed Size (px)
Citation preview
Jakob Engblom, Product Line Manager, System Simulation, Wind River
Graham Morphew, Sr. Director of Product Management, System Simulation, Wind River
Need To Automate Test And Integration Beyond
Current Limits? Use Simulation
Moderator:
Brandon Lewis, OpenSystems Media
Speakers:
3 © 2015 Wind River. All Rights Reserved.
Need To Automate Test and Integration Beyond Current Limits? Use Simulation
3
Maximize Velocity – Webinar Series
Agile, Continuous Practices and DevOps
Graham Morphew & Jakob Engblom
Wind River
4 © 2015 Wind River. All Rights Reserved.
Embedded Software Trends
Software driving majority of value and innovation
IoT driving future differentiation, value & service
Customer expectations have changed to demand new features more often and faster
Pushing software organizations to change and be more agile and efficient
ChangeChange
Change
5 © 2015 Wind River. All Rights Reserved.
Faster Delivery Of Software To Customers
Agile Continuous Practices Cross Disciplined Teams DevOps
New development practices and organization styles getting established for embedded software development
Key Challenges:
• Access
• Collaboration
• Automation
Automation is key because manual processes don’t scale.
Covered in previous webinar
Covered in upcoming webinar
6 © 2015 Wind River. All Rights Reserved.
Developer changes or adds code
Build and test: single unit
Build and test: multiple units
Build and test: entire system
Code OK
Immediate
Coffee break or lunch
Daily
Continuous Practices
Quick feedback to developers
Find different classes of errors at each level
Avoid big-bang integration & associated problems
Avoid regressions by rerunning old tests
= higher quality and shorter lead times
(Typical time frames)
Needs to be Automated!
7 © 2015 Wind River. All Rights Reserved.
The Hardware Test Lab
bus, network, …
Data generator or world model, running on hardware
Target provisioning and control
JTAG, serial, Ethernet, flash programmer, …
System under test
Test manager Build server
Lab test rig – critical resource
Automation based on coordinating
several separate pieces of hardware
8 © 2015 Wind River. All Rights Reserved.
The Challenge of Automation in Hardware
Hardware test lab is difficult to build, scale and maintain
How many lab setups can you afford?
Reconfiguration is manual, even when tests are automatic
Automation is often brittle
Impact• Large equipment expenses
• Unable to scale
• Automation of failure
9 © 2015 Wind River. All Rights Reserved.
Simulation system – all just software
The Simulation Test Lab
Simulated bus, network, …
Data generator or world model, running as software only
Target provisioning and control
System under test - simulated
Test manager Build server
Simulator tools
Model of the world
Use the existing test management system
Automation based on software control, not hardware control!
10 © 2015 Wind River. All Rights Reserved.
Simulation Makes Automation a Reality
Automate any operation on the target system
Test in the context of any system or subsystem
Parallelize, scale & reconfigure test suites in any way
Use any server to run any test – flexibility*
Send the context of test failure to developers to solve*
Value using simulation for automation• Make embedded test automation possible & cost effective
• Do more, effective testing and get higher quality releases
• Deploy faster and more frequently
11 © 2015 Wind River. All Rights Reserved.
Parallelization and Performance
Simulation provides automation and access, which enables more tests to be run in parallel
Test 1 Test 2 Test 4Test 3 Test 5 simulation
Test 1
Test 2
Test 4
Test 3
Test 5
Overnight tests take a few minutes
Weekly tests become daily tests
Test more often = less chance for bugs to creep back into the software – higher quality
1/5 the time
12 © 2015 Wind River. All Rights Reserved.
Test management server
Test run system (such as Jenkins)
Test generation and management
Test server 1
Simics
board
Networkboard
board
Testing agent/slave node
Parallel Testing and Automation Setup
Physics/Electronics/World
Test server n
Simics
board
Network
board
board
Testing agent/slave node
Physics/Electronics/World
board
Test system launches simulated test labs
Each test can be of a different configuration
Sends out test cases & collects results
13 © 2015 Wind River. All Rights Reserved.
Simulation
Expanding Test Coverage: Variation
Simulation
Board A
Simulation
Board A
Network
Board B
Network
Board A
Board A
Board B
Board C
Simulation
Board B
Program 1
Program 2
Program 3
Program 4
generates
generates
generates
System setups are just software
Change setup program to get a new configuration
Infinite supply of each type of system
Randomly generate setups for “didn’t think of that” tests
System creation & setup is automatable
Input file
World / Physics Simulation
14 © 2015 Wind River. All Rights Reserved.
Expanding Test Coverage: FaultsFault injection is a natural match for simulation
Test fault detection, handling, recovery, reporting, …
POST, diagnostic software, firmware, applications, system, …
For example:– Corrupt and drop network packets
– Feed bad data from the environment
– Crash processors
– Change memory contents
– Modify system configuration on the fly
Simulation makes faults:– Repeatable
– Non-destructive
– Scriptable and automatable!
15 © 2015 Wind River. All Rights Reserved.
Automatically capture test failures
Analyze using record and replay of tests
Run and debug in reverse
When Things go Wrong
More on Collaboration with simulation in our next webinar.
16 © 2015 Wind River. All Rights Reserved.
Key Take-Aways Continuous integration and testing requires automation
Automation is made easier by using simulation
Simulation provides the ability to automate any test
Simulation allows testing and automation of what cannot be tested with hardware
Audience Q & A
Jakob Engblom,
Product Line Manager,
System Simulation,
Wind River
Graham Morphew,
Sr. Director of Product Management,
System Simulation,
Wind River
Thanks for joining us
Event archive available at:
http://ecast.opensystemsmedia.com/
E-mail us at: [email protected]