Testing: Chapter 13

"You make it, I’ll break it"

04/19/23Page 2

Purpose:

• Does the product as a whole perform up to specifications? (validation?)

• Does it perform under normal, upper, and lower limits wrt the environment?

• Does it meet or exceed all claims (verification)?

Testing defined

• establishing confidence that a device does what it is supposed to do

• the process of operating a device with the intent of finding errors

• detecting specification errors and deviations from the specification

04/19/23Page 3

Testing continued…

• verifying that a system satisfies its specified requirements or identifying differences between expected and actual results

• the process of operating a device or component under specified conditions, observing or recording the results, and making an evaluation of some aspect of the system or component.

04/19/23Page 4

Testing …

• Is a positive act of destruction.• Is needed for good product

development• Is necessary for consumer

satisfaction…

04/19/23Page 5

04/19/23Page 6

Stressors: (depend on requirements…)Temperature Humidity

Altitude Shock

Vibration EMI

Cell phone, pagers Brownouts

ESD Spillage

Shipping Power Surges

04/19/23Page 7

Figure 13-1 Requirements checklist

04/19/23Page 8

Customer Misuse:

• Coffee spillage• Drop in toilet• Use as a seat• Cross-connecting• Reverse polarity• Dirty hands• Unplugging via jerk on cable

04/19/23Page 9

Testing requires that you define failure!

• Intended application?• Anticipated environment?• Lifetime?• Value vs. time

• Sunken ships make good coral reefs in time …

04/19/23Page 10

Types of testing:

• Event testing – cycling, time to failure• Environmental – operating & storage• Altitude• Threshold (door, elevator, bricks… ) • Vibration• ISTA shipping test• EMI (standards!, see next slide: )

04/19/23Page 11

EN 1000 series standards

• Radiated electric field immunity• Fast transients• Surge, line, magnetic• Dips, variations, V, F variation• On/off, ESU• Walkie-talkie, cell phone• ESU, etc.

Failure DefinitionFor each test and for each device, a failure must be defined. This definition depends on the intended application and the anticipated environment. What is considered a failure for one component or device may not be a failure for another. The test protocol should be as detailed as possible in defining the failure.

04/19/23Page 12

04/19/23Page 13

We want to test some power supplies to prove a MTBF goal of 50,000 hours of operation. How many units do we test and for how long, assuming one failure and 90% confidence level?

(sample size)(test time) = MTBF goal (7.779)/2(sample size)(test time) = 50,000 (7.779)/2

= 194,475 unit-hours

04/19/23Page 14

MTBF goal = (sample size)(test time) (2)/Χ2

α;2r+2

Determining Sample Size and Test Length

Confidence level = 1 ‑ α, where α is the risk factor and r = the number of failures

Using the 90% confidence level, α = 0.10 & no failures (r=0)

(Sample size)(test time) = (MTBF goal)( Χ2 α;2 )/2

(sample size)(test time) = MTBF goal (4.61)/2

to prove a MTBF goal of 50,000 hours of operation:

(sample size)(test time) = 50,000 (4.61)/2 = 115,250 unit-hours

Example---------------------------

Types of Testing: Verification

Procedures that attempt to determine that the product of each phase of the development process is an implementation of a previous phase, i.e., it satisfies it. Each verification activity is a phase of the testing life cycle…

04/19/23Page 15

Types of Testing: Validation

• Validation is the process of evaluating a system or component during or at the end of the development process to determine whether it satisfies specified requirements

Types of Testing: Black Box

Black box testing is a data driven testing scheme. The tester views the device or program as a black box, i.e., the tester is not concerned about the internal behavior and structure. The tester is only interested in finding circumstances in which the device or program does not behave according to its specification

04/19/23Page 17

Types of Testing: White Box

White box testing is concerned with the degree to which test cases exercise or cover the structure of the device or program. The ultimate while box test is an exhaustive path test.

04/19/23Page 18

Types of testing: Software

• Module …• Integration …• System …• Acceptance …

04/19/23Page 19

Types of testing:Fuctional

This type of testing verifies that given all the expected inputs then all of the expected outputs are produced. This type of testing is termed success oriented testing because the tests are expected to produce successful results.

04/19/23Page 20

Types of testing - continued

• Robustness Testing• Stress Testing• Safety testing• Regression Testing

04/19/23Page 21

04/19/23Page 22

Types of testing, continued

• Life test/reliability• Customer misuse• Fluid spillage• Weight test• Keyed connections• Time related• Failure related (MTBF)

Questions?

04/19/23Page 23