Test Patterns in Java - NetBeans...return java.util.ServiceLoader.load(DialogDisplayer.class).iterator.next();}} define abstract interfaces and “lookup” implementation: real implementation

2006 JavaOneSM Conference | Session BOF0220 |

Test Patterns in Java

Jaroslav Tulach, Jesse Glick, Miloš Kleint

Sun Microsystemshttp://www.netbeans.org

2006 JavaOneSM Conference | Session BOF0220 | 2

Automated Testing

Learn why to invest in automated testing and let us give you overview of the wildest test patterns that NetBeans project found useful.

Test is a Kind of Documentation


Agenda

What is the problem?Why do automated tests help?How to write tests?Questions and Answers


The Quality of an Application

http://openide.netbeans.org/tutorial/testpatterns.html

• quality = implementation ∆ specification • quality may not be the right term

• maybe term confidence is better • regressions cost too much • the "amoeba" model


The Amoeba Model



The Amoeba Model



Why write automated tests?● Make sure the code works● Things do not get wrong

● concurrent access ● garbage collection and finalizers

● Algorithms and data structures are sane● fast and small

● Mistakes do not reappear● race conditions● deadlocks


Do tests influence code?

● methods to verify state ● ways to prevent asynchronous behavior ● separation to units

● using lookup and discovery● code against interfaces not implementation

● Java is not single threaded● tests may need cooperation with code

http://www.netbeans.org/download/dev/javadoc/orgopenideutil/org/openide/util/Lookup.html


Test Patterns in Java

● Testing exception states● Hard to invent● Easy to copy and use● Premise: Test everything

2006 JavaOneSM Conference | Session BOF0220 | 1 0

Demo ApplicationExtraordinary Complex and Useless Calculator


Code Against Interfaces


public abstract class DialogDisplayer { public abstract Object notify(javax.swing.JOptionPane what); public static DialogDisplayer getDefault() { return Lookup.getDefault().lookup(DialogDisplayer.class); // or in Mustang return java.util.ServiceLoader.load(DialogDisplayer.class).iterator.next(); }}

● define abstract interfaces and “lookup” implementation:

● real implementation in the application● “mock object” in the test


Setup Testing Environment


public class MyTest extends org.netbeans.nbjunit.NbTestCase { protected void setUp() { org.netbeans.nbjunit.MockServices.setServices(MockDialogDisplayer.class); }}public class MockDialogDisplayer extends DialogDisplayer { public static JOptionPane lastPane; public static Object toReturn; public Object notify(JOptionPane p) { lastPane = p; return toReturn; }}

● fill lookup with own “mock” implementation● replace UI with headless displayer


Memory Leak Tests

● Memory management is important aspect of “amoeba”● easy to regress● hard to describe and verify

● NbTestCase.assertGC(String, Reference);● Cooperates with Insane library

NetBeans JUnit Extensions http://xtest.netbeans.org/servlets/ProjectDocumentList

class SetupAndGCTest extends NbTestCase { public void testTheyCanDisappear() { Calculator same = Calculator.create("powerful"); WeakReference ref = new WeakReference(same); same = null; assertGC("Caches can be GCed", ref); }}


Memory Leak Tests II● Controlling size of data structures

● hard to describe and verify● NbTestCase.assertSize(String, int, Object);

● virtual size of data structure trees● 8/16/16/24 bytes● Based on Insane library

● How big is an empty string?

NetBeans JUnit Extensions http://xtest.netbeans.org/servlets/ProjectDocumentList

Object dataStructure = “”; assertSize( "Less than 16bytes?", 16, dataStructure);

class SizeRegisterFailingTest extends NbTestCase: Calculator calc = Calculator.create("powerful"); Calculator.Register reg = calc.getRegisters().get(0); assertSize("Register contains one integer field => 16 bytes. Will it fit?", Collections.singleton(reg), 16,

new Object[] { calc }); // last array allows to exclude some objects


Speed Test● NbTestSuite.speedSuite and linearSpeedSuite● Size of the test● Comparing speed of various algorithms● Need to eliminate influence of GC and hotspot


protected void setUp () { calc.ensureRegistersSize(getTestNumber()); list = calc.getRegisters();}public void test10() { doTest (); }public void test100() { doTest (); }public void test1000() { doTest (); }public void test10000 () { doTest (); }private void doTest () { for (int i = 0; i < 10000; i++) { list.get(size / 2);}


Randomized Tests


Test for the unknown● Hard to test where “amoeba” does more than expected● Use random sequence of operations:

int op = random.nextInt();

switch (op) {

case 0: list.add(new Integer(random.nextInt()); break;

case 1: list.removeAt(random.nextInt(list.size())); break;

case ....

}

● Record the seed for case of failureseed = System.currentTimeMillis();

random = new Random(seed);


Deadlock Test I


● Deadlocks are biggest contributors of amoeba's shaking● no reasonable theory● hard to simulate from outside


Foreign Code In Critical Section

http://openide.netbeans.org/tutorial/apidesign.html

Deadlock prone code

● Calling foreign code under lock leads to deadlocks● Sometimes hard to prevent private HashSet allCreated = new HashSet(); public synchronized JLabel createLabel() { JLabel l = new JLabel(); allCreated.add(l); return l; }

● java.awt.Component grabs AWT tree lock● HashSet.add calls Object.equals


Deadlock Test II


● Regression test● Reading thread dumps

● Reproducing thread locks● Need more threads● Using time out for the test

● Block them in the right moment● overriding virtual methods● introducing artificial hooks


Race Condition Test


● Similar to deadlock tests● Missing thread dump information● Need more threads● Block them in the right moment

● overriding virtual methods● introducing artificial hooks

if (System.getProperty(“simulate.race.condition.2”) != null) {

Thread.sleep(500);

}


The Beauty of Logging


● Analyze random failuresprotected Level logLevel() { return Level.FINE; }

CharSequence msgs = Log.enable(“logname”, Level.ALL);

● Natural in source codeLogger LOG = Logger.getLogger(“logname”);

LOG.finest(“simulate.race.condition.2”);

● Tests can plug their own Handlersclass TestHandler extends Handler {

public void publish(LogRecord r) {

if (r.getMessage().equals((“simulate.race.condition.2”))

Thread.sleep(500);

}

}


Flow Control using Logging


● Automatic breakpoints● Allows replay of captured log files

● plugs in own handler● blocks and wake the threads up

● Complexity hidden behind simple NetBeans JUnit API:

Log.controlFlow(

“THREAD: main MSG: simulate.race.*

“THREAD: second MSG: cause problems”

);


Summary

● Prevent Amoeba from shaking● from one side – e.g. what is not working

● Automate testing, its cheaper● Treat tests as a complement to documentation● Use our test patterns● Invent new test patterns


References

● BOF2559 – Thu 19:30 Gateway 105● Discovery and Dependency Injection Patterns in

Modular Architectures● TS6218 – Fri 14:30 Gateway 102/103

● How to Write APIs That Will Stand the Test of Time● http://openide.netbeans.org/tutorial/testpatterns.html● download from http://jpackage.org● download from http://xtest.netbeans.org

http://openide.netbeans.org/tutorial/test-patterns.html


Q&AJesse GlickMiloš Kleint

Jaroslav Tulach

Documents

Test Patterns in Java - NetBeans...return java.util.ServiceLoader.load(DialogDisplayer.class).iterator.next();}} define abstract interfaces and “lookup” implementation: real implementation