1

RealProct: Reliable Protocol Conformance Testing with Real

Nodes for Wireless Sensor Networks

Junjie Xiong, Edith C.-Ngai, Yangfan Zhou, Michael R. Lyu

2011-11-18

2

Outline Motivation RealProct Design Evaluation Conclusion

3

MotivationCurrent main methods to fight back the software bugs in Wireless sensor networks (WSNs): Simulation: different from real execution. Testbeds: designed for network performance evaluation rather than software bug detection. Large-scale real deployment: expensive.

uses a small number of real sensor nodes to mimic large-scale WSNs and test the protocol implementation against the specification -> RealProct

uses a small number of real sensor nodes to mimic large-scale WSNs and test the protocol implementation against the specification -> RealProct

4

Outline Motivation RealProct Design Evaluation Conclusion

5

Challenges

Sensor node is difficult to control than a computer. Limited CPU and inconvenient interface.

How to test the protocol with various topologies and events when RealProct only employs a few real sensor nodes.

Volatile wireless environment in WSNs will lead to random packet loss, and cause problems in testing.

6

RealProct Solutions to the Challenges

An architecture that enables testing with real sensor nodes.

Topology virtualization and event virtualization. Dynamic Test Execution.

7

Background of RealProct Protocol conformance testing (PCT) process

IUT (Implementation Under Test)

8

RealProct ArchitectureSUT (System Under Test)

Lower Tester

Upper TesterPoint of Control & Observation

9

Topology Virtualization

Use the tester to virtualize a 3-node topology for SUT

Addr2

Addr1

Addr3

SUT

Tester

Pac

ket 1

Packet 2

Content of Packet 1: Sender address is Addr1.

Content of Packet 2: Sender address is Addr2. The sender has a neighbor with Addr3.

10

Event Virtualization

Use the tester to create a packet disorder event at the SUT. P

acket 2

Larger sequence #

21

4

SUT

3

5

Tester

Route 1

Route 2

Neighboring nodes are connected by dotted lines

Pack

et 1

Smaller sequence #

11

Reason to Use Dynamic Test Execution

Suppose packet loss probability is L0, a test case is executed n times, and it passes n1 times and fails n2 times.

If n1 > n2, declare as pass, the FN (false negative) probability is

If n1 < n2, declare as fail, the FP (false positive) probability is

12

Dynamic Test Execution To guarantee that both the FN and FP error

rates are lower than a required value E, the minimum time to execute each test case is:

The actual execution times are dynamic. Repeat the test case execution until its FN and FP

error rates are lower than E.

13

Outline Motivation RealProct Design Evaluation Conclusion

14

Performance Evaluation Two real TelosB sensor

nodes and a PC. Contiki 2.4: μIP TCP/IP

protocol. Two new bugs and previo

us bug repetition. Bug 1 & 2 – Connect to opened

& unopened TCP ports. Bug 3 – SYN/ACK packet loss. Bug 4 – SYN packet duplication.

15

Performance Evaluation

Bug 1 – Connect to opened TCP ports. Test opened port 0 & 80 (within 0 to 65535).

16

Performance Evaluation

Bug 1 – Client (Tester) connects to opened TCP port 0 of Server (SUT).

Bug: Client expects SYN/ACK response while it receives no reply.Bug: Client expects SYN/ACK response while it receives no reply.

17

Performance Evaluation

Bug 2 – Client (Tester) connects to unopened TCP port 0 of Server (SUT).

Bug: Client expects RST response while it receives no reply.Bug: Client expects RST response while it receives no reply.

18

Outline Motivation RealProct Design Evaluation Conclusion

19

Conclusion

As a protocol testing tool, RealProct finds two new bugs, discovers all the previously detected bugs in the TCP/IP stack of WSNs, and validates the Rime mesh routing protocol.

Propose two techniques, topology virtualization and event virtualization, for testing.

Design an algorithm to tackle the inaccuracy problem caused by non-deterministic events in test execution.

20

Q & AThank you!

21

Codes that Cause Bugs

22

Repeat Bug – SYN Packet Loss

Repeat Bug – Duplicate SYN Packet

TCP client: 1025

[SYN/ACK]seq=0,ack=1

Sends SYN

SYN/ACK received, sends ACK

[SYN]seq=0

[ACK]seq=1,ack=1

SYN received. Server sends SYN/ACK

TCP server: 80

ACK received and connection established.

[SYN/ACK]seq=0,ack=1

[SYN]seq=0

Duplicate SYN receivedBug: Server sends

SYN/ACK

24

Dynamic Test Execution