Certification Test Tool Jon Wheeler Test Lead Microsoft Corporation

Certification Test Tool

Jon Wheeler

Test Lead

Microsoft Corporation

Assumptions

Familiar with UPnP v1 architecture and terminology

Interested in evaluating a device’s compliance against the UPnP standard

Pursuing the UPnP Logo

Overview

Certification Test Tool purpose

What is tested

Test capabilities

Using the tool and interpreting results

Future versions

Tool is posted on www.upnp-ic.org

Purpose of Cert Tool

Drive the certification of UPnP devices

Execute device independent syntax tests against the UPnP v1 standard

Support device dependent syntax and semantic testing

Certification Tests

Govind Ramanathan

Software Development Engineer

Microsoft Corporation

What is Tested

Device independent protocol tests

Device specific syntax tests

Device specific semantic tests

Protocol Tests

Validates SSDP and HTTP packets and their sequence

Tests the following stages: Addressing, Discovery, Description, and Eventing

Protocol tests are executed regardless of device type

Syntax Tests

Test actions can be specified for services that are specific to the device undergoing test

Syntax tests are created by editing the <ServiceId>.SyntaxTests.XML file

Tests are run during the Control and Eventing test stages

Actions are invoked in the order in which they appear in the XML file

Semantic Tests

Verifies specified output parameters, error conditions or events

Create semantic tests by editing the <ServiceId>.SemanticTests.XML file

Actions must be within the same service Results of one action cannot be used as

input parameters in another action

What is Not Tested

Does not test performance specifications

Does not certify device extensions external to the UPnP standard

Test Environment

Test tool requires Windows XP

Run only one UPnP device and a single instance of the test tool on an isolated network

Turn off LAN connection settings (i.e. proxy settings) in Internet Options control panel on Windows XP

Test Environment Diagram

UPnP Cert Tool running

on Windows XP Device under test

Isolated network

DHCP server for AutoIP and DHCP tests

Sniffer for debugging (optional)

Configuring XML Files Tool relies on XML configuration files

<DeviceId>.xml Gives the Services that this device implements and the embedded Gives the Services that this device implements and the embedded

devices.devices. <ServiceId>.xml

Gives the Actions and State Variables implemented.Gives the Actions and State Variables implemented. <ServiceId>.SyntaxTests.xml

Gives the Action names and the ‘In’ arguments for each Action.Gives the Action names and the ‘In’ arguments for each Action. <ServiceId>.SemanticTests.xml

Gives the Action names, ‘In’ arguments and the expected ‘out’ Gives the Action names, ‘In’ arguments and the expected ‘out’ arguments or error code.arguments or error code.

Using XML Config Files

Working Committees write the XML config files Standardized config files are distributed with the

tool

From the XML files the tool verifies if Devices have all required service. Services implement required Actions and State

Variables. Responds correctly for each Action invocation.

Proprietary devices can be tested but require the respective XML config files

Device Description<device>

<deviceType>urn:schemas-upnp-org:device:WANDevice:1</deviceType>

</device>

<device>

<Optional/>

<deviceType>urn:schemas-upnp-org:device:LANDevice:1</deviceType>

</device>

Service Description<serviceStateTable>

<stateVariable>

<name>EthernetLinkStatus</name>

<dataType>string</dataType>

<allowedValueList>

<allowedValue>Up</allowedValue>

<allowedValue>Down</allowedValue>

</allowedValueList>

</stateVariable>

</serviceStateTable>

Syntax Tests XML File

<TestCase>

<Id>1</Id>

<Category>Valid Action And Valid InArgs</Category>

<ActionName>GetEthernetLinkStatus</ActionName>

<ExpectedReturnCode>ACTION_AND_INARGS_ARE_VALID</ExpectedReturnCode>

</TestCase>

Semantic Tests XML File

<ManualAction>

<ActionInstruction> Remove cable. </ActionInstruction>

<VerificationInstruction> Verify link light is off.

</VerificationInstruction>

<ExpectedSuccessResponse />

</ManualAction>

Launch the Application

Select Test Options

Step Mode / Run Until Failure

Step Mode Pauses between each test Click the “Next” button to continue Useful when debugging a failure

Run Until Failure Pauses the test execution at the first failure

Interpreting the Test Log

Test Case 2.0.0.1 Passed

Test Case 2.1.0 Failed to RunFailure Reason :Precondition Failure - Unable to

Construct Device Discovery Message List.

Reason : Unable to Acquire Root Device Description Url.Badly formed Notify Alive Packets Header Value.

Reason : NT Header Value = SSDPBVT is Not Compliant to UDA V1.0

Tips and Future Versions

Jon Wheeler

Test Lead

Microsoft Corporation

Helpful Hints

Review the test log for failure details

Use a network sniffer (e.g. MS NetMon) to monitor network activity

Use Step Mode to help isolate an issue

Read the Test Tool User doc which lists the test cases

More Tips

A full certification pass takes about 25 minutes to complete

For testing purposes, set the CACHE-CONTROL in the device to be less than 60 for SSDP alive notifications.

In your final device implementation, set the CACHE-CONTROL value to be > 1800

Future Version Plans

Support testing against the v1 security standard

Secure logs to prevent tampering

Include more extensive logging

Expose interfaces for developing test scripts

Support future UPnP standards (v2)

Q&A

Tool is posted on www.upnp-ic.org

User documentation is included with the tool

Developer support

Questions?

For the interconnected lifestyle