ISV / Software Solutions competency Testing Process with ...download.microsoft.com/.../isvdays05/isv_comdaysnov05_ms_platform_test_veritest.pdfISV / Software Solutions competency Testing

PROPRIETARY & CONFIDENTIALPROPRIETARY & CONFIDENTIAL

ISV / Software Solutions competencyTesting Process with VeriTestISV / Software Solutions competencyTesting Process with VeriTest

November, 10th & 11th November, 10th & 11th 20052005

© 2003 Lionbridge Technologies, Inc. © 2005 Lionbridge Technologies, Inc.

Agenda

� Veritest Overview

� Platform Test Specifications

� Online Registration Process

� To get ready & to support the test effort

� Questions and Answers


Veritest OverviewVeritest Overview

� Alexis Evrard

- Based in France (Nice)

- 15 Years in the testing area

- 6 Years with VeriTest

- Used to be Technical Account Manager for Certification Programs

- Now Solutions Architect

*Any question/concern about all test services provided by VeriTest BUTcertification, feel free to pick a business card and contact me later


Lionbridge Technologies: Continues to GrowLionbridge Technologies: Continues to Grow

� Stable, Public Company (NASDAQ: LIOX)

- $400M+ in 2005, profitable

- Continued growth expected in 2006

- Recent acquisition of Bowne Global Solutions created one of the world's largest globalization and offshore services company

� Global IT Outsourcing Solutions

- Application Development & Maintenance (Lionbridge)

- Content Development, Conversion & Enhancement (Lionbridge)

- Globalization (Lionbridge)

- System, Application, Product, and Localization Testing (VeriTest)

14 of the world’s 20 most valuable companies are Lionbridge customers.

BusinessWeek 2004

$0

$20,000

$40,000

$60,000

$80,000

$100,000

$120,000

$140,000

$160,000

$180,000

$200,000

2002 2003 2004 2005

(est.)

Continuing Revenue Growth


VeriTest: Setting the Standard in Testing Since 1987VeriTest: Setting the Standard in Testing Since 1987

� World’s largest independent quality assurance and software testing company

- Leading test architects, engineers, and analysts across US, Europe, Asia

- Scalable offshore execution in India and China

- Unmatched, global-networked infrastructure---from 32-way servers to data center class storage lab

� Industry leader

- Exclusive provider and architect of industry-leading certification programs

- Developer of PC Magazine benchmarks

- Test and publish industry-standard benchmarks


Global Clients, Global Response:Global Clients, Global Response:Capability and Capacity Spanning 25 CountriesCapability and Capacity Spanning 25 Countries

Galway, Ireland

Mumbai, India

Los Angeles, CA

San Francisco, CA

Seattle, WAMinn, MN

Boise, ID

Boulder, CO

Amsterdam, The Netherlands

Framingham, MA

Corporate Headquarters: Waltham, Massachusetts, USA

Dublin, Ireland

Ballina, Ireland

ParisSophia Antipolis, France

Beijing, China

Seoul, Korea

Tokyo, JapanFort Collins, CO

Chennai, India

Taipei, Taiwan

Yokohama, JP

Singapore

Rio De Janeiro, Brazil

Sao Paolo, Brazil

Santiago, Chile

Madrid, Spain

Wuppertal, GermanyBrussels, Belgium

Göteborg, Sweden

Coventry, UK

Espoo, Finland

Lisbon, Portugal

Eindhoven, Netherlands

Munich, Germany

Zilina, Slovakia

Montreal, Can

Copenhagen, Denmark

Sindelfingen, Germany

Rome, Italy

Bergen, Norway

Warsaw, Poland

Stockholm, Sweden

Luton, UK

New York, NYWashington DC


Global Program ManagementGlobal Program Management

Develop

Test

Deploy

Globally

Networked Worldwide Infrastructure

Offshore and Onshore

Lionbridge Delivers A Unique Set of CompetenciesLionbridge Delivers A Unique Set of Competencies


VeriTest: Independent Quality Assurance and Software TestingVeriTest: Independent Quality Assurance and Software Testing

QA Consulting CertificationPerformanceTesting

InteroperabilityTesting

FunctionalTesting

Test Automation

Localization Testing


Microsoft MBS Test for ISV Solutions

Microsoft ISA Server Test - 20 pts

Microsoft MBS, CRM

Microsoft ISA Server

Designed for Windows Mobile* – 20 ptsWindows Mobile

Benefits may also include one or more of the following:

� Use of Certification Logo (* only)

� MS Catalog Listing

� Other Product Group BenefitsCommerce Server 2002 Integration testing – 20 pts

Commerce Server

Certified for SQL Server 2000 / 2005

SQL DWA testing

60 points

Verified for SQL Server 2000

SQL Server 2000 / 2005

.NET Connected Logo testing* -10 pts

.NET Framework

Windows Family XP Home / Pro2000 SRV /2003 SRVStandard Ed.Advanced/Enterprise Ed.Datacenter Ed.

Lead to MS GOLD Certified PartnerValidation of platform interoperability, ensuring product meets MS dev standards. Typically 1-4 weeks.

Lead to MS Certified Partner

Validation of platform interaction, ensuring basic ISV commitment. Typically 3-5 days.

Microsoft Technologies

Microsoft Global Certification Microsoft Global Certification ProgramsPrograms at VeriTestat VeriTest

MS Platform Test

for ISV solutions

20 pts to 60 pts

Certified for Windows Srv 2003*

60 pointsDesigned for Win XP*

– 20 pts

Verified for Win 2003 –10 pts

Verified for Win XP –10 pts


Microsoft Platform Test Specifications for ISV Solutions at VeriTest

Microsoft Platform Test Specifications Microsoft Platform Test Specifications for ISV Solutions at VeriTestfor ISV Solutions at VeriTest


Platform Test SpecificationsPlatform Test Specifications

The Platform Test consists of two foundation components, and four elective components.

A software application is required to pass two components of the Platform Test, includingat least one foundation component.

The foundation components are the Windows Server and Windows Client components : Win Server 2003* , Win XP. The elective components are :

•Microsoft Office, •SQL Server 2000 / SQL Server 2005•Web Services plus .NET Framework (.NET Connected logo test),•Managed Code.

The application must pass the Windows Server or the Windows ClientComponent, plus at least one additional component.

Hardware : dual – processor machine

(*) Please note Windows Server 2000 will no longer be an option for the Windows Server component.


Platform Test SpecificationsPlatform Test Specifications

List of the options:

Windows Server / SQL Server (2000 or 2005)

Windows Server / Managed Code

Windows Server / Web Services and .NET Framework

Windows Server / Office 2003

Windows Server / Windows Client (*)

Windows Client / SQL Server

Windows Client / Managed Code

Windows Client / Web Services and .NET Framework

Windows Client / Office 2003

Windows Server plus 2, 3, 4, or 5 components

Windows Client plus 2, 3, 4, components

(*) only if the client code and the server code are different


Checklist for the Platform Test for ISV SolutionsChecklist for the Platform Test for ISV Solutions

Microsoft Windows ClientFundamental Requirements

Desktop applications must comply with all fundamental requirements to pass testing for this component

1. Executes on Microsoft Windows XP and maintains stability while performing primary functionality

2. Uses Windows Resources (heaps, locks, and handles) appropriately

3. Does not attempt to replace files protected by Windows File Protection

4. All device or filter drivers installed by the application are digitally signed by Microsoft WHQL

5. All kernel-mode drivers installed by the application pass Windows driver verification

Optional Requirements

Desktop applications must comply with any one of the optional requirements to pass testing for this component

1. Does not require a reboot during installation, operation, or removal

2. Provides installation program that supports "All Users" Installs

3. Remains stable while performing Fast User Switching

4. Supports use by a Limited User

Note:

There is a constraint of 2 hours for setup and configuration. Installation of products should be intuitive and easy to

follow by anyone with typical Administrator abilities. Products must include a documented installation procedure,

preferably with an automated installation routine.



Microsoft Windows ServerFundamental Requirements

Server applications must comply with all fundamental requirements to pass testing for this component

1. Executes on Microsoft Windows Server 2003 and maintains stability while performing primary functionality

2. Uses Windows Resources (heaps, locks, and handles) appropriately

3. Does not attempt to replace files protected by Windows File Protection

4. All device or filter drivers installed by the application are digitally signed by Microsoft WHQL

5. All kernel-mode drivers installed by the application pass Windows driver verification


Server applications must comply with any one of the optional requirements to pass testing for this component

1. Does not require a reboot during installation, operation, or removal

2. Does not disable other services during installation, operation, or removal

3. Supports Active Directory

4. Supports Windows Management Instrumentation (WMI)

5. Utilizes Windows SharePoint Services

6. Utilizes ASP.NET for Web Applications

Note:

There is a constraint of 2 hours for setup and configuration. Installation of products should be intuitive and easy to

follow by anyone with typical Administrator abilities. Products must include a documented installation procedure,

preferably with an automated installation routine.



Microsoft Office 2003

Applications must comply with any one requirement to pass testing for this component.

Application requires at least one of the programs included in Microsoft Office 2003

Editions to exercise some of its documented functionality. Eligible applications must be one of the following:

1. Application includes a COM add-in for Microsoft Office 2003

2. Application includes a VBA add-in for Microsoft Office 2003

3. Application includes a VSTO add-in for Microsoft Office 2003

4. Application exposes data in Microsoft Office 2003 via Research and Reference feature

5. Application integrates data in Microsoft Office 2003 via Smart Tags

Special notes

MS Access : if the application use a mdb file as a data store, the application is eligible for the Office Component

If the application launches a member of the suite from the Application UI



Microsoft SQL Server

SQL Server 2000 or SQL Server 2005

Fundamental Requirements

Applications must comply with all fundamental requirements to pass testing for this component

1. Supports ADO, ADO.NET, OLE DB, ODBC, or JDBC to connect to SQL Server


Applications must comply with any one of the optional requirements to pass testing for this component

1. Requires SQL Server 2000 SP3

2. Supports SQL Server Authentication or Windows Authentication



Web Services and the .NET Framework

Applications must comply with any one requirement to pass testing for this component

1. Exposes a Web service using .NET Framework or .NET Compact Framework

2. Consumes a Web service using .NET Framework or .NET Compact Framework

Please note : in both cases, dll and exe files need to be written in Managed Code



Managed Code

Applications must comply with all requirements to pass testing for this component

1. All Application Assemblies consist of Managed Code

Tool : FxCop

Note : The .net Framework v1.1 or v2.0 is fine


FAQ‘s :

Which Application to submit ?Which Components to select ? The costs ? What happens if it fails ? How to register and submit my application ?

FAQFAQ‘‘s :s :

WhichWhich ApplicationApplication to submitto submit ??WhichWhich ComponentsComponents to selectto select ? ? The costsThe costs ? ? WhatWhat happenshappens ifif it failsit fails ? ? HowHow to to registerregister and and submitsubmit mymy applicationapplication ? ?


WhichWhich ApplicationApplication to submit?to submit?

� What we can test:

- Software that goes to the market (with 3 client references),

- Components like ActiveX with sample application/code

- Applications with any 3rd party h/w or s/w as long as : compliant and provided by the ISV

� What we cannot test:

- Development libraries (i.e. functions)

- Installation time may be a blocking factor: the base test fee includes up to two hours installation time, including back-end elements. Charges for additional time to install and configure products for testing: $110/h

� You may opt to ship pre-configured hardware for backend elements or provide technical assistance to VeriTest in order to reduce installation time

� Connection through the Internet

� MS Virtual PC for the test environment if none of the application code runs on it (ex: Oracle etc)


WhichWhich test componentstest components to selectto select ??

Application environment :

Is my application running on Win Server or Win XP?

Does my application have a client and a server?

Does my application use MS SQL Server 2000 or 2005 ?

Does my application use MS Office 2003?

Does my application use the .Net Framework?

If yes, does it use / provide Web services?

If yes, is it managed code compliant?

(*) Please note that only client/server applications are eligible for testing for both the Windows Client and Windows Server components of the Platform Test. A client/server application is defined as an application that installs code on the Windows Client platform, and different code on the Windows Server platform, and there is interaction between the code on the two platforms.

List of the options:

Win Server and / or Windows Client (*)

SQL Server 2000

Office 2003

Managed Code

Web Services and .NET Framework


Costs : $400 / componentCosts : $400 / component


Costs : additional charges Costs : additional charges

Microsoft Platform Test for ISV Solutions at VeriTest: Additional fees

Application Interface Language

The basic test fee covers testing of applications with an English user interface. Applications with a user interface in a different language will be charged an extra fee:

$100 per component tested

Installation time

The base test fee includes up to two hours installation time, including back-end elements. Charges for additional time to install and configure products for testing:

$110 per additional hour

You may opt to ship pre-configured hardware for backend elements or provide technical assistance to VeriTest in order to reduce installation time.

Retests

Retests on new builds if the test fails: 65% of the original public test fee.

Credit card transactions : add 3.5% to the total fee


WhatWhat happenshappens ifif mymy applicationapplication passespasses oror failsfails ??

� In case of success, an “Pass report ” email is sent to the ISV as soon as the test is done.

� What if the application fails ?

- A detailed Fail report is sent by email as soon as the test is done.

- The Fail report includes steps to reproduce

- Retest is at 65% of the initial public price

� How to get prepared:

- read and read again the specifications

- pretest on a clean machine : OS from scratch, NTFS, empty Dllcache

- The upper test level may give more precise information


RRegistration Processegistration Process : : VeriTest Web SiteVeriTest Web Site


RRegistration Processegistration Process : : VeriTest Web SiteVeriTest Web Site


Registration ProcessRegistration Process

� Online registration : https://cert.veritest.com/register

- Vendor Questionnaire

- Software : FTP or mail

- Payment : online


OnOn--line registrationline registration

Total time:

45-60 minutes

Once you’ve entered this information, you can always come back later to make changes.



3-5 minutes



10-15 minutes Remember to make a note of your Tracking

Number.







5-10 minutes



5-10 minutes

Repeat this step to select all the test components you wish to have your product tested for.

Note: Your product has to pass testing for at least the Windows Client or Windows Server test component plus one other test component in order to pass testing for the Platform Test overall














How to get readyHow to get readyHow to get ready


How to get readyHow to get ready

How to support the Certification engineer

� All supporting materials in English

� Step by step / click by click installation / configuration document : 10 – 20 pages with screenshots and basic guidelines

� Provide a demo database, sample data set

� Submit the application with the right licensing and/or user/password

� Responsive technical contact as advertised in the online registration

� Alias Email (e.g. : [email protected])


How to get readyHow to get ready

How to get prepared :

�Read and read again the specifications

�Internal preTest on a clean machine :

•install from scratch Windows + SP + Critical updates

•empty c:\windows\drivercache folders (enable show hidden / system files)

•empty c:\windows\system32\dllcache folders

•run in a command line « sfc /CACHESIZE=0 », then reboot

�The upper level may give more precise information :

- Certified for Windows 2003 Server

- Certified for SQL Server

- Designed for Windows XP


SomeSome ToolsTools……

�Resource use : how to use the AppVerifier

search for ACT 3.0 @ www.microsoft.com/downloads

�Managed Code : how to use FxCop

http://www.gotdotnet.com/team/fxcop

�Kernel mode Driver test : how to use verifier.exe

�WsdlVerify : WsdlVerify is a proprietary tool from VeriTest that invokes a Web Service over the Internet to ensure the WSDL file for a Web Service is compliant with the WSI standards. The WSDL file must be accessible from the Internet for testing.


ToolsTools : AppVerifier: AppVerifier


Resources and ContactsResources and Contacts

•[email protected]•http://www.veritest.com/certification/ms/platformtest.asp

•[email protected]

Documents

ISV / Software Solutions competency Testing Process with ...download.microsoft.com/.../isvdays05/isv_comdaysnov05_ms_platform_test_veritest.pdfISV / Software Solutions competency Testing