Waters Quality System Documentation...AutoTest performs automated testing to evaluate such characteristics as correctness, completeness, consistency, traceability, testability and

Waters Quality System Documentation DOCUMENT NUMBER

WAT000108MP TITLE REVISION REVISION DATE

DPG SOFTWARE QUALITY PLAN 8 Mar 25, 2002 DOCUMENT TYPE PAGE NUMBER

MANAGEMENT PROCEDURE 2 of 19

NOTICE THIS DOCUMENT AND ITS CONTENTS ARE EXCLUSIVE PROPERTY OF WATERS CORPORATION, MILFORD MASSACHUSETTS. THIS DOCUMENT MAY NOT BE REPRODUCED IN ANY FORM WHATSOEVER, WITHOUT PRIOR WRITTEN PERMISSION FROM WATERS CORPORATION.

1.0 Purpose

∆ The purpose of this plan is to define the quality actions and responsibilities for software development within the Data Products Group, RD& E Evaluation, and other groups.

2.0 Scope

This plan covers applications software developed by the Data Products Group for sale as part of an external computer system for use with Waters and non-Waters instruments. It pertains to the software development process starting from the market requirements definition activities through release and maintenance. It excludes manufacturing except for the verification of the diskette masters. This plan is just one step in the quality process for applications software development. The entire process was developed to ensure the quality of the final product.

3.0 Definitions

AutoTest - Involves the use of a machine and logical specification language that can automatically analyze a specification for consistency, closure properties, or presentation of cross-referencing information for manual analysis. Methods of verifying and validating a software system which involves detailed automated models and prototypes that describe the system in detail. These tools aid in the verification of such characteristics as correctness, completeness, consistency, traceability, testability and adherence to standards.

∆ Acceptance Tests – Tests done near the end of a given development phase, to ensure that the development goal has been met.

Beta Site - Customers who have the necessary expertise to assist in the evaluation of software.

Boundary test -Challenges the software at the extremes of its defined operating limits to determine how the software will function.

∆ Code Review - A review of the source code implementation of a functional feature or software component with the author and members of the development group.

∆ Design Review – A review of the design aspects of a functional feature or software component with the author and members of the development group.

Informal testing - Testing of an existing or proposed software product, typically by originator’s peers, for the purpose of detecting and correcting product errors.

Module test -Test of a program unit that is discrete and identifiable with respect to compiling, combining with other units, and loading; for example the input to, or output from, an assembler, compiler, linker or executive routine. Documented evidence in task completion results.






3.0 Definitions (continued)

Object-oriented method - A software engineering methodology in which the system is viewed as a collection of objects, attributes of the objects, and operations on the objects, with messages passed from object to object.

Operation test - Verifies the contents of the operator's documentation, his or her training, and the individual's ability to perform standard operations on the system.

Performance test - Verifies that the system can handle, within defined limits, the normal and higher amount of transactions (database applications) of the normal and expected higher limit of input or output (control or data acquisition). Response time within a range of normal and expected system loading is also part of performance testing.

∆ R D & E Evaluation - Instrument and software evaluation laboratories within the R D & E organization.

Recovery test -Confirms that the system can return to a functional state after a failure.

Regression test - The rerunning of automated test cases that a program has previously executed correctly in order to detect errors created during software correction or modification activities.

SQEL- Software Quality Evaluation Lab external to the Data Products Group responsible for evaluation of software developed by the Data Products Group.

Security test - Verifies that the appropriate system security has been designed, put into place, checked for proper operation, challenged for potential violations, and documented.

Stress test -Challenges the system at extremes of its defined operation limits to determine how the system would function under a heavy load.

System test - The activity of checking that the entire software system is fully integrated in its actual hardware environment according to the requirements specification.

Task Completion - Ensures that the required functionality is present and that it works correctly.

Temporary code - Code that is not a part of the released software but is used by a developer to evaluate a program or module.

∆ Traceability – Traceability ensures that features requested in the Marketing Requirements Document are implemented in the code and then evaluated, as evidenced by the Functional Specification, Task Completion Reviews, the Test Plan and individual test cases.

Test case -A specific set of test data and associated procedures developed for a particular objective.

Waters Document Control - Has defined set of procedures whereby all product information is archived.






4.0 Management:

4.1 Organization

This document pertains to the Data Products Group and RD&E Evaluation, the organizational charts for which are shown in Appendix A. An overall Waters organizational chart is provided in Appendix A to show where other groups and the Quality department fit into the organization.

External to the Data Products Group, SQEL (part of Research Development and Engineering Evaluation) has the main responsibility for the evaluation of software products. This group includes the AutoTest group which works with the other groups to automate appropriate test cases. The organizational chart for Research Development and Engineering is shown in Appendix A.

External to the Data Products Group, Instrument Evaluation (part of Research Development and Engineering Evaluation) and the marketing labs (part of marketing) may assist SQEL in evaluation of software products. The organizational chart for Research Development and Engineering is shown in Appendix A.

The Marketing Product Manager plays a key role in the Software Development Process by defining the product requirements and insuring that they are met as defined in the Marketing Requirements Definition Document (Document #000110MP).

External to the Data Products Group, the Director of Regulatory Affairs (in the Operations Division reporting to Quality; see organizational chart in Appendix A) informs the Data Products Group of changes in the Regulatory environment and organizes audits by customers (Procedure for on-site Customer Audits/Applications Software - 000159MP) and the Food and Drug Administration.






4.0 Management (continued):

4.2 Tasks

The Data Products Group specifies, designs, and implements software and maintains the software after release as per the DPG Software Development Process (Document #000104MP), and the Software Release Procedure (Document #000126MP).

SQEL performs integration and system testing, regression testing, recovery testing, operation testing, performance testing, security testing, and stress testing on systems and software.

AutoTest performs automated testing to evaluate such characteristics as correctness, completeness, consistency, traceability, testability and adherence to standards. These test cases are also used for regression testing between versions of applications software code.

SQEL also maintains the software change request database, and releases the software for evaluation by other groups as per the Procedure for Software Change Control-PCS (Document #000123MP) and the Procedure for Release to General Evaluation (Document #000138MP).

Testing on software developed by the Data Products Group may also be performed by Instrument Evaluation and marketing applications labs.

Data Specialists may assist in the product evaluation managed by SQEL.

The Vice-President of the Data Products Development is responsible for the development of software to be used by customers and identifies the Development Project Manager. He/she coordinates activity with other Waters' organizations as needed, sets project development priorities and assigns project responsibility as per the Procedure to Create a Development Plan (Document #000112MP).







4.3 Responsibilities

The Marketing Product Manager is identified by the Director of Data Products Marketing. The Marketing Product Manager coordinates market research and creates the Market Requirements Definition as per the Marketing Requirements Procedure (Document #000110MP). He/she markets the product, is responsible for overall product commercialization and coordinates the activities of the other departments within Waters involved in defining, producing, promoting, pricing, introducing, and maintaining new products. The Marketing Product Manager will work with the Director, RD&E Evaluation to determine the need for beta testing and assist in the selection of customers to be included in the testing as per the DPG Procedure for Beta Testing (Document #000142MP).

Development Project Managers in the Data Products Group write the functional specification as per the DPG Functional Specification Procedure (Document #000114MP), implement the software and coordinate the efforts of the personnel assigned to the project. They plan the project activities and are responsible for meeting schedules. They are also responsible for the generation of documentation identified in the Quality Plan (this document), the DPG Software Development Process (Document #000104MP), and to conduct the required document reviews, and code and design reviews as required by the Procedure to Create a Development Plan (Document #000112MP).

The Director, RD&E Evaluation is responsible for budgeting, internal document control and maintenance, and for providing the processes, test equipment, tools, and skills to verify the software quality and functionality. Individual test engineers are assigned to participate during implementation of the software as active members of the project team. They are responsible for performing tests and reporting test results as per the Procedure to Create Test Cases and Enter Results (Document #077420SO). The Director, RD&E Evaluation is responsible for or assigns a qualified individual to manage evaluation according to the DPG Software Quality Plan (this document), the Test Plan as per the Procedure to Create a Test Plan/Applications Software (Document #000120MP) and Test Summary report(s) generated from the results. The Director, RD&E Evaluation is responsible for the Automated Test Group and assigning them to perform tests and reporting results as per the Procedure to Create a Test Plan/Applications Software (Document #000120MP). The Director, RD&E Evaluation oversees or assigns a qualified individual to oversee specific product evaluation and directs the efforts of Data Specialists, field beta sites and support personnel that assist in product evaluation







The Product Marketing Manager, the Development Project Manager, and the Director, RD&E Evaluation together determine which of the following laboratories and external evaluators in addition to SQEL will perform tests. The Director, RD&E Evaluation coordinates or assigns a qualified individual to coordinate the evaluation activities of:

Instrument Evaluation (DEL) Application laboratories Other individual evaluators who may be selected to test the software for a specific purpose in a particular environment.

The Quality Project Manager participates in each milestone review, to ensure that project document deliverables are complete, consistent, and compliant to applicable procedures.

5.0 Documentation

The following documents are the minimum required for software products. They undergo a formal approval process as per the DPG Procedure to Review and Approve Documents (Document #077432SO) and are stored in the DPG Archives for a minimum of three months then transferred to a secure off-site location as per DPG Records Procedure (Document #000130MP).

Document Responsibility Market Requirements Product Marketing Manager Functional Specification Development Project Manager Test Plan Evaluation Project Manager Task Completion Forms Development Project Manager Test Summary Report(s) Evaluation Project Manager/Manager of AutoTest Development plan(s) Development Project Manager Code & Design Reviews Development Project Manager Test Cases and results Evaluation Project Manager

Various additional documentation may also be produced and stored including but not limited to the following:

Document Responsibility

Monthly reports Manager of R&D Evaluation Development Project Manager

Project decision memos Development Project Manager Training Records Department Managers Software design memos Development Project Manager






6.0 Standards, Practices, Conventions and Metrics

The Vice-President, Data Products Development is responsible for setting the standards and auditing the performance of projects.

The Development Project Manager assures that the software is developed according to the standards and conventions of the Data Products Group, adopting where practical best examples or improvements.

7.0 Reviews and Audits

The Data Products Group Software Development Process Document #000104MP and the specific procedures it references describe the technical and management reviews for the software development process.

Market Requirements Document Review - The Market Requirements Document is reviewed by Marketing for content and accuracy, and by the Data Products Group for its completeness as a basis for developing the Functional Specification and Test Plan. This document is formally approved and signed off. Changes to the document can occur due to technical and/or evaluation constraints (see the DPG Marketing Requirements Procedure 000110MP).

Functional Specification Review - The Functional Specification is reviewed by Marketing to determine whether it satisfies the Market Requirements, by the Manager of R& D Evaluation to assure that a Test Plan can be written and for its completeness as a basis for implementation of the software. Marketing, Data Products, and other groups including SQEL, Instrument Evaluation, and applications labs review the Functional Specification for design quality from a user's perspective. This document is formally approved and signed off (see the DPG Functional Specification Procedure 000114MP).

Project Management Reviews - Periodic reviews of project definitions, project plans, and project status are performed by the Development Project Manager, Marketing Product Manager, the Director of the Data Products Development, Director of Data Products Marketing and other management personnel as appropriate. . .






7.0 Reviews and Audits (continued):

A First Shipment Release Meeting must be held to review the status of the project and make the decision to release the product as per Software Release Procedure (Document #000126MP). The meeting(s) are called by the Marketing Product Manager and include the Development Project Manager, Director, RD&E Evaluation, Evaluation Project Manager, the Vice-President, Data Products Development, the Director of Data Products Marketing, Director of World Wide Quality and any others who could help determine the status of the product.

Test Plan Review - The project Test Plan is reviewed by the Development Project Managers, Product Manager, Manager of RD&E Evaluation, Manager of AutoTest and Manager of SQEL. The manager of Instrument Evaluation (DEL) reviews the Test Plan when appropriate.

∆ Audits – Internal audits are conducted on a routine basis for the entire software quality system. In addition, many project-specific documents are audited by the Quality Project Manager as part of the various milestone reviews to ensure completeness, consistency, and compliance to applicable procedures.

8.0 Test

The Director, RD&E Evaluation generates or assigns an Evaluation Project Manager to generate a Test Plan according to the Procedure to Create a Test Plan/Applications Software (Document #000120MP) and assigns resources to execute it. The Test Plan is based on the approved requirements in the Functional Specification and does not include items that are marked as Deferred. The software product is not signed off for release to manufacturing until the Pass/Fail criteria (as defined by the “Required” elements of the Marketing Requirements Document) are complete and functioning as stated in the Functional Specification and Test Plan for the product.

The Director, RD&E Evaluation is responsible for ensuring that the Test Summary Report is generated from the results database.

The applications software and options will be tested according to the pass/fail criteria specified in the Functional Specification and the Test Plan.

Each developer is responsible for informally testing their own modules before they are submitted for formal functional evaluation. For internal module tests, the developer selects test cases and procedures based on a detailed understanding of the structure and logic of the code. This testing is documented by Task Completion Forms.

For Functional and Structural tests, a test engineer from SQEL is assigned to write test cases and procedures to test each function in the system as according to the Procedure to Create Test Cases and Enter Test Results (Document #077420SO). The test engineer selects test scenarios, test cases, and test procedures to establish actual system performance and uses the functions integrated into an end-user environment.






9.0 Problem Reporting and Corrective Action

Evaluation maintains a Change Request Database. Known problems (change requests) are identified in the database and new requests are reported according to the Procedure for Software Change Control-PCS (Document # 000123MP). It is also the repository for suggested improvements and enhancements to the software. As problems are corrected, their status is updated in the database according to the above procedure.

10.0 Tools, Techniques, and Methodologies

10.1 Project Planning

The Development Project Managers may use third party software to plan, schedule, and monitor projects.

10.2 Implementation

The developers use various third party development tools such as text editors, Borland C++ V3.1 compiler (which includes the linkers and debuggers); PVCS (Polytron Version Control Software) V 5.1 for MS-DOS source code revision control software (see Section 11.0), and other productivity tools in the development of software.

The developers use procedural and object-oriented software design methodologies.

10.3 Evaluation

Evaluators use hand calculations, automated testing, spreadsheets, and side by side comparisons of results from existing products to check results.

The Evaluation Project Manager prepares metrics/reports that reflect current practice in the department. These reports may contain information such as defect density, and the rate of finding problems in the software (software reliability). The graphs are distributed to developers and evaluators in the Data Products Group at the project meetings. Some of the metrics are also reviewed by the New Products Committee during the life cycle of projects that last a year or more.






11.0 Code Control

Revision Control software is used to track and control the code revisions according to the DPG Version Control Procedure (Document #077424SO) and DPG Configuration Management Plan (000106MP). This version control system provides an audit trail of changes to the software and can re-create any older versions of the software. It also prevents file sharing conflicts which can occur in a multi-developer environment. Traceability of changes made to the code is documented according to the Procedure for Code Traceability (Document #077463SO).

12.0 Media Control

∆ Computer code and electronic documentation is maintained on a network server. Server disk backups are made automatically on a periodic, frequent basis according to the Procedure for Server Shutdown/Restart (Document #077470SO) and stored off-site according to the DPG Records Procedure (Document #000130MP). The entire set of newly built executables is up-loaded to the network server as often as the Development Project Manager determines is necessary.

The Software Release Procedure (000126MP) identifies the information required, the responsibility for preparing the information, and the approval process for releasing software for duplication. The procedure also specifies the techniques to control the Software Production Master, the duplication of the software during manufacturing, and the distribution of the product.

13.0 Supplier Control

The Data Products Group purchases the following types of goods/services for software development:

- software development contractors - computer hardware/software - development tools






14.0 Records Collection, Maintenance, and Retention

A corporate policy is established for document control record storage and retention (000006MP and 000050MP).

The following list of documents (as stated on the Software Certificate of Validation ) is the minimum that will be made available to customers who do an on-site audit under non-disclosure (Document 000159MP):

Marketing Requirements Definition Functional Specification Software Quality Plan Development Plan Test Plan Test Cases Test Summary Report Defect Status Reports Source Code Documentation (electronic) Hardware Testing Configurations Revision Status

The originals of these documents except source code documentation, which is on-line, will be stored in the DPG Archives for at least three months after the release of a product. The Archives are administered by the R&D Quality System Manger.

15.0 Training

Developers and evaluators establish their qualifications by experience and by completing appropriate courses in computer languages, operating systems, communications, display graphics, etc.. For particular roles on a project, they may complete commercial courses appropriate to their assignments. Records are maintained according to the Procedure to Document Training (Document #077431SO).

16.0 Quality Commitment

Waters is committed to making every effort to assure that applications software meets or exceed the guidelines described on behalf of customers in the Marketing Requirements Definition. Waters believes that its entire process for software development is part of the total quality assurance for applications' software products.






∆ Appendix A

Waters Division

President, Waters Div.

Marketing

NA/SA Operations

Human Resources

RDE

International Operations

Operations

Finance

Strategic Business

CCD






Marketing

Marketing

Data Products Marketing

(Specialty Marketing)

Core Prod. Marketing

Corporate Comm.

Sales Training






Research, Development & Engineering

RDE

Data Prod. Development

(Instrument Development)

New Technologies

New Business Development

Engineering Services

Evaluation






Evaluation

RDE

Evaluation

SQEL (Software)

Autotest DEL (Instruments)






Operations

Operations

Quality Assembly Operations

Facilities Finance

Machining Operations






Quality

Quality

Regulatory Affairs

Tech. Product Management

Production QA

Quality Systems Reliability






Strategic Business

Compliance Services

Global Serv. Marketing

Customer Education

Serv. Supp. & Training

Data Service Support

LC/MS Marketing

Strategic Business