Telecon 2003-11-05 Minutes - IEEE-SA - Working Groupgrouper.ieee.org/groups/scc20/tii/ATML/Minutes/Meeting... · Web viewSome discussion of configuration control of schemas after

Telecon 2003-11-05 Minutes

Decided that there was a distinction between an Instance of a Unit and a Class of a Unit. Thus was born two common elements UnitInstanceType and UnitIdentificationType. It is intended that both UUTInstanceType and UUTData will extend UnitIdentificationType.

Some attributes of UnitIdentificationType are: partNumber, CAGECode, workUnitCode, modelName.

Some attributes of UUTInstanceType: serialNumber, referenceDesignator, version.

We also began the discussion of what might go into a ConnectorType and how it is related to UnitIdentificationType.

Most people like the working directory structure that was posted with the following comments: Use relative paths for including files, keep all numbered versions in the Archive directory (e.g. ValueType11) with a copy of the latest version in the next higher level but without the version suffix.

Next Telecon, on Wed. Nov. 12, 2003 at 1000 EST


eventClassGeneral discussion on wherther eventClass should be contrained as an enumerate

operator-inputoperator-responsesystem-messagesystem-outputunknown

Altertative proposalidseverity level (enumerated) 1(Message) - 5(Critical) where lower numbers

are less severe than high numbers and level 0 is interpreted as OKeventSource as an xs:stringtimeStamp as xs:dateTime

Action 031119-1John Ralph to update eventClass

UnitTypein ValueType change decimal to doublemaintain the UnitType as a simpleType

Action 031119-2 John to update ValueType from decimal to double

TestResults candidacyIf a common type was only used in one schema it would be in that schema, if it would be used across schemas it should be in common.

The concern is that if we move around the types will the document still validate. The concensious is that any ATML document will still validate provided it uses the correct set of Schemas, and the above rule is still valid.Action 031119-3 John Move IndictmentType into TestResults rather than in a separate file.

Action 031119-4 John to review and modifiy TestResults to modify and simplify WorkOrder. so that the id is used as a link to a full work order reference.

If the TestStartion UUTData or TestProgram changes revsion then the ID should change to reflect that the version that the Test results was run on, rather than hold an instance f the data in the test results.

After an indepth discussion, the meeting was (almost0 convinced to have pretest repairs added into the test results header. Steve Wagner has a proposal to TestResults Action 031119-5 Steve Wagner to offer the propsed V7 of TestResults with pretest repair for comment


Conducted quick review of 11/19/03 meeting action items. No comments.

Review of status of PreTest Repair in TestResults: L-M has reviewed, but not yet prototyped.

Comment: Repair element should be unbounded.

Suggestion made that RepairType should have additional attributes that would enhance Diagnostics.Steve O'Donnell (via Mike Bodkin) to suggest new attributes. Only add if more than one implementor has similar need. Otherwise, just use the "OtherData" (tag/value pairs) element.

Mike Bodkin (L-M) tasked to take lead role on DML. Will coordinate with Tony Alwardt of Boeing.

Steve O'Donnell indicated that Mike Bodkin will not be available until next week to begin prototyping of TestResults. Should expect a report during 12/17/2003 telecon.


TRML looks good as is but limits also belong somewhere else.

Need to figure out approach for Limits held within TRML. LimitType is well defined on its own, so how do we want to seperate the Limits out of the system, reusing the LimitType.

Groove (Saftey of files) everyone has the files in their workspace.When two users make changes there will be a collision.When you drag and drop you get asked whether to replace.

Need a procedure to cover this

Need to assign some project administrationNeed to assimulate questions to ask Groove prior to committing to purchase

CURRENT ANSWERS FOR ABOVE.I have been searching the Groove FAQ and on-line forum. The answers I have so far are:

1. A shared space is exactly that: 'shared'. There is no true owner of the files, as everyone who is invited has a local copy of the files within the space. There is a facility for simultaneous edits of files, provided they are opened from the shared space within Groove. It appears, however, that if two users open the same file at the same time WITHOUT using the co-edit feature, the last save overwrites all prior changes.

If a user is logged in and online, their local copy will be synchronized automatically as soon as an editor saves a file. I believe that good discipline combined with the fact that we usually only have one person at a time editing any files should prevent any problems.

2. The original creator of a shared space is the default Admin. That person may then designate additional Admins. I have designated Tim Davis as an Admin for the ATML shared space.

3. See above (1) regarding files. The same is true of the workspace.

There was also a question of file safety. The response from a Groove representative was that "...as long as one member of the shared space has the space, the files exist." There is also a facility to establish a Groove server, where a 'master copy' of the shared space(s) would live. This is an expensive option, and probably only suitable for large enterprises.


WebEx Meeting 17th Dec

Attendees:Bill Mercer, David Barrington, Ron Salley of SSAI; Bob McGarvey of AAI; Mike Seavey, NGC; Jeff Schwentner, LMIS; Steve Wegener, Boeing; Chris Gorringe, Racal; Tom Gaudette, MathWorks; Tim Davis & Bob Fox, NAVAIR; John Ralph, Aeroflex. Teresa Lopes was monitoring, but could not actively participate.

15:00 - 15:20 BST Review action items from prior meetingsPresenter: John Ralph

Groove Usage for future (10/12/03)The Groove eval licenses are expiring. Some questions needed answering before a recommendation is made to buy this:

1) Shared File Ownership2) Many Project Administrators3) Work space ownership

CURRENT ANSWERS FOR ABOVE.I have been searching the Groove FAQ and on-line forum. The answers I have so far are:

1. A shared space is exactly that: 'shared'. There is no true owner of the files, as everyone who is invited has a local copy of the files within the space. There is a facility for simultaneous edits of files, provided they are opened from the shared space within Groove. It appears, however, that if two users open the same file at the

same time WITHOUT using the co-edit feature, the last save overwrites all prior changes.

If a user is logged in and online, their local copy will be synchronized automatically as soon as an editor saves a file. I believe that good discipline combined with the fact that we usually only have one person at a time editing any files should prevent any problems.

2. The original creator of a shared space is the default Admin. That person may then designate additional Admins. I have designated Tim Davis as an Admin for the ATML shared space.

3. See above (1) regarding files. The same is true of the workspace.

There was also a question of file safety. The response from a Groove representative was that "...as long as one member of the shared space has the space, the files exist." There is also a facility to establish a Groove server, where a 'master copy' of the shared space(s) would live. This is an expensive option, and probably only suitable for large enterprises.

In my use of Groove, I find that it it easier to edit XML/XSD files outside of Groove by selecting the file or directory and selecting 'Copy to..' from the right-click menu. This is because double-clicking from the Groove file display creates a temporary file that is used for editing. Since the included files (for XSDs) don't exist in that temp location, XML Spy gives lots of errors. Once I finish making the changes, I drag-and-drop the modified file(s) back into the Groove window.

Use Case for Limits (10/12/03) OngoingSteve Wegener, had a teleconference, with NI over seperate test limits. This is not Test Results limits but seperate test limits for test programs.Action Steve Wegener

ATML needs to ensure that a solution for Test Program Limits can support both test programs with inline limits and limits separated from the test program .

Meeting #9 2004-01-20, Day 1 Minutes

Update on actions:

8.2, Style Guide, T. Lopes: posted outline draft version to Groove a/o 1/19/04. Continued work during telecons after this meeting.

8.3, Style Guide, LM(Steve): M. Bodkin indicated that review of existing style guide to be undertaken.

8.4, Style Guide, Tim: passed to T. Lopes

8.5, Standards, Ron: updated standards list, not yet posted. General request to membership for applicable standards in use by their organizations.

8.9, IEEE document access through NAVAIR, Tim: cannot access documents for general usage. CLOSED.

TestLimits separation use case; Steve (NI) to take action.

Best practices on versioning; Teresa; incorporated into draft style guide.****Review of schema release process. Suggestion from Tim (Boeing) that final (recommendation) should have an ATML-wide vote. Final process to be modified in accordance with larger body processes.

Suggestion made that a requirements document must be approved by a majority of chairmen (or some other defined group) prior to release for a sub-schema WG to proceed.

Review of status for TestResults in preparation for afternoon sub-WG. NI indicates they have several inputs forthcoming.

Review of agenda for Diagnostics sub-WG in afternoon.


Steve Wegener to chair the TestProgram working group

The question was posed to the group to understand what the group is looking to accomplish. What do you want to get out of the Test Program Working Group?

Reponses included:1) Test concepts 2) Looking at the division between the test results, instrument spec, test

requirements. Looking at the difference between test requirement and the test program.

3) How does test program relate to the ATE versus the UUT level. How to keep the specification generic enough to handle everyone.

4) Looking to understand what test program working group is looking to handle. 5) What to do with legacy TPS.6) Take legacy ATLAS and convert it to XML for new test stations.7) Test requirements are seen as definition and test program uses the same format.8) Trying to understand how to apply ATML to other test stations.9) Looking to understand how this can be used across different languages.10) Looking to understand the division between diagnostic working group and test

program working group.11) Insure the ability to go from XML to LabWindows CVI conversion.12) Re-sequence tests13) Do not see XML as a programming language, but XML has a place for helping in

translation form one language to another.14) How does signal definition fit into the test program workgroup and ATML.15) Looking to see how to have test programs go beyond the communication to the

test executive and grow to communicating to databases, remote locations, test stations, …

16) Looking to understand the what is needed for the test program not the how to run a test program.

Diagnostics Working Group

Mike Bodkin from Lockheed Martin presented in the diagnostic modeling working group. He presented a high level object model and asked the group if this was a valid UML diagram:

High Level Object ModelAreas we are not going to change:

Unit Under TestMaintenance ManagerHistorical Data Collector

Areas we are going to work on:Maintenance Manager AdaptorOperator InterfaceDecision SupportDiagnostic ReasonerTest ExecutiveHistorical Data Adaptor

The group agreed that it was a good approach to come up with a high level diagram and trying to understand what the picture is.

Comments on how the Operator Interface and the Test Executive needs to be combined to be a runtime system.

Timothy Wilmering from Boeing discussed what was done in AI-Estate1) Diagnostics (Reasoner or the Model. Can be analysis as well.)2) User Interface (X, Motif, Windows)3) Application Executive (The glue that controls the system. If you are not using

a Reasoner the fault tree can be a part of the Application Executive)4) Test Controller (Executes the test, collects the information and returns a result

to the Application Executive5) Info Management (Has knowledge about what tests have been run and what

tests passed and failed.) SIMICA. This can contain all of the defined ATML schemas as well as many other things. The Information Management that does not mean a database, but can be other things.

The group believed that the AI-ESTATE model shown by Tim could be used to fit everyone’s use cases. There still needs to be agreement on what is meant at each level but this will be part of the working group.

Some agreement about how the Test Controller is very high level should be implied when talking about the system. The Test Controller could be a LabView program, a DLL entrée point or the result of an operator visually inspecting a part.

Use Cases:The initial interfaces we need to support are:

Diagnostic reasoning as defined by 1232-2002Exchange of Information to support diagnostic reasoning

Exchange of Information to support diagnostic maturationDiagnostic Models (e.g. Fault Trees, Bayseian, Neural, etc.)

(Create XML version of the Express Fault Trees.)

It was noted that Test Program Working Group needs to allow for the ability to have a test that can be reused and run with input parameters being varied.

Discussion about how to handle callout coming back from a test and how to return this to the reasoner.

Action Items:

Tim W. Take on role of working group chair.Tim W. Send Mike Bodkin papers regarding maturation.Tim D. Look into adding on an additional weekly telecom for Diagnostics only. Find a time that’s good for as many people as possible, especially the chair.Mike Bodkin Research maturationTim W. Report to DMC the current activities and direction with regard to IEEE 1232-2002 and discuss creating XML schemas for all 1232 models.Group: Provide input to chair for organization into a set of use cases and the requirements document.Teresa Create Groove space for Diagnostics group and invite Tim W., Tim D., Tom G., Steve W., Bob M.


Wrap upTim Wilmering, chair of DiagnosticsSome agreement on AI-ESTATE model(s) against current use cases. XML versions of EXPRESS models to be investigated.Two use cases still need investigation.Suggestion to change weekly telecon to Thur. Bob F. to investigate.ACTIONSTim to send info to ??Add wkly telecon for diagnosticsBodkin research maturationGroup provide input to chair on use cases and requirements document

Tim Davis, TestProgramWG meeting mostly overview of user's needs. Tim to develop scope for group. Workspace to be established. Use case development next phase.

Bob Fox, AdminExamine IVI, SCC20, IEEE as candidate 'parent' organization. Investigation should be complete w/in 2-4 wks.Largest concern is how to post/publish schema namespace. Recommendation for parent to be delivered at next meeting.Purpose is to provide legal and other protections.Big concern over our need to publish schemas in an on-line, globally available location.

Discussion of next meeting location. Current decision is to meet in UK, third week in May, 2004. RACAL to host. Suggestion to hold interim working group meeting to move diagnostics along.Suggestion that an April meeting would work better. St. Louis was suggested (Boeing) and accepted for the week of April 19th. This pushes meeting after that to mid or late July in UK at Racal.

Next meeting agenda.1. Choice of standards group (parent for ATML)2. Continuation of diagnostics: use cases ready (maturation, run-time), draft rqmts doc ready for approval, begin preliminary schema work, 3. Present final documentation (requirements, style guide, etc.)4. Present TestResults use case from LM (Jeff Schwentner), NAVAIR (Leo Errico).5. Present something on TestProgram.====================================================TestProgram discussion 1/22/2004

Steve Wegener to serve as chairman of group.

Presentation of preliminary Mission Statement: "Develop an XML schema that supports descriptions of test programs."

DefinitionsTest Program: A collection of one or more tests, supporting information and optional sequencing capability. It is used to verify or diagnose the health of a UUT.

Test (n): A set of stimuli, either applied or known, combined with a set of observed responses and criteria for comparing these responses to a known standard. (from AI-ESTATE)

Suggestion that use cases be skipped and proceed directly to requirements. Ion felt that use case necessary to develop proper detailed rqmts.

TOP LEVEL REQ:Provide structure to capture information necessary to represent a test program independent of a specific test system language.(Must be able to ingest/represent existing test system languages while still being language independent.)

Must represent/describe a fault tree.

USE CASESRepresent ATLAS in TestProgram (Steve W.)"Text executive" must be able to execute test directly from the TestProgram instance document. (i.e., enough information in TestProgram to lead the executive to all required pieces of data to run a test). This includes reference or linkage to external objects.

TestProgram instance document will describe WHAT a test does, not HOW it does it. Put another way, TestProgram instance will contain the data for execution of a test, and the processes germane to the "physics" of the test, but will be independent of the process and syntax imposed by the implementation language/system (i.e. implementation independent).

TestProgram interoperability with an external "reasoning engine" for diagnostics. That is, test selection and diagnosis functionality is handled by an independent reasoner with mediation via a "test executive" (client/server or component relationship)(support reasoning scenarios detailed in IEEE1232).

Should be able to specify in/out parameters that specialize instances of a test.

Test instances within TestProgram must be able to support references to external test procedures or other external software (code?). No limitation on how many Test instances may reference a single external procedure.

Schema definition must accommodate heirarchical test structures (a Test can be a collection of Tests - recursion).

Must provide for state transition definitions (pre- and post- conditions).

Must have structure in the schema to support automatic generation of Test Requirement Document (TRD).[This may eliminate the need for a separate TestRequirements schema.]

Provide for use of IEEE P1641 signal and measurement definitions.

Some mechanism to designate data elements as being "variable" (i.e. modifiable at run-time) with some notion of scope.

=====================================================Review of documentation and due dates.

Four text documents status (requirements framework, style guide, standards reference, ??)Value and Limit documentation - requirements due ASAP, ready for Candidate on 2/2/2004TestResults needs a requirements document. Input due from membership 2/6/2004. John Ralph and Ion Neag to coordinate and release by 2/13/2004.Diagnostics draft requirements document due next meeting (Tim Wilmering).TestProgram draft requirements due next meeting (Steve Wegener).Globally replace Tim Davis with "NAVAIR" in the Team Members column of the schedule.


Telecon Meeting 17 Jan 04PeopleTim Davis (NavAir)Jeff Schwentner, (LM STS or what was LMIS);Mike Bodkin (LM STS or what was LIMS);Mike Seavy (NGC)Chris Gorringe (Racal Instruments Group)Bob McGarvey (AAI)Tom Gaudette (MathWorks)Steve Wagener (Boeing)John Ralph (all spaces)Teresa Lopez (Teradyne)

Bob Fox (NavAir)

Matters ArrisingSeperate Test Limits (Steve Wegener)

Liase with NI and someone to address this at the Orlando meetingFace To face Agenda (Chris Gorringe)

See updated Orlando AgendaUse case For Limits (Steve Wegener)

Steve to bring to the meetingL-M prototype of Testresults (Mike Bodkin)

Review Status at Orlando MeetingNew attribute for RepairType (Steve O'Donnell)

Dynamic information to itemInstanceType, PowerOnCycle, last calibrartionDate stuff to be ready brfore the 19th jan

Review PreTestRepair (Steve Wagener)

Initiate Disscussion of Schema versioning (John Ralph)Options for versioning Schemas

Using the version tag in the schemaUtilise a seperate tag fro the schemaUsing Schema URL as version control

Not invalidating existing documents is paramount

It is absolutely imperative that existing XML documents should never be invalidated because of a change in ATML schema documents.

Need to capture this in Style Guide as a requirement and ensure our management documents highlight this.

Follow the best practice

AoBAdd An Area on Groove for master Minutes and Agenda (Chris)Agenda responsibility (Tim Davis)Add Agenda Item for Position of etesters Groove and ATML.org discussionLM STS (LIMS) meeting infrastucture

Network accessTwo RoomsSVGA Projectorsexpecting <40 people

Let Bob Know if need conference call to Bob Fox

Test Program WG 2004-01-21 Minutes

The question was posed to the group to understand what the group is looking to accomplish. What do you want to get out of the Test Program Working Group?

Reponses included:1) Test concepts 2) Looking at the division between the test results, instrument spec, test

requirements. Looking at the difference between test requirement and the test program.

3) How does test program relate to the ATE versus the UUT level. How to keep the specification generic enough to handle everyone.

4) Looking to understand what test program working group is looking to handle. 5) What to do with legacy TPS.6) Take legacy ATLAS and convert it to XML for new test stations.7) Test requirements are seen as definition and test program uses the same format.8) Trying to understand how to apply ATML to other test stations.9) Looking to understand how this can be used across different languages.10) Looking to understand the division between diagnostic working group and test

program working group.11) Insure the ability to go from XML to LabWindows CVI conversion.12) Re-sequence tests13) Do not see XML as a programming language, but XML has a place for helping in

translation form one language to another.14) How does signal definition fit into the test program workgroup and ATML.15) Looking to see how to have test programs go beyond the communication to the

test executive and grow to communicating to databases, remote locations, test stations, …

16) Looking to understand the what is needed for the test program not the how to run a test program.

Test Results WG 2004-01-21 Minutes

Discussion/recommendation to make the Limits element ‘optional’. ACCEPTED; change made.

Some discussion of adding additional enumerations to the testOutcome attribute. This based on National Instruments input that some TestResult instance documents may not be for an actual test, and this should be indicated in the testOutcome attribute. NO DECISION MADE.

TestGroupType also made ‘optional’ based on above.

Additional discussion revolved around the ItemInstanceType common schema. T. Lopes recommended an explicit distinction between hardware and software items. Current attributes are common to both. Split would make ItemInstanceType an abstract, add two further elements to that sub-schema for hardware/software distinction.

This change would do away with the need for a VersionGroup attribute group.

Additionally, the powerOnTime attribute has been changed to an xs:duration type to force a format, yet adequately deal with an elapsed time.

ATML Management WG 2004-01-21 Minutes

Action 8-ATML-1 Bob Fox

For the future all draft versions for non release should initiate with a ‘0 ‘For Candidate Bob to report back on version items

Action 8-ATML-2 Bob Fox (SCC20) - CompletedTo approach the IEEESCC20 and IVI groups to identify the pro and cons of aligning the ATML imitative with those various groups

Les Orlidge provided the group with an update of how IEEE SCC20 operates

To be an IEEE standard:1) IEEE expect you to be an IEEE Ballot plus a member of Standard Association2) Format of the document has to meet IEEE style guide3) Can not put anything in that is proprietary or patented

If there is an outstanding negative ballot i.e. un-resolvable issue, need take it to the standard board

Consensus is defined as more than a significant majority. A ballot constituency is created where 75% of the balloting constituency must ballot.

Need to know the IEEE position on shared software components such as XML Schema that will be available on a Web site and published in an IEEE Standard.

Action 9-ATML-1 Les Orlidge

Action 8-ATML-3 Jeff Hulett (IVI)To approach the IEEESCC20 and IVI groups to identify the pro and cons of aligning the ATML imitative with those various groups

IVI have in place processes for dealing with software components, and distributing these components, through a web site, and installation.

IVI is a paid foundation; all companies must be a member to participate at the meetings. They currently inherited

VISA SCPI IVI Drivers

IVI have found it challenging in getting outside reviewing of their specifications, usually because companies have one vote the interesting parties/companies are generally on the working group.

IVI have a well defined policy on intellectual property or patented material.

IVI has a marketing committee to handle pushing the IVI standard forward.

Better definition of what or how ATML and IVI would co-exist and propose to the IVI as to whether they could accept ATML into the IVI group.

Action 9-ATML-2 Jeff Hulett

Provide Pro’s and Con’s for ATML being owned by the Test & Diagnostic Consortium (TDC) and pass to Bob

Action 9-ATML-3 Mike Malsich

Co-ordinate the Pro’s & Con’s for membership of one of these two standards organization gathering input from Action 9-ATML-1 to Action 9-ATML-3

Action 9-ATML-4 Bob Fox 28th Feb 2004

The Pro&Con document will be distributed to the ATML membership with a vote being taken closing date at the next meeting as to which Standards organization ATML want to join

Action 9-ATML-5 Bob Fox

Action 8-ATML-4 Chris Gorringe - CompleteThe issue number 1.01 does not follow 0.01 version numbering.

Action 8-ATML-4 - ContinuesWorking group issues will be collected by the Working group chair and consolidated by the ATML secretary in to a master issue log.Create an issues log document with sections for each working group

Action 9-ATML-6 Chris Gorringe

Action 8-ATML-5 - Steve Wegener In progressProvide SMART TPS use cases for DML

Action 8-ATML-6 Jeff Hulett - OngoingUse Cases need to be broken down in to identify the impact on individual working groups, and schemas. Responsibility falls to the groups to show that the requirements are met.

Action 8-ATML-7 Steve Wegener - OngoingCommon components are currently individual files where the document hold list of the common types.

Action 8-ATML-8 Chris Gorringe - CompletedConsider the document style and come up with a common format for the next meeting

Review of ATML Documents1. ATML Document (Chris)

Mission Statement (2 sentence)ScopeSystem Framework

ScheduleRelease processList of Documents with synposisChairman of each Document

2. Glossary (Chris)3. ATML Style Guide (Teresa)

Process for VersioningTemplate Documentation for Schemas (Tesesa)

3. ATML Requirement and Use Cases (Jeff Hulett)4. Survey Document (Ron Taylor)5. Common Components (Steve Wagner)6. Schemas (Working Groups)7. Action/Issues Spread Sheet (Chris)

Update ATML documents of above list and publish on Groove the following documents ATML Document.doc ATML Requirement and Use Cases.doc L Style Guide.doc

Action 9-ATML-7 Chris Gorringe


Low attendance.

Short review of action items from last several meetings.

Answer to concern over publishing XML namespace: according to John Sheppard of SCC20, the IEEE has only minimal requirements prior to publishing an XML namespace from their Web site. This answers the question raised during meeting #9 regarding these legalities should ATML join IEEE SCC20.


ATLAS parser discussion

Some consensus (initiated by Peter Richardson - BAE) that the Test ResultsSchema is far too complex - Peter having trouble generating instancedocuments programmatically. Specification and requirements need to be betterdocumented - general agreement. Requirements docs should be high level - adesign or other specification document can clarify element definitions andrelationships, and even traceability to requirements and use cases.

Tim D and Tim W reported on the ATML-DMC interactions at the recent DMCmeeting. Tim D, John Ralph, and Ion Neag participated and contributedgreatly to DMC discussions re XML representations of 1232-2002 models. TimD and John R. are working initial cuts at model schemas and Tim W willprovide input as to correctness etc - Ion Neag was assigned action to createFault Tree model.

SCC20 may be meeting in April - is there interest in arranging coincident

meeting with ATML in St. Louis?

Test Program - requirements? Use SCC20 standard (ATLAS or TD standard?) -appears to mean the old ATLAS standard...need to model the data separatedfrom the process - this could support use of XML data files at runtime bytest program applications.


Mike SeaveyJohn RalphBob FoxTim DavisIon NeagTom GaudetteSteve WegenerNuria HeviaCesar FernandezJose Manuel GonzalezMatt CornishJeff SchwenterSteve O'DonnellMike BodkinMichelle HarrisLeo ErricoTeresa LopesAnand JainJeff HulettRengan Rajendran

Topic 1: Clear up any confusion about what ATML is trying to do.Develop schemas to express use cases. One aspect is to create an interface between systems/users. Standard format for some things, like RESULTS.

TestRequirements and TestProgram possible issue as some confusion over whether ATML is trying to create a new language. WE ARE NOT. ATML is not a language, just a way to express requirements and programs in a format that can be transferred or translated between languages.

Little benefit in just capturing an existing language syntax into an XML structure. The ATML instance document(s) should be language-independent, providing a STRUCTURE within which requirements and strategies can be expressed. This will permit a single statement of "what to test" to be used by any system to generate code instructing a system "how to test".

NOTE: We should stop using acronyms to avoid confusion over which schema we are referring to.

Possible that group is too tightly focused on implementation. Must back away from implementation and begin looking at the bigger picture of defining the purpose of a test. The immediate goal is to enable diagnostic interface, but we need to recognize that acceptance of ATML will require the big picture view, not just focusing on the immediate goal.

Best value will come from supplying a TestRequirement along with a TestProgram and InstrumentSpecification. There will be linkage from the TestProgram back to the TestRequirement (accuracies) that drove the measurement specified in the TestProgram, and linkage to the Instrument where capabilities are specified.

A concern was expressed that "program-driven" development may be too implementation specific for wider ATML use. However, schemas developed by a member in response to their program/project may be a valid starting point from which to generalize for wider ATML usage.

DECISION MADE: TestProgram will be distinct and separate from TestRequirements. Establish linkage mechanism between TestProgram, Diagnostics and TestResults.

SEE ACTION ITEMS TABLockheed volunteered to prototype Steve Wegener's TestProgram schema.

Ion expressed need for an requirements for what the TestRequirement and TestProgram designs are intended to accomplish.

From Steve W.1. Interoperability with Diagnostics and TestProgram. (some of this has already been captured.)

PLEASE REVIEW AND COMMENT ON FOLLOWING USE CASE DIAGRAM


Participants:Dave RifkinJeff SchwentnerMichelle HarrisSteve O'DonnellAnand JainTim DavisMike SeaveyBob McGarveyMatt CornishTim WilmeringJohn GomesJose FernandezTeresa LopesJohn RalphIon NeagLeo ErricoSteve WegenerTom Gaudette

---------Review of proposed ValueType changes. See Files area for details. Proposed changes will not result in a "final" document - additional evaluation still required. Comments on proposal due by 11 March 2004.-----------Discussion of user issues with TestResults schema. Pete Richardson attempted to use schema, but found little documentation. This made initial usage a bit of a problem. Familiarity over time increased usefulness. Suggestion made that more complete documentation be developed, using a format similar to IEEE specification documents. John Ralph to draft proposed format for approval by Admin group.----------Discussion moved to TestProgram. Steve Wegener's posted document (TestProgram Development.doc.doc ) outlines development approach for TestProgram.

Three phase process.

Initially submitted use cases mapped to these phases. If all Phase 1 cases are in, group will move to requirements generation. Tim Wilmering suggested Fault Tree belonged in Phase 2; Steve agreed to move it.

Group to review and provide initial comments on document next week (4 March 2004).

Additional use cases solicited; due prior to 4 March 2004 telecon. These will be refined, and requirements generated prior to April meeting in St. Louis.


Attendees:

John Ralph - AeroflexBob McGarvey - AAITim Wilmering - BoeingMike Seavey - NGCJeff Schwentner - LM

Michelle Harris - LMSteve O'Donnell - LMTracy McQuillen - ARCJoe Colson - ARCSteve Wegener - BoeingJose Manuel Gonzalez - INDRANuria Hevia - INDRAEva Fernandez - INDRACesar Fernandez - INDRATim Davis - NAVAIRTom Gaudette - MathWorksMatt Cornish - RacalChris Gorringe - RacalDave RifkinLeo Errico - NAVAIR

Classified information discussion.Use case from Wegener: no classified information is ever passed from the ATE.Suggestion made that it might be better to push the responsibility for protection of classified information to the user. No specific classification identifier will be included in the schema(s).=================================================================

ValueType discussion.Leo Errico provided initial feedback that change had no impact on his instance documents.

=================================================================

TestProgram development document discussion

Mike Seavey has suggested that no distinction is needed between upper and lower level external schemas, thus merging Phase 1 and Phase 2.

Leo Errico brought up his use case of needing to external to a VISA controlled piece of equipment. Is there a provision for capturing the VISA command strings in TestProgram?Steve W. states that it is preferred to have only a high-level interface to an external command set (VISA, SCPI, whatever). The schema should be flexible enough to permit inclusion of specific external command strings within a structure that indicates their applicability.

Some discussion of configuration control of schemas after publishing. If any change is made by a user, they will be responsible for their own schema. If an extension is used for generating instance documents, these documents must still validate against the published schemas to be considered "transportable".

Ideally, the user would include retain the '"http://www.atml.org/xxxx/...." link and add an additional "xmlns:[??]=[some URI]" . Their extensions would then be indicated by "[??]:[element/type]" tags.

-------

Leo Errico pointed out that some of the use cases in the document imply a requirement on the instance documents rather than the schema.-------Suggestion (Chris G.) that para 4.1.2 needs to be reworded to better reflect the "real" purpose of the TestProgram instance document. This should reflect that there is still some 'test executive' that interprets the instance document to determine the specific system commands to accomplish the test.So, the TestProgram instance document is an instruction set (or rule set) that gets "animated" by some "actor" (i.e. the test executive).-------Emphasized that additional use cases are needed for TestProgram.Steve Wegener to publish current TestProgram schema as a straw-man.

Respectfully submitted...etc.


Present:T. DavisT. WilmeringT. GaudetteT. LopesS. WegenerJ. SchwentnerJ. GomesD. RifkinM. SeaveyI. NeagN. HeviaC. FernandezM. GonzalesR. TaylorJ. ColsonT. McQuillen

S. Wegener posted a file to eTesters and sent to J. Ralph for posting to Groove. It is a zipfile containing: .xsd, .tpml implementation, ATLAS Source, and a PowerPoint slide.

LM STS may have a TPML use case for submittal by next week.

M. Seavey has several use cases also to be submitted by next week.

Indra has posted comments on the Test Program Development Document to Groove as well as a use case diagram for TestProgram.

J. Gomes also commented on the Test program development document, mainly having to do with wording. J. Ralph posted to Groove.

S. Wegener went through his sample: the concept of how the blocks fit together (in the .ppt), then how the ATLAS got translated into their TPML implementation.

S. Wegener will post a document which will show how their implementation of ATML for RTCASS uses the various components of ATML.


Attendees:John Gomes, NAVAIRTeresa Lopes, TeradyneJohn Ralph, AeroflexDave Rifkin, M&TMike Seavey, NorthropAnand Jain, NI

INDRA:M. GonzalesN. HeviaC. FernandezM. Gonzales

Steve Wegener, BoeingTim Wilmering, Boeing

Lockheed Martin:Mike BodkinMichelle HarrisSteve O'DonnellJeff Schwentner

Bob McGarvey, AAITom Gaudette, MathWorks------------------Discussion of INDRA's submitted Use Case document (in response to S. Wegener's Test Program Development document).Question from T. Wilmering regarding the difference between a UUT Fault Tree and a Test Sequence. Concensus seems to be that a Test Sequence is dynamic, while a Fault Tree is static. INDRA's position is that a fault tree is more of a failure modes and safety/hazard analysis (thus static).

Group to review the INDRA document and post comments before next week.----------------ValueType change proposal discussed. T. Wilmering initiated discussion of the distinction between "Sets" and "Lists". This arises from AI-ESTATE. Sets are unordered groups, Lists are ordered. J. Ralph pointed out the use of "Collection" and "Array" in the ValueType meets this distinction.----------------Discussion moved to the issue of classification. S. Wegener discussed method used at Boeing to determine what data is classified. They use a stand-alone station with a classified removable hard drive when doing classified tests. Suggestion is to meet the intent of the executive order on classified data by adding a classificaion attribute to all of the schemas. This will be optional with a default value of "UNCLASSIFIED". This will be mostly informational only, not truly "binding" in a security sense. M. Seavey agreed with S. Wegener's suggestion that only enumeration will be "CLASSIFIED" or "UNCLASSIFIED".

M. Seavey requested that NAVAIR give approval to this suggestion prior to implementation.-----------------

T. Wilmering requested that membership review DIAGNOSTICS requirements posted in the Diagnostics space. ATML Diagnostic Schema Requirements Doc.doc ----------------Group actions for next meeting:Membership to review Mike Seavey's Test Program document (TPML_concepts_scenarios_interfaces.doc )Membership to review INDRA's Test Program use case document (Comments to Development Document from Indra.doc )Membership to review T. Wilmerings diagnostics requirements document (ATML Diagnostic Schema Requirements Doc.doc )


Participants:Tom GaudetteTim WilmeringJohn RalphTeresa LopesIon NeagAnand JainDave RifkinMike Bodkin

Cesar FernandezNuria HeviaRon Taylor

Steve WegenerJeffSteveMichelleVernon Woodward

--------------------Discussion of INDRA comment document.Reference to UUT Fault Tree may be removed. Seems to be not needed as it was oriented towards a safety fault tree, not diagnotics.

--------------------Discussion of Test RequirementsOverlap seen between test program and test requirements. Current documentation doesn't clearly distinguish between the two, and in fact there is some hint that Test Program may drive Test Requirements. This seems backwards and must be discussed/resolved.INDRA use case did not include Test Requirements. Since there will be some overlap between the two, there may need to be a "common" schema to contain this overlap, with inclusion as needed in Program and Requirement.

INDRA feels that nothing specific to an ATE should be in the Test Program document.

Steve W. has an issue in that some requirements are structured such that there is no distinct line between the requirements document and the test program. There are also cases where the Requirement is so decoupled from the Test Program, that there is no clear linkage - complex implementation may be necessary to fulfill the rqmt.

Anand J. indicated that NI has customers take an ID/IDREF approach to provide the linkage between Rqmts and Program.

TYX also provides a format to permit expression of specific "program" elements within a requirements document.

Bottom line is that there must be some mechanism to link any decoupled data elements - perhaps xPath.

Steve W. expressed thought that UUT requirements are not as critical at this point in the process. Will be more important when we move to new UUT testing.

Suggestion from Tim W. that an "open issues" list be created to permit capture of ideas and questions. John R. to post Excel spreadsheet in Files area to permit users to contribute. Suggestion that information capture include contributor, date submitted, issue, status, resolution, date resolved.

--------------------Discussion of diagnostics.Agreement among participants that requirements document should be kept as generic and high-level as possible.

Bodkin to release documents related to diagnostics. Use cases and class diagrams.


Attendees

Tim Davis (Navy)Mike Seavey (NGC)Anand Jain (NI)Ion Neag (Tyx)Tim Wilmering (Boeing)Hael Thibeault (Boeing)Matt Cornish (Racal)Ron Taylor (BAE)Teresa Lopes (Teradyne)Dave Rifkin (M&T)Cesar Fernandez (Indra)Eva Hernandez (Indra)Jose Manuel (Indra)John Ralph (Independent Contractor)Vern Woodward (LMSTS)Jeff Schwentner (LMSTS)Mike Bodkin (LMSTS)Michele Harris (LMSTS)

Discussion of M. Bodkin Use Case Diagram for Test ProgramLMSTS submitted documents (Use case diagrams, etc.) for consideration by the group at large. Reserve time at the April meeting for discussion (2 hrs.). The use cases were derived from the Indra post from several weeks ago.

Possible problem of including sequence information in a Test Program document. T. Wilmering suggested a review of IEEE Std. 1232 could resolve some of these

questions - the EXPRESS model in that document is very similar to both the INDRA and Lockheed use case diagrams.

We'll make it the goal of the April meeting to finish discussion of the problem domain in order to generate the necessary artifacts, post-meeting.

Matt Cornish suggested a review of IEEE P1641 might be in order, as there is some similarity to the Test Program problem space. Matt assigned action to develop short presentation of appropriate information.

Ron T. asked about the IEEE P1598 (TeRM Spec). Tim W. suggested it was maybe a bit too abstract for our uses.

Test ResultsIon N. has done some experiments with TestResults and has some results to share.

Standards Body DiscussionAction items were assigned at the January, 2004 meeting regarding this. Mike Seavey to put together a pro/con list for ATML folding into IEEE/SCC 20.Ron Taylor to remind Jeff Hulett about IVI pro/con presentation.

Diagnostics for St. Louis meetingDiagnostic use cases need to be expanded for presentation at next meeting. M. Bodkin to work on maturation use case.A presentation will be made on the approach taken to convert the IEEE 121 EXPRESS schema in XML.

Other Planning for St. LouisDecision to be made as early as possible on first day of April, 2004 meeting on whether Diagnostics and Test Program should meet in parallel. Probably post-1232 presentation.


Attendees:John RalphTim DavisSteve WegenerTeresa LopesTom GaudetteSteve O'DonnellMike BodkinJeff SchwentnerMichelle HarrisRon TaylorTim WilmeringJohn GomesDave Rifkin

Brief discussion of Groove usage by NAVAIR personnel. There is still an issue with the IT department there.

Quick review of outstanding action items since last WG meeting; only open items are Submit Additional Use Cases for Test Results, draft Schema Specification Document.

Reviewed the current proposed agenda for the St. Louis meeting; another request for agenda items was made.

As there was no new business, telecon was ended.


3-15-04

Tim DavisBob FoxLeo ErricoBob McGarveyDave RifkinJeff SchwentnerSteve O'DonnellMichele HarrisMike BodkinTim WilmeringTom GaudetteIndraTeresa LopesMike SeaveySteve Wegener

Tim W. talked to Mukind Modi Re: working DTIF into the ATML. No one else has heard from him yet.

Still awaiting several people to send in security info. Will forward to security by lunchtime today (4/15).

Steve W. proposes a working "box lunch" on Wednesday as well as a 30-40 minute tour of Boeing space/air stuff @ 1pm.

Request from M. Bodkin to handle Diagnostics breakout on Tues. afternoon or Wed. morning. This can likely be accommodated.

Approx. 40-50 people are planning to attend next week's meeting.

Plan to do presentations Tues. AM (show up at 8, plan on starting at 8:30), break into groups Tues. PM, Reconvene briefly together Wed. AM, then rebreak for the remainder of Wed. Then, recap everything Thursday and set the plan for the future.


IEEE P1641 review/presentation (Matt Cornish)A review of the IEEE P1641 standard was given with specific emphasis on the standards XML representation of creating signal definitions from its basic building blocks and capturing these in libraries (TSF) for reuse and interchange.

Pro/con presentation on standards bodies (Bob Fox)The presentation provided the Pro’s and Con’s for the IEEE, IVI and TMC standards organisation. The general consensus of the meeting was that IEEE would be the best home for ATML.

Action ATML-10.1 Bob Fox - Start the process (of generating a PAR) to move ATML over to IEEE

Presentation: Structured Test Definition & Development (David Poole)This presentation will cover the design of structured tests in XML,using model-driven component based architectures, and the experiencegained from their deployment. It will include a real-time demonstrationof a typical system, and will offer the design and architecture for useby the ATML committee in the formulation of their standards.

Short presentation on current Diagnostics approach (IEEE 1232) (John Ralph)Review of ATML Diagnostics with respect to SCC0 DMC. Take AI-ESTATE (IEEE 1232) and transfer into ATML XML schema

What’s Available Today?Preliminary Common Element Model ReadyPreliminary Fault Tree Model readyWill be presented next week at DMC meeting

Applicability to ATMLPortions of the schemas may be directly usable by ATML Diagnostics groupSchemas will be posted to Diagnostics space as ready

Discussion of the STEP already having an EXPRESS to XML mapping using DTC rather than Schemas as part of a published Part-28

Review problem domain based on M. Bodkin Use Case (Mike Bodkin)Presentation of several scenarios envisaged as use Cases for Diagnostics Runtime Environment

o Receive Work ordero Perform Maintenance Actiono Retrieve Bito Execute Testo Execute repair Procedureo Present recommendations

Non-Recurring Engineeringo Characterise Unit Under Test Typeo Design test Stationo Define Initial test Strategyo Implement test Strategy

Sustained Engineering Environment

o Collect Datao Generate Diagnostic reasoningo Reduced Lifecycle Costso Perform Knowledge Discovery and Data Mining

The Diagnostics ATML should concentrate primarily on the interface between the reasoner and the test executive rather than the implementation of the reasoner.

Tim Wilmering pointed out that the three inputs of Test Results, observations and historic data are in the diagnostic reasoning community collectively called tests

P1636 Standard for the… (Ask Tim)

Discuss collapse of Groove spaces into single space (John Ralph)If no one has any comments John will combine all the work spaces into the one ATML space

Action 10.2 - John Ralph Combine all spaces into one And Add separate discussion tabs for each of the working groups

Results of Test Results implementation (Ion Neag)Review of implementation of TestResults based on experience of incorporating into ATLAS tool set.

This activity generated several issues need to discuss with the test results group.

Working Group SummariesDiscuss TestProgram Development document (Steve Wagener)Do more Requirements and use Cases at the meeting and nail down requirements.Lock down use cases and generate requirements.

Test Results (John Ralph)Review of Ion Neag’s experience with implementing the TestResults output format. Several issues and questions arose. Recommendations were made on structural changes to the schema. Additional changes suggested by Anand Jain to improve usability. These changes are significant enough to warrant a major revision of the schema. Summary of changes will be in the TestResults revision history document.

Additional input from Peter Richardson regarding the need for a revision history. Since we are not making use of a version control system, it was suggested that a R.H. document be generated and posted to the Groove space.

Convened the TestProgram and ATML Management subgroups

TestProgram contained much discussion which is summarized here TestProgramMinutes.doc

ATML Management summary is found here Management Status and Schedule.ppt


Diagnostics Working Group MinutesFollow this link 2004-04-21 Diagnostic WG Minutes.doc

Date of next meeting The next meeting will be in approximately 3 months (Last week of July Week 12th or Week 19th)Location options are

Ferndown (Racal)Northrop Grumman (Chicago)LakehurstBoston (teradyne)

Everyone to secure funding for UK visit and make sure they can goBackup to be Northrop (Chicago)


John RalphChris GorringeTom GaudetteMike SeaveyTamara EinspanjerTeresa LopesIon NeagBob FoxTim DavisLeo ErricoAnand JainJeff SchwentnerMichele HarrisRon TaylorDave RifkinTony AlwardtAngela NielsonBob McGarveyJosé Manuel GonzálesEva Fernandez

also see Participants tab

MEETING 11 PLANNINGSurvey of the group to determine if a quorum was able to attend the UK meeting. Seems that we are GO for UK.

DIAGNOSTICS REVIEWDiscussion of presentation made at DMC meeting 28 April. DMC liked the approach being taken so far. There was no discussion at the DMC meeting of next steps in

developing the AI-ESTATE schema. B. Fox is concerned that the AI-ESTATE schema work may not be completed in a reasonable amount of time (i.e. in step with ATML's schedule). We suggested that Fault Tree Model needed to be worked further. I. Neag agreed that FTM would be completed. B. Fox would like ATML to map services from the ATML perspective that treat diagnostic reasoners as 'black box' devices. DMC had a concern about ATML use of the term 'Indictment'. B. Fox to discuss with L. Orledge.

ATML needs to continue being involved with DMC. DMC may change their schedule to meet concurrently with ATML. Fox to contact cognizant people on DMC to work out details. ATML mtg with DMC needed prior to August (DMC's current plan).

TEST PROGRAM RQMTS STATUSMuch input, good ideas being submitted, lots of common points. No consolidation as yet, but B. Fox to pull together and publish next week. First-cut deadline now passed.

TestProgram steering group to meet on 18 May concurrent with CTI (Chicago/Northrop). Fox suggested that this would be the planning session to set up monthly meetings of this group in order to push progress along. This needs to be a 'core' group that is doing the "hard" work.

I. Neag suggested a central place on Groove for all of these initial WhitePapers and to refrain from commenting on the use cases until the May 18th meeting. We'll store them in Working Groups/TestProgram/Requirements & Use Case Documents so B. Fox can begin to combine them.

Mike Seavey to publish a preliminary agenda for the Chicago meeting. Plan is to meet 18-19 May. Currently planned attendees are: Fox, Davis, Seavey, Jain, O'Donnell (or some LM person), Neag, Ralph, Wegener. Racal/UK unlikely to attend.

INSTRUMENTSNothing ready as yet. Instrument use cases requested to be submitted to Teresa Lopes by 1 June. Northrop, Teradyne, Racal, BAE, Aeroflex plan to submit formal use cases. Others welcome. These will be compared against the current draft schema to ensure coverage. Lopes suggested a 'capabilities' document be created by each contributor.

SCHEDULEFox to post current schedule on Groove. Will be there tonight.

Next meeting confirmed to be in UK July 20-22, 2004


PARTICIPANTSJohn RalphTeresa LopesTony AlwardtAngela NielsonTim DavisDave RifkinMike SeaveyTim WilmeringJeff SchwentnerJim Babb

Steve WegenerAnand JainPeter RichardsonTim WilmeringTamra EinsingerMatt CornishINDRA (sorry, I didn't catch the name)

DISCUSSIONIssue of null strings arose this week. Several 'name' and 'ID' attributes were defined as xs:string. This permits null values, which doesn't make sense for either ID or name. It has been proposed that a new type be defined that will be of minimum length 1. This will be defined in Common and will be called 'String'. The TestResults schema will be modified to make use of this new type.

Mike Seavey requested a 'show of hands' for attendance at next week's meeting. Looks as though 8 people will attend.

Discussion of use of Groove. There is a need for a listserv functionality separate from etesters. John Ralph to investigate how Groove can handle this for non-Groove users. After ATML becomes part of IEEE, there will be access to a Web server and a listserv for group e-mails. It was suggested that the group should consider continuing use of Groove as a collaboration tool.

A reminder was issued that Instrument use cases and requirements are requested.

No further business, so meeting was adjourned.

2004-05-18 Test Program WG Minutes

ATTENDEESBob FoxMike SeaveyJohn RalphAnand JainIon NeagBob Noble

DISCUSSIONThe major points of this mornings discussions have centered around what ATML actually wants TestProgram to be. The concensus is that a mere recasting of ATLAS is not the right approach.

The most often recurring theme in the submitted documents is a need for a "description" of the test - not an ATE-specific programming language. This description should describe HOW to test a UUT. This would include some free-form narrative as well as specific placeholders tocontain information related to the "goes-intos" and "comes-outas" for the UUT such as voltage levels, input signals, etc.

Suggestion is made that we rename this section to TestDescription. This will avoid the semantic baggage that goes along with the TestProgram name.

The TD must permit a hierarchical structure. Granularity down to the "atomic", or smallest element that can generate an outcome. An outcome is something that can be compared to a limit for a quantitative evaluation.

There must be a capability to specify the flow or order of atomics (sequencing). Thus, atomics must be uniquely identifiable.

Atomics should be groupable into "mini-sequences". The thought is that in the TD file, all atomics can be defined with no implied sequence. Another section of the document will provide the sequence.

Only some atomics will be test entry points. This implies that we must be able to specify the predecessor atomic for any given atomic. Atomics will be group-able in the hierarchy.

The eventual schema must provide for prose documentation.

Any flow in the structure of the atomics must be flexible. Perhaps an external reasoner (or other executive) would need to execute the atomics in another order.

The structure should also permit a sequence to be built as a fault tree. The schema structure should permit an external program to resequence the atomics as needed.

HOLD THIS THOUGHT: there is a need to hold intermediary results data that might be an input to the next test in a sequence.

There will also probably be a need for user-modifiable input parameters for the test.

DAY 2Telecon participants today:Chris GorringeMatt CornishCesar FernandezNuria HeviaJose PascualTeresa Lopes

Attending:Bob FoxIon NeagBob NobleAnand JainSteve WegenerSteve O'DonnellJohn RalphTim DavisMike SeaveyTamara Einspanjer

Racal (Chris) expressed concern redgarding the idea of 'variables'. There was a worry that this started us down the path of defining a language. The group DOES NOT want this to be the case. We are not defining a language. The thought was more that the ML would permit the declaration of data elements that are "outputs" from a test.

INDRA explained their comments to yesterday's minutes (Indra. Comments to Meeting Minutes 18May2004.doc ). Their diagram looks like a fault tree. They stated that the 'tree' doesn't have to be balanced(?).

Further explanation of the concept of "atomic". It is worth noting that an atomic may not actually have an outcome - for instance, "power on" or "power off".

There was some discussion of what we mean by 'state'. It may not be necessary to have an explicit state definition - the state of an atomic is implied by its inputs.

In response to INDRA's question on atomic inputs - we did not intend to imply that the system operator would be modifying parameters. The real intent was that the test developer could specify the parameters.

Discussion of INDRA comment 13 on templates. The thought is that an atomic with input parameters may be a test template, with input values specified at the time of instantiation. Alternatively, the atomic may have internally-defined parameters that can't be changed at instantiation. In the second case, the atomic isn't really a template.

Extensive discussion of the difference between "test group" and "test sequence" in the TestDescription schema. After exploring the difference and comparing what is meant by "Test Group" in the TestResults schema, it was agreed that TestDescription will not use the term "test group".

For convenience, the smallest component (previously "atomic") will be called "test". A group of tests will be called TestSequence. Each TestSequence will have a unique identifier.

Test will have parameters that are accessible at the TestSequence level.

Both a Test and a TestSequence will have outcome/qualifier pairs defined. Scope of outcome/qualifier pairs will be determined based on where they are defined in the instance document. The XSD definition of these will probably be the same.

Test will also have outputs (not outcomes).

Each outcome/qualifier pair will point to a target. This may mean that outcome/qualifier/target "sets" may be unique and have a UID. That target may be another Test, another TestSequence or an outcome/qualifier pair of the current sequence.

The schema will provide a way to specify possible outcome qualifiers in the "definition" section of an instance document.

The Test definition can include Limits. The structure of Limits will match what was defined in TR.

There was a discussion of how to use "indictment". No decision was made, but it was agreed that an outcome/qualifier pair probably has an associated indictment.

Some features of Test:UUT, name, uid, description, list of possible (or interesting) outcome/qualifier pairs, parameters, outputs, limits, indictment (in some format).

Began new discussion of what data elements will be part of the interface/interaction with a diagnostic system.


PARTICIPANTSTim Davis Jose Manuel GonzalesTim Wilmering Fernandez (INDRA)Leo Errico Jose Manuel GonzalesMike Seavey John GomesTeresa Lopes Tom GaudetteJohn Ralph Bob FoxBob McGarvey Chris GorringeJeff Schwentner Michelle HarrisSteve O'Donnell Dave RifkinJoe Stanco

DISCUSSIONBegan with a recap of the 18 May Test Program working group meeting. (See minutes for details of that meeting)

Some discussion of the boundary that defines what a "test" is. T. Wilmering stated that, to the maximum extent possible, a test should be "independent".

It was pointed out that a test is related to an outcome, and a UUT is related to components. Thus, there is an inherent relationship between outcomes and components.

It was pointed out that XML-ifying existing ATE languates e.g. ATLAS, was of marginal benefit. Merely "wrapping" existing code in an XML structure would still require a good deal of integration work in a re-host situation.

J. Ralph and T. Davis will extract requirements gathered at the 18 May meeting and post them for comment. Other members should be prepared to comment as soon as practical.

At the next telecon, the group will review the open action item list (per C. Gorringe). Each sub-group lead should review their actions to ensure status is properly updated.

DIAGNOSTIC GROUPNo activity since the Nashville meeting. The group will begin regular meetings again, as part of the usual Thursday telecon. General discussion will be limited to 1 hour, with interested parties remaining on-line for the Diagnostics WG meeting.

Mike Bodkin has begun some definition work on the services between ATML and diagnostics. This must be reviewed. Currently, participants seem to be J. Ralph, T. Gaudette, M. Bodkin, T. Wilmering, M. Seavey and T. Davis.

Meeting closed at 11 a.m.


Present:Steve WegnerTony AlwardtAngela NielsonJim BabbTim WilmeringDave RifkinLeo ErricoTim DavisBob FoxMukind ModiJoe StancoSteve PolonuiMike SeaveyTamara EinspanjerAnand JainIon NeagMichelle HarrisJeff SchwentnerSteve O'DonnellVern WoodwardTeresa LopesTom Gaudette

Discussions:ManagementSome discusssion on where M. Seavey posted the IEEE SCC 20 process.

Test DescriptionRequest for volunteers for TestDescription chair. M. Seavey volunteered. M. Seavey and B. Fox will discuss offline where this group should head.

InstrumentsReceived 6 use cases/requirements. T. Lopes posted a Word template and example document as a proposed template for documenting schemas and is soliciting input on it.

DiagnosticsJ. Ralph not present to discuss status of CEM approach. T. Alwardt and I. Neag posted use cases to Groove for comment (Action to working group). T. Wilmering will post a scenario for mapping the reasoner use case to AI-ESTATE services.


AttendeesTim DavisMike SeaveyTamara EinspanjerIon NeagJeff SchwentnerMichelle HarrisLeo Errico

Bob FoxMatt Cornish

ReviewShort discussion on IEEE template and our modification.

There was general agreement that separate specifications would be preferrable. In the future, after all specs are completed, they could be recombined into a single document.

Suggestion was made that including a bulleted list of attributes for each type/element at the beginning of description section would be useful.

Ion N. felt that we should not use XML Spy diagrams in the specification as the format is proprietary to Altova.

Discussion of the use of "ID" in Test Results. Uniqueness not required, but a better explanation should be in the document.

General feeling is that the purpose of schema spec is more to describe a schema's elements and their relationships to one another as well as to the "real world". Spec should deal with semantics of the schema. Should also provide enough detail to recreate most of the schema.

ACTION to all: Read and comment on Mike Seavey document for Test Description. Deadline for the requirements document is June 30.

Need a decision on use of eTesters site. Do we want a push or pull solution? Current feeling is to use eTesters only for broadcast e-mail, use Groove for all file posting/access. (Bob F.)

New policy: any user who gets flagged as not being active in Groove will be suspended. They will be re-invited upon request. (Tim D. suggested, all agreed)

INSTRUMENTEnough information has been submitted that a rqmt document can be ready for review at the July meeting. Bob F. to e-mail Teresa L. with a planned schedule. Must be discussed next week.

TEST RESULTSTYX has done extension testing and had good results. Everything is functional. Bob F. asked that Ion present results at July meeting.

DIAGNOSTICSIon N. has done some work with the AI-ESTATE CEM. Feeling is that the CEM XML schema should become part of the IEEE-1232 document; ATML will create schemas (as needed) drawing from the types defined in the CEM. Ion N. is of the opinion that any schemas related to diagnostic reasoner data exchange be consistently parented...i.e. either they belong to 1232 or they belong to ATML. Concensus seems to be that the diagnostics-related schemas should belong to 1232 and ATML will make use of them as needed. More discussion needed. ADD TO AGENDA FOR NEXT WEEK.

Ion: on fault tree work, changes were needed to CEM. Ion to set up a Webex meeting to go over the changes he made.

No further business; meeting adjourned.


AttendeesLeo ErricoTim WilmeringTim DavisTeresa LopesTom GaudetteIon NeagJeff SchwentnerMichelle HarrisTamara EinspanjerMike SeaveySteve PoloniMatt CornishJoe StancoMukund Modi

DiscussionInstrument report from T. Lopes. Draft document planned for release one week prior to July meeting. Full discussion will be at that meeting.

Test Description. T Davis submitted comments on Groove. L Errico e-mailed comments directly. M. Seavey to incorporate comments and publish draft document. Document planned to be a draft Standard for presentation at July meeting. Schema and standard to be worked at July mtg. T. Davis has begun a draft schema that can be used as a starting point. M. Seavey and T. Davis to work the draft schema.

Discussion of how to notify folks of new items. Some members were not aware of the Groove "new item" notification icon.

Use of term "Indictment". Some discussion of the difference between "indictment" and "diagnosis". Semantically, members decided that it was proper, in the context of Test Results, to continue using "indictment". I. Neag requested some additional time to research. Item will be posted to next week's agenda for further discussion.

New agenda items added for July meeting.

Current expectation is 20-30 attendees at July meeting. Twenty-one confirmed bookings @ local hotels.

Working group breakdown created by C. Gorringe. Posted to Groove & eTesters.

Webex for Diagnostic discussion to be moved to Monday, 28 Jun @ 10a.m. EDT.


AttendeesJohn RalphTim Wilmering

Mike SeaveyBob FoxTim DavisMatt Cornish

GeneralJ. Ralph to update e-mail lists based on new membership.

Test DesriptionSeavy developing draft Test Description schema. Should be posted for initial comments by next week. Requirements document will also be updated.J. Ralph to complete Diagnostic Common Element and forward that along with Fault Tree to M. Seavey. There may be interaction between Diagnostic and Test Description.

InstrumentT. Lopes not on, so no discussion.

Test ResultsAwaiting Lockheed Martin test case prior to making any changes. T. Davis questioned process for modifying if any definitions are moved to Common. Because of namespace definitions, any such move will invalidate existing instance documents. Test Results is still in "Candidate" status, so what is the process?

DiagnosticsQuick review of last week's meeting. See above Test Description notes.


Attendees

Teresa LopesJohn RalphBob FoxJohn GomesMatt CornishMike SeaveyJim HearnSteve PoloneyJim BabbEva FernandezIon NeagMichelle HarrisJeff SchwentnerTim WilmeringTom GaudetteJoe StancoLeo Errico

InstrumentIt appears that all use cases & requirements submitted so far are in general agreement. Plan is to review use cases & requirements prior to meeting. T. Lopes would like to have a straw-man schema ready for face-to-face.

Test DescriptionM. Seavey has consolidated and prepared good information. Should be ready for work at the face-to-face. Seavey has not had time to tie in Diagnostics, but there is recognition of an interface. There is good concensus on where Test Description is headed.

DiagnosticsB. Fox questioned how unit-level fault data will be pushed to ATML-compliant systems. No expectation that field-level fault data will be ATML format; only needs to be ATML format when that data crosses a system boundary into an ATML-compliant system. ATML will specify what information is desirable in the "next-level" system.


Attendees:John RalphMike SeaveyIon NeagAnand JainTim DavisDave RifkinMatt CornishJeff SchwentnerSteve O'DonnellBob FoxTom GaudetteJoe StancoMukund Modi

Discussion:Brief overview of travel logistics for next week's meeting. Racal has arranged a 17-seat shuttle from the Royal Bath to the Racal site. Shuttle will leave the Bath at 8:30a.m. promptly.

Steve O'Donnell to present Lockheed Martin test case of Test Results. There will be deltas, and they may impact schema design. Still tentative on whether presentation will be ready, but there may be enough information for an off-line discussion.