Audit Plan, Audit No. HQ-93-03, Audit of the CRWMS M&O

OFFICE OF CIVILIANRADIOACTIVE WASTE MANAGEMENT

U.S. DEPARTMENT OF ENERGYWASHINGTON. D.C.

PAGE 1 OF 12

AUDITISURVEILLANCE

NO. H-93-03

* Ayil I A a _ II , __ame---..

ORGANIZATION EVALUATED

M&O (Vienna, VA)

DATES OF EVALUATION

February 1 to 5, 1993

IPQ EXTERNAL

I I INTERNAL

I AUDIT

[ SURVEILLANCE PREPARED BY Robert L. Howard

9 Vl

nATe 1ms. o7 1QQ

CONTROLLING DOCUMENT (Title, Number, Revision) ACTIVITY EVALUATEDProlect Milestone Reviews, QAP-3-14, Revision 0 Project Milestone Reviews

REMARKSNEOM. CHARACTERISTICS TO BE EVALUATED Record objective evidence reviewed, method RESULTS

of verification.oersonnel contacted

1 . Determine what project milestone reviews were conducted bythe M&O Contractor.

INDICATE RESULTS: SATISFACTORY (SAT), UNSATISFACTORY (UNSAT), NOT APPLICABLE (N/A)

REV. 09/91

'' -"-7


U.S. DEPARTMENT OF ENERGYWASHINGTON, D.C.

SHEET 2 OF 12

AUDIT/SURVEILLANCE

NO. HQ-93-03

I A A A I

CHARACTERISTIC TO BE EVALUATED

Verify the Assistant General Manager assigns a Project MilestoneReview leader, determines the type and rigor of the reviewrequired, and documents this information on a Project MilestoneReview Notice, Attachment 1, and transmits this information tothe cognizant office manager. [5.3.21

REMARKSRecord objective evidence reviewed, methodof verification,personnel contacted

ITEMNO.

2.

1

RESULTS

REV. .9/91

' ' ,Z

I



SHEET 3 OF 12

AUDIT/SURVEILLANCE

NO. HQ-93-03 1IW A A At l fAl *1[I:M; t iI&

CHARACTERISTIC TO BE EVALUATEDREMARKSRecord objective evidence reviewed, methodof verification,personnel contacted

4 4 I

Verify that the Project Milestone Review Leader develops a reviewplan and that the plan documents the following:

A. The exact scope of the review, including thespecific system, structure, component, or itemsubject to the review.

B. Identifies all design output documents subject tothe review.

C. A determination of all disciplines that mightaffect or be affected by the system, structure, orcomponent, or item subject to the review.Consideration should be given to operations,maintenance, construction, radiological safety,environmental compliance, and materialsengineering disciplines.

D. Considering the complexity and state of the artof the design, establish Project Milestone ReviewTeam qualification requirements.

E. Consideration of design requirements documents,safety analysis documents, calculations,computer code and hardware documentation,background information supporting advanced orstate of the art engineering techniques, codes,standards, and interface control documents fromthe information provided by the designorganization [5.4.31

I _____________

ITEMNO.

3.

RRESULTS

REV. .9191

-1-/-

I



SHEET 4 OF 12

AUDITtSURVEILLANCE

NO. H-93-03

I


* 4* I.

Verify the review leader contacts the responsible designorganization and established a detailed schedule for the review andincluded the schedule in the review plan. 15.4.41

Verify the Assistant General Manager approves the plan. [5.4.51

ITEMNO.

4.

5.

RESULTS

REV. 9191



SHEET_ _ OF 12

AUDITtSURVEILLANCE

NO. HQ-93-03

I I I


Verify that reviewers' independence from the design organizationis documented on Attachment III, Reviewer Qualificationstatement. 15.4.61

Verify that the review team selection is documented onAttachment 11.


ITEMNO.

6.

7.

.RESULTS

& & 4-

REV. .9/91

,i IN

-

I



SHEET 6 OF 12

AUDITISURVEILLANCE

NO. Q-93-03 1A_

CHARACTERISTIC TO BE EVAWATED

Verify demonstrated competence of the reviewers in their assignedaspects of the review is documented. This documentation shouldreference academic degrees professional certifications andaffiliations, and summarize relevant experience. 15.4.6 Al (Submitlist of personnel to Team C.)

Verify the cognizant office manager approves review teammembers prior to executing the review. 5.4.71


ITEMNO.

8.

9.

RESULTS

4- 1 5

REV. .9/91

. .



SHEET 7 . OF 12-

AUDIT/SURVEILLANCE

NO. H-93-03


I I I

PREPARATION

Verify the review leader ensures the team is prepared for thereview. Preparation should include the following:

A. Team members are aware of the scope,schedule, and plan for the review and thetechnical requirements of the particular design.Copies of applicable requirements documentsmust be provided to team members.

B. Team members must be familiar with QAP-3-14.Note: this should be verified by the auditorcovering criterion 2.

C. Team members should be familiar with thestructure, system, component, or item designs.

D. Assignment of team members to design areasand preparation of checklists or instructions to beused for the review, as appropriate.

E. Checklists developed by review team members.[5.2.21

ITEMNO.

10.

RESULTS

REV. .9191

1 .. ' - -- - ,;



SHEET 8 . OF J20

AUDIT/SURVEILLANCE

NO. H-93-03

_91dI* I A *' g 1 11 111

_s | - , , I~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

CHARACTERISTIC TO BE EVAWATED

Verify the Project Milestone Review Leader reviews andapproves checklists or instructions developed by reviewteam members. [5.5.31


ITEMNO.

11.

1RESULTS

-& I

REV. .9/91

I



SHET 9 . OF 72

AUDITPSURVEILLANCE

NO. H-93-03

* ., , , I I


-t I

EXECUTION

Verify that the review team conducts the review in their assignedareas according to the review plan and the checklist or instructionsthat have been developed. 15.6.3]

Verify that the review team documents the review comments incomment resolutions in accordance with M&O QAP-3-1, TechnicalDocument Review. [5.6.5]

ITEMNO.

12.

13.

RESULTS

I I I

REV. .9191

''v'---

OFFICE O41VILIANRADIOACTIVE WASTE MANAGEMENT


SHEET 10 O 12

AUDIT/SURVEILLANCENO. H-33-03

I I I


I S I

REPORTING

Verify the review leader prepares a report of the review results.Verify the report includes the following, as appropriate:

A. Scope of the review, including specific systems,structures, components, and items.

B. Identity of the review team members and designorganization personnel contacted during thereview.

C. Summary of the results of the review.

D. Any significant problems encountered ordeficiencies identified and the resolutions,including corrective actions initiated.

E. Open issues and actions to be taken.

F. Recommendations. [5.7.1]

ITEMNO.

14.

RESULTS

REV. .9191

I I



SHEET 1 OF 12

AUDITISURVEILANCE

NO. HQ-93-03 1* £i£* - I


Verify the review leader signs the report andAssistant General Manager. 5.7.31

forwards it to the

Verify a package consisting of the review plan, checklists, orprocedures, comments, and resolution records, reviewerqualification records, and reports have been submitted to M&Orecords management system. [5.7.4] (Coordinate with Team D.)


ITEMNO.

15.

16.

RESULTS

4 4 .4

REV. .9/91

'

I

I-



SHEET_ 12 OF 12

AUDIT/SURVEILLANCE

NO. HQ-93-03

* A A A *~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


Procedure Adequacy:

Why is the use of Attachment I not specifically required in theprocess section of the procedure?


ITEMNO.

1RESULTS

-4 '4 .4

REV. .9/91

I



PAGE 1 OF 6

AUDIT/SURVEILLANCENO. HQ-93-03

II a .


M&O (Vienna, VA)

DATES OF EVALUATION


[)Q EXTERNAL

[ I INTERNAL

PI AUDIT

1 I SURVEILLANCE PREPARED BY I %AoAr% e-r%0r%%M r%ATr 1.... u7 1q

CONTROLLING DOCUMENT (Title, Number, Revision) ACTIVITY EVALUATEDProcurement Document Control, OAP-4-1, Revision 0 Procurement Document Control

REMARKSITEM CHARACTERISTICS TO BE EVALUATED Record objective evidence reviewed, method RESULTSNO. of verification.oersonnel contacted

1. Verify that the designated preparer (OAP-7-1) has prepared a draft Ac C 7 4Ie7 7occ S

technical procurement document and has referenced all applicable 7technical requirements, quality affecting classification and applicable c 3 -c r d L4 UJ 4

QA requirements. [5.21

* INDICATE RESULTS: SATISFACTORY (SAT), UNSATISFACTORY (UNSAT), NOT APPLICABLE (NIA)

REV. 09/91

%Jwn�%Jlv �1 �.Pmwxxv �'. Uwv."



SHEET 2 OF 6

AUDIT/SURVEILLANCE

NO. HQ-93-03

M . SW .'i LI ! . = *. . L1


Verify that for procurements important to safety and/or wasteisolation, the applicable supplier(s) have been advised (andannotated on procurement document) that 10 CFR Part 21 applies.If no purchase orders have been written, verify that there is aprocedure designating conditions that Part 21 will apply. [5.21

Verify that the preparer has had the entire procurement documentreviewed by the affected manager(s), as identified in theprocurement plan (OAP-7-1 ), to ensure that technical and qualityrequirements, that affect common interfaces, are addressed,completed, correct and meet the needs of the affected manager(s).Reviews shall be documented on one or more PDRR. 5.31


Q

Lovj I t'-0-

ITEMNO.

2.

3.

RESULTS

REV. .9/91

Y-DIER



SHEET 3 OF 6

AUDIT/SURVEILLANCE

NO. H-93-03

M l 111 il1111,1 1 iI I I I


Verify that the Department Manager has arranged for anindependent check of the procurement documents by a qualifiedReviewer for completeness, accuracy and the required technicalreviews, as specified in QAP-3-8, and that the independentreviewer signed the PDRR. [5.41

Verify that cognizant Department Manager has reviewed theprocurement document to ensure that applicable technical andquality assurance requirements are complete and correct andconcurred with the document on the PDRR. [5.41

REMARKSRecord objectiveevidence reviewed, methodof verification,personnel contacted

7ITEMNO.

4.

5.

REV. .9/91

, '



SHEET 4 OF 6

AUDIT/SURVEILLANCE

NO. HQ-93-03

* A A .1 * S -.~~~~~~~~~~~~


Verify that the System QA Manager has reviewed the QArequirements for completeness and accuracy and has concurred bysigning the PDRR. [5.51

Verify that the Systems QA Manager has resolved problems, ifany, with the preparer and/or Department Manager. 15.51


-I. I

ITEMNO.

6.

7.

RESULTS

REV. .9191

-



SHEET 5 OF eAUDIT/SURVEILLANCE

NO. H-93-03

M l A A 1 a S * 6i^ il ,


Verify that the Subcontracts/Purchasing Manager reviews theprocurement documents and concurs by signing the PDRR. 15.61

Verify that changes, if any, affecting technical or QA requirementsresulting from the SubcontractslPurchasing Manager review hasbeen documented on the PDRR and accepted by the DepartmentManger and Systems QA Manager. 15.6, Note]


ITEMNO.

8.

9.

RESULTS

REV. .9/91

I



SHEET 6 OF 6

AUDIT/SURVEILLANCE

NO. HQ-93-03

I I I I

CHARACTERISTIC TO BE EVAWATEDREMARKSRecord objective evidence reviewed, methodof verification,personnel contacted

4- I

Verify that once the procurement document has been acceptedthat the preparer has processed the document in accordance withQAP-7-1. [5.7]

Verify that changes to the procurement made after acceptance aredocumented as revisions to the procurement document andsubjected to the same level of review and documentation as theoriginal. 15.81

ITEMNO.

10.

11.

RESULTS

REV. .9/91

I

'� I



PAGE 1 OF 45

AUDITISURVEILLANCE

NO. HO-93-03

I- I i I


M&O (Vienna, VA)

DATES OF EVALUATION


[X] EXTERNAL

[ ]INTERNAL

[X] AUDIT

[ I SURVEILLANCE PREPARED BY LEONARD GORDON DATE January 28,199E

CONTROLLING DOCUMENT (Title, Number, Revision) ACTIVITY EVALUATEDControl of Purchased Items and Service, OAP-7-1, Revision 0 Control of Purchased Items and Service

REMARKS *ITEM CHARACTERISTICS TO BE EVALUATED Record objective evidence reviewed, method RESULTSNO. of verification, personnel contacted

1. Verify that the Department Manager has designated a Plan Preparer toprepare and document a Procurement Plan to ensure a systematicapproach to control each major task of the procurement process. [5.2.1]

' INDICATE RESULTS: SATISFACTORY (SAT), UNSATISFACTORY (UNSAT), NOT APPLICABLE (N/A)

REV. 09/91



SHEET 2 OF 45

AUDIT/SURVEILLANCE

NO. HQ-93-03

11.L a -i.;^.: i'. z t ;..1; *01 11 i


Verify that the Procurement Plan include the following eight criteria:[5.2.1]

A. What is to be accomplished by the supplier and theby the M&O;

B. Who is to accomplish it;

C. How it is to be accomplished;

D. When it is to be accomplished, including sequenceof actions and milestones;

E. Organizational responsibilities;

F. Use of a Source Selection Official and SourceEvaluation Board, (if applicable);

G. Technical and quality assurance requirements, (tobe refined further in applicable procurementdocuments);

H. Approach to be used for offerer qualification,proposal evaluation and selection of suppliers.

REMARKSRecord objective evidence reviewed, methodof verification, personnel contacted

ITEMNO.

2.

RESULTS

REV. .9/91



SHEET 3 OF 45

AUDIT/SURVEILLANCE

NO. H93-03

0 - 0 --


Verify that the Plan Preparer has evaluated the planned procurementsin accordance with QAP-2-3 and made a preliminary determinationwhether it is quality affecting. [5.2.2] If the procurement isdetermined to be quality affecting.

Verify that should the procurement be determined to be not quality-affecting, that it is noted in the plan and concurred by signature by theDepartment Manager and the Systems QA Manger. [5.2.2 Al


r

ITEMNO.

3.

4.

RESULTS

REV. .9/91

J

I



SHEET 4 OF 45

AUDIT/SURVEILLANCE

NO. H93-03

I I~~~~~~~~

CHARACTERISTIC TO BE EVALUATEDREMARKSRecord objective evidence reviewed, methodof verification, personnel contacted

1 t I.

Verify that the Plan Preparer has determined its classification andapplicable quality assurance requirements, has reviewed theclassification and quality assurance requirements with the DepartmentManager and the Systems QA Manger and obtained their agreement,and has included the classification and applicable quality assurancerequirements in the procurement plan. [5.2.2 B]

ITEMNO.

5.

RESULTS

.1-

REV. .9/91



SHEET 5 OF 45

AUDIT/SURVEILLANCE

NO. H93-03

-n - 0i I:


Verify that Procurement Plan include the procurement strategy, OAclassification and applicable quality assurance provisions, methods tobe used in accomplishing the major procurement tasks, sequence ofactions and milestones to indicate completion, and preparation ofapplicable documents prior to initiation of each of the followingactivities: [5.2.3]

A. Procurement document preparation, review, andchange in accordance with QAP-4-1, includingidentification of procurement documents requiredand designated preparers;

B. Solicitation;

C. Proposal/offer evaluation and qualification ofofferers;

D. Contract award (including procurement sourceselection);

E. Monitoring contract performance to include:

1. Verification (surveillance, inspection oraudit) activities by the M&O includingnotifications for hold or witness points;

2. Control of supplier nonconformances;

3. Supplier corrective action;

F. Delivery and acceptance;


ITEMNO.

6.

RESULTS

+ 4

REV. .9/91

.11 .11

\ I



SHEET 6 OF 45

AUDIT/SURVEILLANCE

NO. H93-03

_~~~~~~~~~~~~~~~~~~ . ,


G. Quality assurance records.


ITEMNO. RESULTS

REV. .9/91



SHEET 7 OF 45

AUDIT/SURVEILLANCE

NO. HO-93-03


+ 4 4

Verify that the Plan Preparer has completed preparation and obtainedwritten concurrence of the plan in accordance with QAP-4-1, obtainedwritten concurrence from the designated COTR (if assigned), anddistributed to the Department Manager, the Systems QA Manager,Affected Manager(s), the M&O Contracting Officer TechnicalRepresentative (COTR) and the Subcontracts/Purchasing Manager.[5.2.4]

Verify that changes to the procurement plan made after planacceptance has been accomplished, as specrfied in OAP-4-1, andthere is written concurrence by the M&O COTR. [5.2.5]

ITEMNO.

7.

8.

RESULTS

REV. .9/91

11

_;;;__&;fi~~~~~~~~~~~~~~~~- vYL Us MqZ 114 ;. 11 1--f I=

. . -- i

'. V



SHEET 8 OF

AUDIT/SURVEILLANCE

NO. HO-93-03

45

_ ~~~~~~~~~~~~I

11 A 6 _ .


PROCUREMENT DOCUMENT PREPARATION AND CONTROL

Verify that the Procurement Document Preparer or the SourceEvaluation Board (SEB) facilitated has developed offerer qualificationcriteria, technical and quality proposal evaluation criteria for theplanned procurement, and has documented the proposal evaluationcriteria on the Proposal/Bid Evaluation Criteria form (Attachment II)and obtained written concurrence in accordance with QAP-4-1.[5.3.2]

Verify that the designated Preparer has assembled the procurementauthorization package (including the Purchase Requisition, reference3.1.8), accepted procurement plan, accepted procurementspecification(s) and/or SOW with approving applicable PDDRs,referenced M&O requirements documents, offerer qualification criteria,and the proposal evaluation criteria. [5.3.]


ITEMNO.

9.

10.

RESU LTS

REV. .9/91

kY



SHEET 9 OF 45

AUDIT/SURVEILLANCE

NO. H93-03

I0


Verify that the designated Preparer has reviewed all quality relatedprocurement documents in the procurement authorization package toensure that they are still current, applicable to the proposedprocurement, and that any requirements documents referenced in theprocurement authorization package has been approved, released foruse, and are still applicable to the proposed procurement, and thereview is documented on the PDRR. [5.3.4]

Verify that the designated Preparer has obtained review andconcurrence of the procurement authorization package in accordancewith QAP-4-1, and obtained written concurrence from the designatedCOTR, if assigned. [5.3.5]


KITEMNO.

11.

12.

RESULTS

REV. .9/91

\1

I



SHEET 10 OF 45

AUDIT/SURVEILLANCE

NO. H93-03

I I~~~~~~~~~~

0 .~~~~~~~~~~~~~~~~~~~~~~~~~I 1110 L* 1. !1_s


Veriy that the Procurement Document Preparer has presented thecoordinated procurement authorization package, including PurchaseRequisition, to the Systems QA Managerfor coordination signature onthe Purchase Requisition, as required by the TRW ProcurementManual, that the package is forwarded to the S ubcontracts/PurchasingManager. [5.3.61

Note: If offerer qualification criteria limit the procurement toone known source, a justification of the single sourceshall accompany the Purchase Requisition.

Verify that the Subcontracts/Purchasing Manager has incorporatednon-quality affecting procurement documents, e.g., terms andconditions, boiler plate clauses, proposal format and instruction,certifications, etc., and has approved the procurement package.[5.3.7]


ITEMNO.

13.

14.

RESULTS

REV. .9191

I



SHEET 11 OF 45

AUDIT/SURVEILLANCE

NO. HO-93-03

I 5 5 5 l


SOLICITATION

Verify that the Subcontracts/Purchasing Manager has coordinatedresponses to offerer's questions and/or proposed changes to technicaland/or quality requirements, as necessary, using the same proceduresemployed in development of the original document. [5.4]


ITEMNO.

15.

RESULTS

REV. .9/91

I

I I/



SHEET 12 OF 45

AUDIT/SURVEILLANCE

NO. H93-03

.. i!l1-1; N] 21*x 0 - 0 --


OFFER/PROPOSAL EVALUATION AND NEGOTIATIONS

Verify that the Subcontracts/Purchasing Manager has screened outoffers and proposals that do not meet the offerer qualification criteriaand has had the remainder evaluated for conformance to technicaland quality proposal evaluation criteria by the Department Managerand the Systems QA Manager. 15.5.11


ITEMNO.

16.

RESULTS

REV. .9191

--- - -- I

I/



SHEET 13 OF 45

AUDITISURVEILLANCE

NO. HO-93-03

III - - - --


Verify that the technical and quality proposal evaluation has beendocumented on the Proposal/Bid Evaluation Criteria Forms(Attachment 11) and include the following subjects, as applicable:[5.5.1]

A. Technical Criteria;

B. Quality Assurance Program requirements;

C. Supplier personnel capabilities;

D. Supplier production capabilities;

E. Supplier past performance;

F. Supplier recommended alternatives;

G. Supplier exceptions to technical and/or qualityrequirements.

Note: If any of the above activities areperformed by the Source EvaluationBoard (SEB), the SEB will report itsevaluation results to the SourceSelection Official, who will select thoseoffers of proposals for furtherconsideration and negotiations.


ITEMNO.

17.

RESULTS

REV. 9/91

\. I \ I



SHEET 14 OF 45

AUDITISURVEILLANCE

NO. H-93-03

. ~ ~ ~ ~ ~ ~ ~ * -


4 4 +

Verify that Systems QA Manager has evaluated each offer requestedby the Subcontracts/Purchasing Manager for quality affectingprocurements and determined if the source is capable of providing theitem(s) or service(s) in accordance with the requirements of theprocurement document. [5.5.2]

ITEMNO.

18.

RESULTS

REV. .9/91

\ v

I /



SHEET 15 OF 45

AUDIT/SURVEILLANCE

NO. H93-03

~01111il q k 4w . .i ,


Verify that the Systems QA Manager has evaluated the capabilities ofthe qualified offers and documented the results on an Offerer QualityEvaluation form, (Attachment I) and the original has been sent to theSubcontracts/Purchasing Manager. The following methods may beused to evaluate the capabilities of qualified offerers: [5.5.3]

A. Review of current offerer history of providing similarproducts that have performed satisfactorily in actualuse;

B. Examination of current offerer quality recordssupported by available qualitative andquantitative information which can be objectivelyevaluated;

C. Evaluation of the offerer technical and qualitycapability based on direct evaluation of offerer'sfacilities, personnel, and quality assurance programimplementation.


ITEMNO.

19.

RESULTS

REV. .9/91



SHEET 16 OF 45

AUDIT/SURVEILLANCE

NO. H93-03

_4tWAF1U--9J;L' _ to] 4 q I24 _&f _ _ _ _


Verify that the Subcontracts/Purchasing Manager has coordinated andhas approved any changes to technical and/or qualrty requirementsthat were proposed during negotiations by using the same proceduresfollowed in the development of the original document. [5.5.4]


ITEMNO.

20.

RESULTS

REV. .9/91

'- / \ /



SHEET 17 OF 45

AUDIT/SURVEILLANCE

NO. HQ-93-03

S 0 *i0.~~~~~~~~~~~~I~-


4- I F

CONTRACT AWARD

Verify that the Subcontracts/Purchasing Manager has awardedcontracts in accordance with applicable procurement policies andprocedures (TRW Procurement Policies) 15.6] and has completed thepreparation of a finished contract which includes: [5.6.1]

A. Coordinated Statement of Work (SOW)incorporated by reference and attachment orincorporated as specification(s) within the body ofthe contract;

B. Name of supplier and evaluated location(s)approved for performance of work.

C. Clear reference to all applicable drawings andtechnical specification, (all referenced documentsshall be forwarded to the supplier with thecontract);

D. Establishment of lines of communication betweenthe supplier an the M&O for both written and verbaltechnical and/or quality related communications;

E. Where applicable, a requirement that quality affectwork not start until resolution of all open qualityissues and final M&O QA acceptance of thesupplier.

ITEMNO.

21.

RESULTS

REV. .9/91

J I/



SHEET 18 OF 45

AUDIT/SURVEILLANCE

NO. H93-03

I ~~IIII


Verify that prior to the contract award the Subcontracts/PurchasingManager has obtained review and concurrence with the contents ofthe contract in accordance with OAP-4-1, has obtained review andconcurrence from the M&O COTR, and it was documented on aPDRR form. [5.6.2]

Verify that if a contract is awarded prior to resolution of open qualityissues and final M&O QA approval of the selected source or one ofits supporting sub-tier contractors, the Subcontracts/PurchasingManager has documented the advanced release on the PDRR, andthe Department Manager and Systems A Manager is in agreement.[5.6.3]


ITEMNO.

22.

23.

RESULTS

REV. .9/9t

I



SHEET 19 OF 45

AUDITISURVEILLANCE

NO. H93-03 19 A * --


CONTRACT PERFORMANCE

Verify that the COTR has performed the oversight and managementactivities described in QAP-7-1, paragraph 4.5. [5.7.1]

Verify that proposed change affecting a technical or qualityrequirement has been controlled in accordance paragraph 5.1.3.[5.7.2]


ITEMNO.

24.

25.

RESULTS

REV. .9/91

--1 - '



SHEET 20 OF 45

AUDIT/SURVEILLANCE

NO. H93-03

M. -


+ -t ±

Verify that the Systems QA Manager and Department Manager havedetermined the extent and the means for evaluating supplier qualityperformance and is documented on a Supplier PerformanceEvaluation Plan/Record form (Attachment l1l). Also, verify that themeasures used to evaluate the supplier quality performance are notless than those specified in the procurement plan and include thefollowing: [5.7.3]

A. Reviewing quality requirements related procurementdocument(s) with supplier to assure anunderstanding of the requirements;

B. Requiring supplier to identify planning techniquesand processes to be utilized in fulfilling procurementdocument requirements;

C. Reviewing supplier quality and/or technicaldocuments generated or processed during activitiesfulfilling procurement document requirements;

D. Identifying and processing necessary changeinformation (reference paragraph 5.13);

E. Establishing method of document informationexchange with supplier (reference 5.6.1 D);

F. Establishing extent of supplier surveillance andinspection activities.

ITEMNO.

26.

7RESULTS

I- _ _ _ _ _ _

REV. .9/91

I

I



SHEET 21 OF 45

AUDIT/SURVEILLANCE

NO. H93-03

C - 01:j~ ~


+ 4

ACCEPTANCE OF ITEMS OR SERVICES

Verify that the supplier has verified through documentary evidencethat the item(s) or service(s) furnished complied with the procurementrequirements prior to M&O acceptance, installation or use. [5.8.1]

Verify that the Systems QA Manager and the Department Managerhave determined the means by which specific procurements are to beaccepted and documented on Purchase Acceptance QAPlan/Record(s), (Attachment V), and that the measure used are oneor a combination of Methods listed, but not less than specified in theprocurement plan. [5.8.2]

ITEMNO.

27.

28.

RESULTS

I- 4- 1

REV. .9191




METHOD A

Verify that the Supplier Certificate of Conformance has met all theminimum criteria below: [5.8.2]

1. The certificate identifies the purchased item(s),such as, by the contract number and contract linertem number;

2. The certificate identifies the specific procurementrequirements met by purchased item(s), such as,codes, standards, and other specifications, andinclude any approved changes, waivers, ordeviations applicable to the subject item(s);

3. The certificate identifies any procurementrequirements that have not been met, together withan explanation and the means for resolving thenonconformances.

4. The certificate is signed or otherwise authenticatedby a person who is responsible for this qualityassurance function and whose function and positionare described in the M&O or supplier's qualityassurance program;

5. The certification system, including the proceduresfor filling out certificates and administrativeprocedures for review and approval of certificates,is described in the M&O or suppliers qualityassurance program.

SHEET 22 OF 45

AUDIT/SURVEILLANCE

NO. H93-03


/

ITEMNO.

29.

RESULTS

REV. .9/91

- -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~-

N1- I



SHEET 23 OF 45

AUDIT/SURVEILLANCE

NO. HQ-93-03

I~~~~~~~~~~~~~~~~~~


6. The means provided to verify the validity of suppliercertificates and the effectiveness of the certificationsystem, such as during the performance of auditsof the supplier or independent inspections or testsof items.


ITEMNO.

1

RESULTS

REV. .9/91

I

-1 ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ '



SHEET 24 OF 45

AUDIT/SURVEILLANCE

NO. HO-93-03

I I


1- 4 4

METHOD B

If source verification is used, verify that it is performed at intervalsconsistent with the importance and complexity of the item or serviceand that it is implemented to monitor, witness, or observe activities.[5.8.21

METHOD C

If receiving inspection is used, verify that purchased items areinspected to verify conformance with specified requirements, andobjective evidence is used for verification in accordance with QAP-7-1, paragraph 5.10. [5.8.21

ITEMNO.

30.

31.

RESULTS

- - -__________ 4 A S

REV. .9/91

I

IOFFICE OF CIVILIAN

RADIOACTIVE WASTE MANAGEMENTU.S. DEPARTMENT OF ENERGY

WASHINGTON, D.C.

SHEET 25 OF 45

AUDIT/SURVEILLANCE

NO. H93-03

F M_ I ---


METHOD D

If post delivery testing is used, verify that the test requirements andacceptance documentation are mutually established between thesupplier and the M&O. [5.8.2]

Verify that the results of the specified method of acceptance aredocumented on the Purchase Acceptance QA Plan/Record along withthe identity of the M&O personnel making the evaluation and that wassubsequently reviewed by the Systems QA Manager and theDepartment Manager for accuracy and completeness anddocumented on the Purchase Acceptance QA Plan/Record. [5.8.31


ITEMNO.

32.

33.

RESULTS

REV. .9/91

I I . ,~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



SHEET 26 OF 45

AUDIT/SURVEILLANCE

NO. HO-93-03

ME II* v *- 0


For service procurements, verify that the Systems QA Manager, incoordination with the Department Manager, has determined theacceptability of the services and documented the results on theapplicable Purchase Acceptance QA Plan/Record. [5.9.1]


REV. .9/91

ITEMNO.

34.

11 �

\- I/



SHEET 27 OF 45

AUDIT/SURVEILLANCE

NO. HQ-93-03

~~~~~~~~~~~~~~~~S L n l .


Verify that the A acceptance method(s) are identified in theprocurement plan, e.g.,: [5.9.11

A. Technical verification of data produced;

B. Surveillance and/or audit of the activity;

C. Review of objective evidence for conformance tothe procurement document requirements, such as,certifications, stress reports, etc.


ITEMNO.

35.

RESULTS

REV. .9/91

l l

f

t I v



S * * -~~~~~~~~iiiiiim- -


f f F-

Verify that after completion of the specified method(s) of acceptance,the results have been documented on the Purchase Acceptance QAPlan/Record along with the identity of the M&O personnel making theevaluations. Also, veriy that the completed acceptance record hasbeen reviewed and verified for completeness and accuracy by theSystems QA Manager and the Department Manager, and has alsobeen documented on the Purchase Acceptance QA Plan/Record.[5.9.2]

ITEMNO.

36.

SHEET 28 OF 45

AUDITISURVEILLANCE

NO. HO-93-03

RESULTS

__________ F. -I. 4

REV. .9/91

/



SHEET 29 OF 45

AUDIT/SURVEILLANCE

NO. HQ-93-03

01 d11.: ' 1,. . . arTiI


RECEIVING INSPECTIONS

Verify that the Systems QA Manager, in coordination with theDepartment Manager, has prepared an inspection plan for eachreceiving inspection, that the plan is documented on a PurchaseAcceptance QA Plan/Record, and the plan identifies or incorporatescharacteristic references, methods and acceptance criteria. [5.10.1]


ITEMNO.

37.

RESULTS

REV. .9/91



SHEET 30 OF 45

AUDIT/SURVEILLANCE

NO. HO-93-03

I I I


1- 4

Verify that the results of receiving inspections are documented on thePurchase Acceptance QA Plan/Record and it includes the following:[5.10.2]

A. The item(s) inspected;

B. Date(s) of inspection;

C. Inspector(s);

D. Type(s);

E. Inspection criteria;

F. Characteristics inspected;

G. Material and test equipmentinspection;

used for the

H. Results or acceptability;

I. Reference to information on action taken inconnection with supplier nonconformances.

ITEMNO.

38.

RESULTS

REV. .9/91



SHEET 31 OF 45

AUDITISURVEILLANCE

NO. HQ-93-03

9S 1 .ii"111.A 4 A.3i . * S;&0 -,-0;C ".


A-- f

Verify that modifications, repairs, or replacements of items, as a resultof the receiving inspection, have been reinspected, and/or retested,as appropriate, to verify acceptability. [5.10.3]


ITEMNO.

39.

RESULTS

REV. .9191

K I I



SHEET 32 OF 45

AUDIT/SURVEILLANCE

NO. H93-03

- -- - -* *1 £ IIC-lt m - 0o i - -

CHARACTERISTIC TO BE EVALUATEDREMARKSRecord objective evidence reviewed, methodof verification, personnel contacted.

F -t -+

Verify that the Systems GA Manager and the Department Managerhave reviewed the tems listed below to confirm the completeness,accuracy, and conformance to the procurement requirements, andthat the review has been completed prior to final acceptance andrelease for M&O use and has been documented on the applicablePurchase QA Plan/Record. [5.10.4]

A. Results of the receiving inspection;

B. Review of supplier documents submitted;

C. Review and resolution of reported suppliernonconformances.

I

ITEMNO.

40.

RESULTS

REV. .9/91

\1 I/ JI

N,.y.I I/




CONTROL OF SUPPLIER GENERATED DOCUMENTS

Verify that procurement required documents generated by the supplierfor M&O review and/or approval have been submitted to theSubcontracts/Purchasing Manager, and that technical and/or qualityrelated documents have been reviewed by the Department Managerand the Systems QA Manager and the results documented on aSupplier Document Evaluation Record (Attachment IV). [5.11]

SHEET 33 OF 45

AUDIT/SURVEILLANCE

NO. HQ-93-03


+

ITEMNO.

41.

RESULTS

REV. .9/91

_s~~~~~~~~~~~II=:11K



SHEET 34 OF 45

AUDIT/SURVEILLANCE

NO. H-93-03

I _ I , -I , I


CONTROL OF SUPPLIER NONCONFORMANCES

Verify that supplier nonconformances have been submitted to theSubcontracts/Purchasing Manager for approval the followingdispositions: [5.12.1]

A. Technical or material requirement was not met;

B. M&O approved requirements in the supplierdocuments have not been met;

C. Nonconformance cannot be correct by continuationof the original manufacturing process or by rework;

D. The rtem dose not conform to the originalrequirement and restoration will not permit the temto function unimpaired;

E. Any other condition identified by the M&O ofsupplier nonconformance.


ITEMNO.

42.

RESULTS

REV. .9/91



SHEET 35 OF 45

AUDIT/SURVEILLANCE

NO. H93-03

I iI


Verify that the Subcontracts/Purchasing Manager has received thesupplier's nonconformance reports, including the supplier'srecommended disposition with technical justification, for review and/orapproval, and has had the documents reviewed by the DepartmentManager and Systems QA Manager. [5.12.2]


ITEMNO.

43.

RESULTS

REV. .991

\� / 'I



SHEET 36 OF 45

AUDIT/SURVEILLANCE

NO. H93-03

9 £ .4 i '


I t

Verify that the Department Manager has completed the following:

1. Reviewed the noncomformance report, accepted orrejected the supplier's recommended disposition,and documented the results on a SupplierDocument Evaluation Record (SDER);

2. Obtained a review of the nonconformanceevaluation by the Systems QA Manager;

3. Provided a copy of the SDER toapplicable, and returnSubcontracts/Purchasing Manager.

the COTR, ffit to the[5.12.3]

Verify that the Systems QA Manager has documented the QA reviewon the SDER. [5.12.3]

ITEMNO.

44.

45.

RESULTS

REV. .9/9t

\J-



SHEET 37 OF 45

AUDIT/SURVEILLANCE

NO. HO-93-03

S i i ~~~~~~~~~Sii L* n"^ - ^ :...S a; ,i 0 --i


Verify that the Subcontracts/Purchasing Manager has completed thefollowing:

1. Advised the supplier of the M&O's decision andobtain a written verification that the M&O'sapproved disposition has been implemented;

2. Documented the supplier's verification on theSDER, placed a copy of the verfication with theSDER, and sent a copy of both to the Systems QAManager and the COTR. [5.12.4]


ITEMNO.

46.

RESULTS

REV. .9/91

I



SHEET 38 OF 45

AUDIT/SURVEILLANCE

NO. H-93-03

I* 0 1 Y'i MM -I ........ ... .~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


CONTROL OF CHANGES TO PROCUREMENT DOCUMENTS

Verify that the Subcontracts/Purchasing Manager has obtainedreview and concurrence of any subsequent changes to the technicalor quality provisions of the contract, in accordance with QAP-4-1, andfrom the M&O COTR on the PDRR. [5.13]


ITEMNO.

47.

RESULTS

REV. .9/91



SHEET 39 OF 45

AUDIT/SURVEILLANCE

NO. H-93-03

S * a.L;,,,., , -,,, ,, ,.,- 1 1


COMMERCIAL GRADE ITEMS

Verify that any procurement item considered commercial grade isidentified in an approved design output document. [5.14.1 A]

Verify that source evaluations and selections are in accordance withparagraph 5.5 and 5.6. [5.14.1 B]


ITEMNO.

48.

49.

RESULTS

REV. .9/91



SHEET 40 OF 45

AUDITISURVEILLANCE

NO. HQ-93-03

a a a 0 ~~~~ ~~~~~~~~- l


Verify that commercial grade items are identified by themanufacturer's published product description in the contract, e.g.,catalog number. [5.14.1 C]

Verify that after receipt of commercial grade items that damage wasnot sustained during shipment, the items received were the itemsordered, inspection and/or testing was done to ensure conformancewith published requirements, and documentation was received,reviewed and was acceptable. [5.14.1 DI


ITEMNO.

50.

51.

RESULTS

REV. .9/91



SHEET 41 OF 45

AUDIT/SURVEILLANCE

NO. H93-03

* .C E I

CHARACTERISTIC TO BE EVALUATEDREMARKSRecord objective evidence reviewed, methodof verrfication, personnel contacted

4 I 4

Verify that approval of commercial grade items were documented onan approved procurement document or a PDRR. [5.14.21

Verify that receiving inspection planning and results have beendocumented by personnel on a Purchase Acceptance QAPlan/Record. [5.14.3]

ITEMNO.

52.

53.

RESULTS

REV. 9/91

<IOFFICE OF CIVILIAN


WASHINGTON, D.C.

SHEET 42 OF 45

AUDITISURVEILLANCE

NO. HO-93-03

I -iili ;


THE FOLLOWING REFER TO THE M&O QAPD, REVISION 2.

Verify that proposed suppliers of items and supplies have submittedcopies of their QA Program Descriptions. [7.1]

Verify that the M&O has received proposed suppliers' OA ProgramDescriptions, and documented acceptance prior to allowing work onquality affecting activities. [7.1]


RESULTS

l l_l

REV. .9t91

ITEMNO.

54.

55.



SHEET 43 OF 45

AUDITISURVEILLANCE

NO. HQ-93-03

I L -I111 1 I-1


I- 4

Verify that the M&O has identifiedability to meet expectations. [7.2]

measures to evaluate suppliers'

Verify that the applicable procurement documents include informationspecified in QAPD, paragraph 7.2.1, e.g., schedules, items andservices to be procured, bid evaluations, hold points... [7.2.1]

ITEMNO.

56.

57.

RESU LTS

REV. .9/91

f \ /



SHEET 44 OF 45

AUDIT/SURVEILLANCE

NO. H93-03

S s 0 - ;i -


± t *

Verify that procurements [suppliers] were selected in accordance withapplicable M&O OAP's. [7.2.2]

Verify that the performance by suppliers, including sub-tiercontractors, were evaluated by audits and/or surveillances. [7.2.3]

I

-.-. /

ITEMNO.

58.

59.

RESULTS

REV .9/91

I



SHEET 45 OF 45

AUDIT/SURVEILLANCE

NO. HQ-93-03

a - -I


Veriy that the means for accepting procured items or services hasbeen identified, e.g., review of data and reports, certificates ofconformance, receipt, and inspections. [7.2.5]


ITEMNO.

60.

1

RESULTS

REV. .9/91

I



PAGE 1 OF 4

AUDIT/SURVEILLANCE

NO. HQ-93-03


M&O/TRW

DATES OF EVALUATION

February 1-5, 1993

I_I I I

[X] EXTERNAL

[ ]INTERNAL

[X] AUDIT

[ ]SURVEILLANCE PREPARED BY Fred Bearham DATE 1/22/93

CONTROLLING DOCUMENT (Title, Number, Revision) ACTIVITY EVALUATEDQAP-1 -1, Rev. 1, Escalation of Quality Disputes Quality Disputes


1 Verify that personnel assigned to administer this element of the QAprogram receive appropriate orientation. (Para. 1)

* INDICATE RESULTS: SATISFACTORY (SAT), UNSATISFACTORY (UNSAT), NOT APPLICABLE (N/A)

REV. 09/91

/ \.- /OFFICE OF CIVILIAN


WASHINGTON, D.C.

SHEET 2 OF 4

AUDIT/SURVEILLANCE

NO. HQ-93-03

£ -


Verfy that the difference between Quality Disputes and QualityConcerns is clearly identified. (Para. 3.2)

Verfy that QAP-1 -1 and OAAP 1.2 are included in training schedules.(Para. 4.4)


4 4

ITEMNO.

2

3

RESULTS

REV. 11/90

"I



SHEET 3 OF 4

AUDITISURVEILLANCE

NO. HQ-93-03

C 0 * 0 -- ~ ~ - m


Review the rationale for documenting a difference of opinion. (Para.5)

Review the rationale for escalating a dispute into a quality concem.(Para. 5.1.11)


It ----------- 1--

ITEMNO.

4

5

RESULTS

REV. 11/90

.-



Review documentation of this function and verify that t is collectedand maintained in accordance with QAP-17-1. (Para. 6.1)

Determine the effectiveness and adequacy of this procedure.

SHEET 4 OF 4

AUDIT/SURVEILLANCE

NO. HQ-93-03

_ I


- -

ITEMNO.

6

7

RESULTS

REV. 11/90

.-

"K



PAGE 1 OF 5

AUDIT/SURVEILLANCE

NO. HQ-93-03


M&O/TRW

DATES OF EVALUATION

February 1-5, 1993

I I ~~~~~~~~~~~I I

[XI EXTERNAL

[]INTERNAL

[XI AUDIT

[ ] SURVEILLANCE PREPARED BY Fred Bearham DATE 1/22/93

CONTROLLING DOCUMENT (Title, Number, Revision) ACTIVITY EVALUATEDOAP-2-1, Rev. 2, Indoctrination and Training Indoctrination and Training


1 Verify that managers/supervisors designate M&O personnel, includingsubcontractors and consultants, undertheir supervision to attend programindoctrination classroom training. Training is mandatory for all personnelprior to performing quality-affecting activities. (Para. 5.1.1)


REV. 09/91



SHEET 2 OF 5

AUDIT/SURVEILLANCE

NO. HQ-93-03

p 11*1 C4 01f 154M


All classroom training shall be scheduled through the trainingdepartment. (Para 5.1.2)

Note: Verify that this includes training by outside agencies.

Verify that program indoctrination classroom training include thefollowing: (para. 5.1.3)

QARD (DOE/RW-0214)* M&O QAPD* QAP-1-1* QAP-2-1* QAP-2-3* QAP-6-1* QAP-1 6-1* QAP-1 7-1(Para. 5.1.3)


ITEMNO.

2

3

RESULTS

REV. 11/90

,----4--



SHEET 3 OF 5

AUDITISURVEILLANCE

NO. H-93-03

* A -l0 --"eleJL0


Review objective evidence individuals have read the current revisionof the procedures listed prior to attending program indoctrination andtraining. (Para. 5.1.4)

Review objective of completion of training above, including: lessonplans, attendance sheets, & training matrices (if applicable)(Para. 5.1.5)


ITEMNO.

4

5

RESULTS

REV. 11/90

I



SHEET 4 OF 5

AUDIT/SURVEILLANCE

NO. H-93-03

_ A _ I ____~~~~~~~~411- q k , K*i1t0K4 ' {3ffI_ _Il


Review records of "briefing" types of training (Para. 5.2)

Verifythat manager/supervisors submitaWaiverof Required Training,to exempt M&O personnel from all or part of the programindoctrination requirements. (Para. 5.3.1)


I ffi-

ITEMNO.

6

7

RESULTS

REV. 11/90

I



SHEET 5 OF 5

AUDITISURVEILLANCE

NO. HQ-93-03

I e I r, | w r.J; z | ~~~~~~* v A A@; e e 0 *;w I ''' 1


Review justification for waiving training provided by themanager/supervisor and approval by the Training Manager before anyquality-affecting activities are conducted (para. 5.3.2)

Verify that waivers of Required Training Form are part of anindividual's training file. (Para. 5.3.3)


7ITEMNO.

8

9

RESULTS

REV. 11190

__1__



PAGE 1 OF 4

AUDITISURVEILLANCE

NO. HQ-93-03


M&O/TRW

DATES OF EVALUATION

February 1-5, 1993

I ~~~~~~~~~~~~~I III

[X] EXTERNAL

[ ] INTERNAL

[X] AUDIT

[ ] SURVEILLANCE

CONTROLLING DOCUMENT (Title, Number, kt vision)OAP-2-2, Rev. 1, Verification of Personnel Qualifications

PREPARED BY Fred Bearham DATE 1/23/93

ACTIVITY EVALUATEDVerrfication of Education and Experience


1 Review Position Descriptions (PDs) for several personnel for accuracy,completeness and relevancy. (Para. 5.1.1)


REV. 09/91



SHEET 2 OF 4

AUDIT/SURVEILLANCE

NO. HQ-93-03

I I 1 ;


Verify that Managers/Supervisors obtain generic Position Descriptions(PDs) from the M&O Human Resources Manager and ensure thatPDs describe the minimum skills, knowledge, education, andexperience for each position wrthin their organization.

Review PDs for several disciplines for accuracy and consistency(Para. 5.1.1)

Verify that manager/supervisor list quality-affecting activities in theappropriate section (Para. 5.1.2)


ITEMNO.

2

3

RESULTS

REV. 11/90

I



SHEET 3 OF 4

AUDIT/SURVEILLANCE

NO. HQ-93-03

;||l~~ 0 *lli 14 ^; SN 1 l;"


Verify that manager/supervisor sign the completed PositionDescription and return it to the M&O Human Resources Manager.(Para. 5.1.3)

Verify that the M&O Human Resources Manager verfies anddocuments the highest level of education of each person on theCRWMS M&O Verification of Education Form (Attachment II).(Para. 5.2.1)


ITEMNO.

4

5

RESULTS

REV. 11/90



SHEET 4 OF 4

AUDIT/SURVEILLANCE

NO. H-93-03

==£11 * R


Verify that managers/supervisors verify relevant experience bychecking personnel documentation such as applications or resumes,for personnel under their supervision, and so document by signing thePosition Description. (Para. 5.2.2)

When minimum education and experience cannot be specificallyverified, check that managers/supervisors provide a statement andjustification for the personnel assignment. (Para. 5.2.3)


ITEMNo.

6

7

RESULTS

REV. 11/90



PAGE 1 OF 11

AUDIT/SURVEILLANCE

NO. HQ-93-03


M&O/TRW (Vienna)

DATES OF EVALUATION

February 1-5, 1993

[X] EXTERNAL

[]INTERNAL

[X] AUDIT

[ ]SURVEILLANCE PREPARED BY Fred Bearham DATE 1122/93

CON-ROLLING DOCUMENT (Ttle, Number, Revision) ACTIVITY EVALUATEDQAP-2-3, Rev. 3, Classification of Items Classifying tems

REMARKSITEM CHARACTERISTICS TO BE EVALUATED Record objective evidence reviewed, method RESULTSNO. of verification, personnel contacted

1 Review the available Q-Lists. (Para. 2)


REV. 09/91

\- /




Verify that department managers ensure that individuals participatingin the classification process are qualified to perform quality affectingwork per QAP-2-1, Indoctrination and Training, and QAP-2-2,Verification of Personnel Qualifications. (Para 5.1.1)

Veriy that department managers ensure that a database forconsolidation of the recommended Q-List is created and maintained.(Para 5.1.2)

SHEET 2 OF t1

AUDIT/SURVEILLANCE

NO. HQ-93-03


ITEMNO.

2

3

RESULTS

-4- I. .

REV. 11/90



SHEET 3 OF 11

AUDIT/SURVEILLANCE

NO. HQ-93-03

* 0 - 0 --

-~ W, -- - Q lUW I


Verify that department managers ensure the designers use theclassification checklists (Attachments I and 11) to classify all itemsidentified in the current Technical Baseline and any new itemsidentified throughout the design process. (Para 5.1.3)

Verify that department managers ensure that any reference materialsupporting the classification process is controlled per applicableQuality Administrative Procedures. (Para. 5.1.4)


ITEMNO.

4

5

RESULTS

REV. 11/90

V



SHEET 4 OF tt

AUDITISURVEILLANCE

NO. HQ-93-03

I _


Determine if any design is in the post conceptual phase and verify thedepartment managers ensure that final classification is accomplishedduring each post conceptual design phase and that recommended Q-Lists are developed/revised and submitted to the applicable officemanager at the end of each post conceptual design phase.(Para. 5.1.5)

Verify that designers shall use the attached checklists (AttachmentsI and 11) for classifying items. Items that do not meet any of thecriteria contained in the attached checklists shall be consideredconventional quality. (Para. 5.1.6)


ITEMNO.

6

7

RESULTS

REV. 11190

----------



SHEET 5 OF 11

AUDIT/SURVEILLANCE

NO. HQ-93-03

I I


Verify that the QACMD MRS or the OACMD MGDS is used forevaluating the questions in the attached checklists.(Attachments I or 11)

Note: Prior to the beginning of the site pecific design phase, itemsmay be evaluated without using the QACMD; however, theseevaluations shall be conservative to ensure items Important to Safetyand Waste Isolation: are identified. Any items that are not evaluatedusing the QACMD shall be reevaluated using the QACMD before thedesign phase is completed. (Para. 5.1.7)

Verify that designers ensure that engineering calculations and/oranalyses, required by the QACMD, are completed to support eachclassification.

Note: These calculations and/or analyses shall be accomplished perQAP-3-9, Engineering Calculations and Analyses. The designersmust also ensure that applicable calculations and/or analyses arecompleted to justrfy those items determined to be conventional quality.(Para. 5.1.8)


ITEMNO.

8

9

RESULTS

REV. 11190

NS I/



SHEET 6 OF 11

AUDITISURVEILLANCE

NO. HQ-93-03

=


Verify that design records packages include the classificationchecklists and the calculations and analysis performed per QAP-3-9,Engineering Calculations and Analyses, and obtain a RecordsPackage Tracking Number from the Local Records Center. Thispackage shall include QAP-3-9 Attachment I, Tale Page andCertification of Engineering Calculation." (Para. 5.1.9)

Note: Review with Team B"

Verify that designers update the recommended Q-List database withthe following information concerning the classrfication of the item:

a. The Item Nameb. Classification Nomenclature (i.e., Important to Waste Isolation)c. The Configuration Identifier of the Itemd. A Description of the Iteme. The Records Package Tracking Number (for tracking of basis

information)f. The Date Classified (same as the approved by" date from the

"Ttle Page and Certification of Engineering Calculation" form).

Review recent addition to the Q-List for updates. Para. 5.1.10)


ITEMNO.

10

11

RESULTS

REV. 11J90

I



SHEET 7 OF 11

AUDITISURVEILLANCE

NO. HQ-93-03

p . h


Verify that designers identify within the database all items that havebeen reclassified to a different classification or conventional quality(Para. 5.1.11)

Verify that, biannually, the department manager shall compile aconsolidated recommended 0-List from the database. Therecommended 0-List shall contain the same elements of informationoutlined in paragraph 5.1.10 and be sorted by ClassificationNomenclature in the following order:

a. Important to Radiological Safetyb. Important to Waste Isolationc. Important to Radioactive Waste Controld. Important to Fire Protectione. Important to Seismic Interactionf. Important to Physical Protection of Facility and Materialsg. Important to Radiological Protection (Para. 5.2.1)


RESULTS

4 +

REV. 11/90

ITEMNO.

12

13

I /



SHEET 8 OF 11

AUDIT/SURVEILLANCE

NO. HQ-93-03

I I ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~~~~~~~~ mI I


Verify that department managers forward the recommended Q-List tothe applicable office manager for consolidation into a single database.(Para. 5.2.2)

When the design phase is complete verify that the office managerdistributes the recommended Q-List under cover letter as follows:

a. The recommended Q-List for MGDS shall be forwarded to theYucca Mountain Project Deputy Project Manager for review andapproval and eventual issuance of a project level 0-List.

b. The recommended 0-List for MRS MSA shall be forwarded toOCRWM's Office of Storage and Transportation (RW-40) forreview and approval and eventual issuance of a project level 0-List.

c. The cover letter shall also include a list of those items that havebeen reevaluated to a different classification or conventionalQuality since the previous recommended Q-List submittal.

Note: It appears that YMPO Procedures control the MGDS 0-List andthat the HQ Office Manager is responsible for distribution.(Para. 5.2.3)


ITEMNO.

14

15

RESULTS

REV. 11190

/



I I~ ~~~~~~~~ . ~ ~ ~ ~ ~ ~ ~ ~ 1111462I1M M

3~~ ===II


VerHy that revisions to the Q-Lists are controlled in accordance withthe following:

a. The classification process is iterative and begins at the systemsrequirement level and proceed ultimately to the component levelas the design increases in detail. Therefore, revisions to therecommended Q-List are required as the design matures.

b. After a recommended Q-List has been submitted to the DOE,the Office Managers shall review the recommended -Listreceived per step 5.2.2 and forward a list of all new items anditems that have been reclassified to the DOE.

Note: Para.5.2.2 is not a review step.

C. Revisions to the recommended 0-Lists shall be accomplished bycompleting the steps outlined in paragraphs 5.1.1 through 5.2.3above. (Para. 5.3)

SHEET 9 OF 11

AUDIT/SURVEILLANCE

NO. HQ-93-03


ITEMNO.

16

RESULTS

REV. 11/90

- ~ ~~~ ~~~~~~~~~~~~~~~\ /



SHEET 10 OF 11

AUDIT/SURVEILLANCE

NO. H-93-03

I Ii1ll illE ~~~~~ ~ ~ ~ ~~~~~~~~~~~I I I --- I i


Review any available evidence that prior to beginning an activity, theinterfacing manager responsible for the activity determines if thatactivity is quality affecting. (Para. 5.4.2)

Review items that have more than one classification depending ontheir application. (Para. 5.5.1)


ITEMNO.

17

1 8

RESULTS

.1- 4- J

REV. 11/90

" I



SHEET 11 OF ii

AUDIT/SURVEILLANCE

NO. HQ-93-03

S - _ 0 * 0_iZ~~ ~~~~~~~~~~~~~~~~~~~ I_


Verify DOE QA review and approval of QAP-2-3. (Para. 2.4)

Verify that QAP-2-3 has been reviewed against RW/0333Prequirements. (General)


ITEMNO.

19

20

RESULTS

_______ _________________________________________ -� 4

REV. 11/90

, .

\� V \� -I



PAGE 1 I

AUDIT/SURVEILLANCE

NO. HQ-93-03


M&O TRW (Vienna)

DATES OF EVALUATION

February 1-5, 1993

I _l

[X] EXTERNAL

[]INTERNAL

[X] AUDIT

I ]SURVEILLANCE PREPARED BY Fred Bearham DATE 1/25/93

CONTROLLING DOCUMENT (Title, Number, Revision) ACTIVITY EVALUATEDQAP-2-6, Rev. 1, Readiness Review Readiness Reviews (RRs)

REMARKSITEM CHARACTERISTICS TO BE EVALUATED Record objective evidence reviewed, method RESULTS

NO. _________________________________________________________________ of verification, personnel contacted

1 Verify that guidelines for the performance of readiness reviews include:location of the review (site or Office); details of the design phase,process, or other Program activity to be reviewed; items requiring specialattention such as potential problem areas, items that impact schedule oritems requiring specialized technical expertise; results of applicablemanagement assessments, per reviews, design reviews, technicaldocument reviews, and readiness reviews; appropriate acceptance criteriato be used during the readiness review; and identification of personnelassigned to assist the reviewers. (Para. 5.1)


REV. 09/91

OF 10

'S /



SHEET 2 OF 10

AUDIT/SURVEILLANCE

NO. H-93-03

9 S - I


Verify that team members have, at a minimum, experience in thedisciplines undergoing readiness review. Team members shall betrained on this procedure and other appropriate procedures asassigned by the Readiness Review Team Leader (RRTL).(Para. 5.2.2)

Verify that the Readiness Review (RR) Chairperson, Review Boardand Team Leader are assigned by the General Manager. (Para.5.4.2)


ITEMNO.

2

3

RESULTS

REV. 11/90

I



SHEET 3 OF 10

AUDIT/SURVEILLANCE

NO. HQ-93-03

p . - 0~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


Verify that the Readiness Review Team (RRT) prepares theReadiness Review Plan using the guidelines provided by the GeneralManager the Plan and includes the following, as appropriate:

A. Introduction and overview

B. Scope and areas to be covered

C. Objectives to be determined

D. Reference procedures to be used

E. Readiness review guidelines

F. Readiness review schedule, indicating significant milestonesand due dates for reports

G. Identification of RRT members and their area ofresponsibility. (Para. 5.4.4)

Verify that RRTL reviews the Plan with the Readiness Review Boardand obtains the approval of the Board Chairperson, and develops anAttribute List (Paras. 5.4.5 and 5.4.6)


ITEMNO. RESU LTS

REV. 11/90


Verify that the RRT uses the approved Attribute List to conduct thereadiness review and ensure that each identified prerequisite isevaluated. Team members shall initial and date each attribute whenit is verified that the prerequisite has been satisfied or identify theOpen Number for attributes that are not closed. (Para. 5.5.1)

Verify that team members document any attribute remaining open onan Open Item Report and that the Open Rem Report identifies:

A. The Attribute and Open Item Numbers

B. Descriptions of the prerequisites and the open Rem

C. Required actions, and the responsibility and estimatedcompletion date for each action

D. Name of the preparer and date

E. Identfication of any hold points that are established H workis permitted to start or resume prior to closure of the openitem




SHEET 4 OF

AUDITISURVEILLANCE

NO. H-93-03

ITEMNO.

6

7

10

RESULTS

REV. 11/90

-



SHEET 5 OF 10

AUDIT/SURVEILLANCE

NO. HQ-93-03

pnwwftzo~~~~~~~~ -- --9 t 0 - 0 -


-4- 4

Verify that Open Items are evaluated and processed as a CorrectiveAction Report, i appropriate (Para. 5.5.3)

Verify that an Open Items List is prepared for tracking of open tems.(Para. 5.5.4)

ITEMNO.

8

9

RESULTS

.1 I. a

REV. 11/90

/



SHEET 6 OF 10

AUDIT/SURVEILLANCE

NO. H-93-03

I I~~~~~~~~~~~~~9C0-0-


Verify that the Readiness Review Report indicates that all items onthe Attribute List have been closed or have been incorporated onOpen Item Reports, as appropriate. (Para. 5.5.5)

Verify that the report includes a list of team members and anyrecommendations regarding readiness to start or continue theundergoing. (Para. 5.5.5)


ITEMNO.

10

11

RESU LTS

REV. 11190

'



SHEET 7 OF-

AUDIT/SURVEILLANCE

NO. H-93-03

I I l | l !E s S l l | l l S - *


Verify that the Readiness Review Report is approved by theReadiness Review Board and that any questions from the board areclarified (Para. 5.5.6)

Verrfy that the Readiness Review board advises the Chairperson asto the disposition of the report. The board may accept the report ordirect changes to reflect the decisions of the board. The Chairpersonwill sign the final report approved by the board and forward it to theGeneral Manager for approval. (Para. 5.5.7)

REMARKSRecord objective evidence reviewed, methodof verrfication, personnel contacted

10

ITEMNO.

12

13

RESULTS

REV. 11/90

\i Z J Iv



SHEET 8 OF 10

AUDIT/SURVEILLANCE

NO. H-93-03

___A -


Verify that the Readiness Review Report is reviewed and approvedby the General Manager and distributed to affected organization(Para. 5.6.1)

Verify that open items are tracked by the QA Manager and that theGeneral Manager is notified upon completion of closure activities.(Para. 5.6.2)


ITEMNO.

14

15

RESULTS

REV. 11/90



SHEET 9 OF 10

AUDIT/SURVEILLANCE

NO. H-93-03

- - 0~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


Review the procedure for control of Hold Points. (Para. 5.6.3)

Verify that authorization by the General Manager to start or continuethe Program activity is documented. (Para. 5.6.3)


ITEMNO.

16

17

RESULTS

REV. 11190

'------ "

1--

/ \. I \ IOFFICE OF CIVILIAN


WASHINGTON, D.C.

SHEET 10 OF 10

AUDITISURVEILLANCE

NO. HQ-93-03

__C C 4 * 4 --


Verify the Readiness Review Plan, Open Item Reports, ReadinessReview Team Report, final Readiness Review Report signed by theReadiness Review Board Chairperson, and General Manager'sauthorization to start or continue work are QA records collected andmaintained in accordance with QAP-17-1, QA Records Management(Para. 6.0)


ITEMNO.

18

RESULTS

REV. 190

II \1 I/

\ I


U.S. DEPARTMENT OF ENERGYWASHIN(iTON. DC.

PAGE 1AUDITSURVEILLANCE

NO. HQ-93-03


M&O/TRW (Vienna)

DATES OF EVALUATION

February 1-5, 1993

[XI EXTERNAL

E INTERNAL

[XI AUDIT

[ SURVEILLANCE PREPARED BY Fred Bearham DATE 1/25/93

CONTROLLING DOCUMENT (Ttle, Number, Revision) ACTIVITY EVALUATEDQAP 2-9, Rev. 0, Development and Conduct of Training Training


1 Review lesson plans for instructor development training. (Para. 5.2.1)


REV. 09/91

.I. k

OF 8

j



SHEET 2 OF_

AUDITISURVEILLANCE

NO. H-93-03

l I I " -* -O


+ 4 t

Review the process for planning and coordination of trainingadministered by external agencies. (Para. 5.1.4)

Review the process for interfacing between managers/supervisors andthe M&O Training Manager. (Para. 5.1.1)

ITEMNO.

2

3

8

RESULTS

1. I L

REV. 11/90

/

\ -i



SHEET 3 OF 8

AUDITISURVEILLANCE

NO. H-93-03

_ L A S - O I-


Review the selection process and position descriptions for SMEs andtraining department personnel. (Para. 5.1.3)

Review the Initial Instructor Development Training Course to verifythat it prepares personnel to develop, evaluate, and presentclassroom instruction using techniques consistent with the systematicapproach to training. (Para. 5.2.1)

REMARKSRecord objective evidence reviewed, methodof verffication, personnel contacted

ITEMNO.

4

5

RESULTS

REV. 11190



SHEET 4 OF 8

AUDIT/SURVEILLANCE

NO. H-93-03

l Z c 4 ' l l


Review the Initial Instruction Development Training records and 5-yearre-certification process for instructor, if appropriate.(Paras. 5.2.2 and 5.2.3)

Review the process for Subject Matter Expert (SME) input to lessonplan 5


ITEMNO.

6

7

RESULTS

4 4

4 4

REV. 11190



SHEET 5 OF 8

AUDITISURVEILLANCE

NO. H-93-03

I,m -- - . I0I

I I~~~~~~~~~~~~~~~~~~~~


Are Subject Matter Experts (SMEs) preparing initial lesson plans andrevisions in accordance with Attachment III? (Para. 5.3.1)

Are the lesson plans being approved by certified instructors and theTraining Manager? (Para. 5.3.2 and 5.3.3)

REMARKSRecord objective evidence reviewed, methodof verfication, personnel contacted

ITEMNO.

8

9

RESULTS

REV. 11/90



SHEET 6 OF 8

AUDIT/SURVEILLANCE

NO. H-93-03

PCXI -


+ + t

Verify that the Training Department shall issue notification to M&Opersonnel of classroom instruction availability, scheduled class time,and location of training classes. (Para. 5.4.1)

Review documented evidence of briefings. (Para. 5.5.10)

ITEMNO.

10

11

RESULTS

REV. 11/90

\. / \ /



SHEET 7 OF 8

AUDITISURVEILLANCE

NO. HQ-93-03


Verify that training records generated as a result of this procedureshall be processed by the Training Department as privileged recordsin accordance with QAP-17-1. (Para. 6.1.1)

Verify that training records for M&O personnel are processed by theTraining Department at each M&O location and that MRS personneltraining records are processed at the M&O Headquarters.(Para. 6.1.2)


RESULTS

lII ._I

REV. 1/90

ITF MNO.

13

,__ _ _ _ _ _ _ _ * 1 ..

.< / /

_~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



SHEET 8 OF 8

AUDIT/SURVEILLANCE

NO. HO-93-03

i II--I


± I -t

Verify that the following documents are processed as privilegedrecords by the Training Department and forwarded to the LRC whencompleted in accordance with QAP-17-1, Program RecordsManagement: Record Source Responsibilities.

* M&O Instructor Certification* Completed and approved Lesson Plans* Examinations and test results.

(Para. 6.1.4)

ITEMNO.

14

I1 '

RESULTS

4- 4 .4.

REV. 11190



PAGE 1 OF 13

AUDIT/SURVEILLANCE

NO. HQ-93-03

e j A;14 I ; ' a i

I ~~~~~~~~~~~~~~I III


M&O - Vienna, VA

DATES OF EVALUATION

February 1-5, 1993

[x] EXTERNAL

[]INTERNAL

[x] AUDIT

[ ]SURVEILLANCE PREPARED BY Marlin Horseman/Hank Greene DATE 1/21/93

CONTROLUNG DOCUMENT (Title, Number, Revision) QAP-5-1, Rev. 1, ACTIVITY EVALUATEDPreparation of M&O Quality Administrative and Implementing Line Procedures QAP/ILP Procedure Control


1 Verify that instructions exist to implement the M&O QAPD, 5.0requirement that "planning, preparation, review, approval, issuance, andtraining of personnel to these documents is accomplished prior to thestart of quality affecting work.'


REV. 09/91

!5 - - -- --



SHEET 2 OF 13

AUDIT/SURVEILLANCE

NO. HQ-93-03

I A A A B -i e M -- a I


Verify that recently issued QAPs were prepared, using the appropriateformat (Attachment II). (Para. 5.2.1) (look at QAPs 2-1, 2-3, 2-9, 16-1, and 17-1)

Verify that the QAPs (from Item #2) were reviewed by the appropriatemanagers. (Para. 5.3. 1A)


ITEMNO.

2

3

RESULTS

REV. 11/90



SHEET 3 OF 13

AUDIT/SURVEILLANCE

NO. HQ-93-03

C411"11 I WAA* 1; ' 0 * MSI --. liiii


-4- -

Verify that adequate review criteria are identified on the ProcedureReview Record (PRR) (Para. 5.3.2) (Reference Attachment ll Criteria)For QAP 2-3, paragraph 3.0, verify that review criteria for the 20references was factored into the review cycle. Also, evaluate theimpact of the non-availability of References 3.1.19 and 3.1.20(Methodology Documents).

Interview a sample of reviewers to ascertain their familiarity with thespecific requirements of Attachment 1II.

ITEMNO.

4

5

RESULTS

REV. 11/90



SHEET 4 OF 13

AUDIT/SURVEILLANCE

NO. HQ-93-03

* A A A I* -fiiiiiiiii - _


If a procedure review meeting was held in lieu of collecting individualPRRs, review meeting attendance memo to ensure all organizationshave provided Input. (Para. 5.3.3)

Review a sample of mandatory and non-mandatory comments andverify f they are consistent with the definition (Para. 3.2.10) Alsoverify if no mandatory comments are identified it is so noted on thePRR. (Para. 5.3.4A)


ITEMNO.

6

7

RESULTS

4- £ A

REV. 11/90



SHEET 5 OF 13

AUDIT/SURVEILLANCE

NO. HQ-93-03

I I T l


Verify that reviewing organizations consolidate their comments.(Para. 5-1, 5.3.4A)

Review any comments received after the assigned due date todetermine If they were Incorporated or resolved. If not, were thecomments mandatory? (Paras. 5.3.4B and 5.3.5)


ITEMNO.

8

9

RESULTS

+ 4-

REV. 11/90



SHEET 6 OF 13

AUDIT1SURVEILLANCE

NO. HQ-93-03

; . I . | . . | i l ;. | .;L . S i; 1 .fl ;0 i


-+ 1-

Verify that all mandatory comments were resolved on the PRR (or atreview meeting) or the matters were escalated. (Para. 5.3.6A)

Verify that agreed to mandatory comments were Incorporated Into thefinal document. (Para. 5.3.7)

ITEMNO.

10

1 1

RESULTS

REV. 11/90



SHEET 7 OF 13

AUDT1SURVEILLANCE

NO. HQ-93-03


Review the QAP Training Coordination sheet for recommendedTraining. (Para. 5.3.7 and 5.3.10)

Select a sample of personnel and verify that the recommendedtraining (classroom or reading) was completed and documented.(Para. 5.3.7) (Co-ordinate with Team C)


4- t

I 11~~~~~~~~~~~~~~~~~~~ A

ITEMNO.

12

13

RESULTS

REV. 1190



SHEET 8 OF 13

AUDIT/SURVEILLANCE

NO. HQ-93-03

I II


+ 4- 4-

Review QAP/ILP Review Sheets (Attachment VI II) for completion andadequacy. (Paras. 5.3.8B and C)

Verify that any mandatory comments received during the concurrenceprocess are reviewed by the Responsible Manager and the QAManager and that this review is documented and approved. (Para.5.3.9)

ITEMNO.

14

15

*

RESULTS

REV. 11190



I II I


Verify that the final approved QAP was submitted to DocumentControl (QAP-6-1) and a distribution list was provided to DocumentControl. (Para. 5.4.1)

Verify that a complete Records Package was submitted to the LRC(QAP-1 7-1). The package shall contain:

* PRRs (mandatory and nonmandatory)* Nonmandatory comments* Memo* Draft submitted for review* Procedure review meeting notifications* Procedure meeting memo with markups* Comments made during concurrence review* All drafts beyond preliminary draft stage

(Para. 5.4.2 and 6.0)


SHEET 9 OF 13

AUDIT/SURVEILLANCE

NO. HQ-93-03

ITEMNO.

16

17

RESULTS

REV. 11190



SHEET 10 OF 13

AUDIT/SURVEILLANCE

NO. HQ-93-03

* i A -L a *


I I.

Verify that QAP revisions are accomplished in accordance with(Paras. 5.5.2A and 5.2 through 5.4)

Verify that any cancelled QAPs have been cancelled In accordancewith (Para. 5.5.4)

ITEMNO.

18

19

RESULTS

REV. 11/90



Verify that any expedited Procedure Change Notices (PCNs) havebeen processed in accordance with (Paras. 5.6.1 through 5.6.5)

Review PCNs for proper authorizing signatures. (Para. 5.7.2)

SHEET 11 OF 13

AUDIT/SURVEILLANCE

NO. HQ-93-03

I I A I I -


ITEMNO.

20

21

*

RESULTS

REV. 11/90

,



SHEET 12 OF 13

AUDITISURVEILLANCE

NO. HQ-93-03

I I i- I - -


Verify that the Document Control Center has issued PCNs to allcontrolled procedure recipients. (Para. 5.7.3)

Verify that PCN records are sent to the LRC In accordance with QAP-17.1. (Para. 5.7.4)


ITEMNO.

22

23

RESULTS

REV. 11/90



SHEET 13 OF 13

AUDtT/SURVEILLANCE

NO. HQ-93-03


+ +

Verify that QARD ICNs resulted in a documented (memo or PRR)review of applicable procedures. Also verify that the reviewdocumentation was processed at the LRC.

A I

ITEMNO.

24

RESULTS

REV. 11/90

QAP-5-1 PROCEDURE CHECKLIST TABLE

REVIEW OF MANDATORYFORMAT REVIEW ADEQUATE REVW CONSOL MANDATORY COMMENTS

QAPALP REV. OK? LIST OK? CRITERIA? REVIEWER INTERVIEWS MTG? COMTS? COMMENTS RESOLVED?

OAP-2-1 0

QAP-2-3 0

QAP-2-9 2

CAP-16-1 0

GAP-1 7-1

CHKLIST 2 3 4 5 6 8 9 10QUEST# I .

M&O AUDIT - fQ-93-01 (211-215193) - CHKLST DATED - 1/21193

QAP-5-1 PROCEDURE CHECKLIST TABLE

MANDATORY TRAINING QAPALP CONCUR TO DOCCOMMENTS COOR TRAINING REVIEW SHTS CMNTS CONTRL & RECORDS AT

QAPILP REV. INCORP? SHT OK? COMPLETED? OK? OK? DISTBD? LRC & COMP?

QAP-2-1 0

QAP-2-3 0

OAP-2-9 2

OAP-16-1 0

QAP-1 7-1

CHKLIST 11 12 13 14 15 16 17QUEST #

M&O AUDIT - HQ-93-01 (2/1-2/5/93) - CHKLST DATED - 1/21/93

PA&E 2

1---~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~'



* A A -A ~ ~ ~ ~ --


M&O - Vienna, VA

DATES OF EVALUATION

February 1-5, 1993

[x] EXTERNAL

]INTERNAL

[x] AUDIT

[ I SURVEILLANCE PREPARED BY Marlin Horseman/Hank Greene DATE 1/1 5193

CONTROLUNG DOCUMENT (Title, Number, Revision) ACTIVITY EVALUATEDQAP-6-1, "Document Control", Rev. 1 Document Control


1 Review a listing of controlled documents for:

* Engineering * QA * Software* Procurement * Records * Others

Ensure that all appropriate procedures are identified as controlleddocuments. (Para. 5.2.1)


REV. 09/91

PAGE 1 OF 5

AUDT/SURVEILLANCE

NO. HQ-93-03

v



SHEET 2 OF 5

AUDIT/SURVEILLANCE

NO. HQ-93-03

I I 1 d i iS 6 | ;^ | ;; | "31 --


Follow the documentation trail for several procedures to ensurecompliance with (Paras. 5.2.1 through 5.3.11)

Sample various procedural sets of various personnel to ensure theyhave made the latest changes as identified by document control.(Paras. 5.3.6 through 5.3.8)


ITEMNO.

2

3

RESULTS

A I

REV. 11/90



SHEET 3 OF 5

AUDITSURVEILLANCE

NO. HQ-93-03

I A A 1; 0 * 0 i -

If I I


Verify that a list of controlled documents has been prepared annuallyand distributed. Also verify that individual recipients received a listingof the controlled documents assigned to them. (Paras. 5.4.1 through5.4.2)

Review records packages have been submitted to the LRC.(Para. 6.1)


ITEMNO.

4

5

RESULTS

I .1.

REV. 11/90



SHEET 4 OF 5

AUDRI/SURVEILLANCE

NO. HQ-93-03

* A A A * --~~01 (

I I Ia


4 4 4

Discuss the adequacy of M&O procedure 6-1.

*

RESULTSITEMNO.

6

REV. 11/90



PAGE 1 OF-

AUDIT/SURVEILLANCE

NO. HQ-93-03

9 I i *; s J :- ', - ' i L


M&O - Vienna, VA

DATES OF EVALUATION

2/1-5193

[x] EXTERNAL

[ ] INTERNAL

[x ] AUDIT

[ ] SURVEILLANCE PREPARED BY Marlin Horseman/Hank Greene DATE 1/27193

CONTROLLING DOCUMENT (Title, Number, Revision) QAP 17-1, Rev. 2, ACTIVITY EVALUATEDProgram Records Management: Record Source Responsibilities Records Management


Kecords yA1 Verify that procedures containing Activities Affecting Quality (MQ) Procedure Rev Desig. Indicated

identify records to be initiated and maintained. (Para. 4.3.1) Do theseprocedures identify which are QA Records?


REV. 09/91

8

I I



SHEET 2 OF 8

AUDIT/SURVEILLANCE

NO. HQ-93-03

9 4 - 4 -;;;. Si iai^.ll:.: | .:i, AL ;;&1 ^,, ; | "^Si1~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~I IIkrino ii iI


Sample on-going AAQ and determine records generated (e.g. CARsAudit Reports, etc.). For completed records (authenticated) have theybeensenttotheLRC? (within10days?) (Para.5.3.1) Wasarecordtransmittal properly prepared? (Para. 5.3.1)

For the selected sample, verify that paragraph 5.1 requirements forrecord identification have been met. (Paras. 5.1.1. through 5.1.6 and5.1.9)


Record Comp? Xmittal? 10 days LRC?

ITEMNO.

2

3

RESULTS

.1-

REV. 11/90

_



SHEET 3 OF 8

AUDIT/SURVEILLANCE

NO. H-93-03

4 I :1 d - 0& an l ii'I I Im


Through interviews and observation verifythat in-process records heldby record sources are being protected from loss and damage.

Evaluate any corrections requested by the LRC of the RecordsSource. Were requirements met? Paragraphs 5.6.1 through 5.6.7,as applicable, and QAP-17-2, paragraph 5.13.7.


ITEMNO.

4

5

RESULTS

REV. 11/90

\1 _1 - v_

_



SHEET 4 OF 8

AUDIT/SURVEILLANCE

NO. H-93-03

*~ ~~~s2 AS A A- - 0 -.:l S:l &


Verify that record packages are being properly validated inaccordance with paragraphs 5.3.2 through 5.4.3. Sample previouslyvalidated packages for adequacy. Verify that the validators are on thelist at the LRC.

Determine if any records have been lost. If so, were they processedin accordance with paragraph 5.1 and OAP-17-2, paragraphs 5.17and 5.18.


RESULTS

-� ______________________________________ _________

REV. 11190

ITEMNO.

6

7



SHEET 5 OF 8

AUDITISURVEILLANCE

NO. HQ-93-03

A ~ ~~~ _ A ^ *j SS -- ; l


1- 4

Verify (for any obliterated or missing or damaged records) that animpact statement has been prepared, signed and dated). (Para.5.1.7)

Determine if any one-of-a-kind" records (Definition in QAP-17-2,3.2.10) have been processed. If so, is there a descriptive document?(Para. 5.3) What happened to the one-of-a-kind" record?

J -L

ITEMNO.

8

9

RESULTS

REV. 11/90



SHEET 6 OF 8

AUDIT/SURVEILLANCE

NO. HQ-93-03

_* 3A A 3 -3 _


+ 1 4

Determine if any privileged records have been processed inaccordance with Paragraphs 5.3 and 5.5.2 and Paragraph 5.1.3.

Evaluate the record retrieval system controls by requesting trainingrecords (System 80) and non-system 80 records. Paragraphs 5.5.1through 5.5.2 and QAP-17-2, paragraph 5.15.

ITEMNO.

10

11

RESULTS

REV. 11/90



SHEET 7 OF 8

AUDIT/SURVEILLANCE

NO. HQ-93-03

_ A-, £ Au ; *lI""L:^i15SI


Review to determine if the maximum period that records may bestored in temporary storage is identified in procedures (ASME NQA-117, paragraph 4.4.3. Verify that this requirement is being met.

Verify that M&O procedures and OCRWM ILP 12.17.01 adequatelydefine interfaces between the CRF and the OCRWM QRC.


C I.

ITEMNO.

12

1 3

RESU LTS

REV. 11/90

OFFICE OF CIVILIANRADIOACTlVE WASTE MANAGEMENT


SHEET 8 OF 8

AUDIT/SURVEILLANCE

NO. HQ-93-03


Review procedural questions with records management. Commentsare recorded in procedures. Are any clarifications needed?

I.


I~~~~~~~~~~~~~~~~~~~~

ITEMNO.

14

RESULTS

REV. 11190

l l l~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



PAGE 1 OF 5

AUDITISURVEILLANCE

NO. HQ-93-03

-- A


M&O - Vienna, VA

DATES OF EVALUATION

February 1-5, 1993

[x] EXTERNAL

[ ]INTERNAL

[x ] AUDIT

[ ] SURVEILLANCE PREPARED BY Marlin Horseman/Hank Greene DATE 1/25/93

CONTROLLING DOCUMENT (Title, Number, Revision) QAP-17-2, Rev. 0, Program ACTIVITY EVALUATEDRecords Mgmt: Receipt & Hnding of Prog. Records & Records Packages & PCN Records Management


1 Verfy that records in the LRC are protected against loss and damage.(Paras. 5.1.1 through 5.1.6) Note: QAP-17.2, paragraph 5.1.1 says storein 1 hr. cabinets; QAP 17-3, paragraph 5.12 says 2 hrs; OAP 17-5 says1 hr at CRF; and OCRWM ILP 12.17.01 says 1 hour.


REV. 09191

k

1��

I

\. I



SHEET 2 OF 5

AUDIT/SURVEILLANCE

NO. HQ-93-03

II


Determine if any records have been lost. If so, were they processedin accordance with QAP-17-1, paragraph 5.1 and QAP-17-2,paragraphs 5.17 and 5.18.

Determine if any Electronic Records have been processed. If so,were the requirements of Attachment II and paragraphs 5.3 and 5.12met?

REMARKSRecord objective evidence reviewed, methodof veriication, personnel contacted

ITEMNO.

2

3

RESULTS

REV. 11190

'I



SHEET 3 OF 5

AUDIT/SURVEILLANCE

NO. H-93-03

I C C 0 -.

2~~~~~*]" 1*01-10U1 :' a_MM117' I-,.


Evaluate any corrections requested by the LRC of the RecordsSource. Were requirements met? QAP-17.1, paragraphs 5.6.1through 5.6.7 as applicable and QAP-17.2, paragraph 5.13.7.

Evaluate the record retrieval system controls and adequacy byrequesting training records (System 80) and non-system 80 records.QAP-1 7.1, paragraphs 5.5.1 through 5.5.2 and QAP-17-2, paragraph5.15.


I

ITEMNO.

4

5

RESULTS

REV. 1190

OFFICE 'OVICIVILIANRADIOACTIVE WASTE MANAGEMENT


SHEET 4 OF 5

AUDITISURVEILLANCE

NO. H-93-03

nS-0i .-i-;.1i@ R.m __


Review QA records packages submitted to the CRF. Do the recordsreceive an accession number in accordance with para. 5.8?

Verify that the records packages are arranged in accordance with theaccession number (Para. 5.10.1). Does this arrangement comply withthe Table of Contents? Is the arrangement logical? (Para. 5.13.6)


ITEMNO.

6

7

RESULTS

REV. 11/90



SHEET 5 OF 5

AUDIT/SURVEILLANCE

NO. HQ-93-03

I


Verify that QA records packages to be submitted to the CRF complywith the requirements of para. 5.13.6.


ITEMNO.

8

RESULTS

REV. 11/90



PAGE 1 OF 2

AUDMTISURVEILLANCE

NO. HO-93-03

S A - A


M&O - Vienna, VA

DATES OF EVALUATION

2/1-5/93

[x] EXTERNAL

[ ]INTERNAL

Ix] AUDIT

[ I SURVEILLANCE PREPARED BY Marlin Horseman/Hank Greene DATE 125/93

CONTROLLING DOCUMENT (Ttle, Number, Revision) QAP-17-4, Rev. 0, ACTIVITY EVALUATEDProgram Records Management: Microfilming Program Records Records Management


1 In light of information received during HQ-93-01 audit verify that the CRFis microfilming records. If not, verify that C/As are being vigorouslypursued and that records are being properly stored within the temporarystorage guidelines.


REV. 09/91

' I > 'S - ' - >V



I I I I


Verify that film boxes containing the undeveloped microfilm (silvermasters) are labeled in accordance with paras. 5.4.12 and 5.6.

Review the methods for performing the microfilm quality reviewidentified in para. 5.7. Are checklists used? Are results documentedin the Microfilm Tracking Log?

SHEET 2 OF 2

AUDIT/SURVEILLANCE

NO. H-93-03

REMARKSRecord objective evidence reviewed, methodof verfication, personnel contacted

.1.

ITEMNO.

2

3

RESULTS

REV. 11/90

-

I



PAGE 1 OF 3

AUDITSURVEILLANCE

NO. HQ-93-03

I I. ' - ' s A-. I


M&O-Vienna, VA

DATES OF EVALUATION

February 1-5, 1993

[x] EXTERNAL

[ ]INTERNAL

fx AUDIT

[ ]SURVEILLANCE PREPARED BY Marlin Horseman/Hank Greene DATE 1/25/93

CONTROLLING DOCUMENT (Title, Number, Revision) QAP-17-5, Rev. 0, ACTIVITY EVALUATEDProgram Records Management: Indexing Program Records

REMARKSITEM CHARACTERISTICS TO BE EVALUATED Record objective evidence reviewed, method RESULTSNO. of verffication, personnel contacted

1 Verify that QA records in the CRF are protected and System 80 recordsare secured. (Para. 5.1)


REV. 09/91

-1 - '� " 7

I



SHEET 2 OF 3

AUDIT/SURVEILLANCE

NO. HQ-93-03

; C __ I _ -


Verify that the "OCRWM Indexing Manual" is used to index records.

Verify that the quality control review of indexing activities is performedin accordance with para. 5.4.


=1ITEMNO.

2

3

RESULTS

REV. 11190

\ i



SHEET 3 OF 3

AUDITISURVEILLANCE

NO. Ho-93-03

_== I 1_~~~~~~~~~~~~~~~~~~~~~~


Verify that as records are received from the Receipt and HandlingStaff at the CRF they are entered on a Batch Sheet (Attachment 1)(Para. 5.2.1)

Veriy that prior to microfilming staff members perform an up-frontquality review of each record and record package to ensure thatprescribed procedures (screening/accessioning) have beenappropriately applied. (Para. 5.2.3)


ITEMNO.

4

5

RESULTS

REV. 1190



PAGE 1 OF 5

AUDIT/SURVEILLANCE

NO. HQ-93-03

K I I -1111 1111


M&O-Vienna, VA

DATES OF EVALUATION

February 1-5, 1993

[x] EXTERNAL

[ ]INTERNAL

[x] AUDIT

[ ]SURVEILLANCE PREPARED BY Marlin Horseman/Hank Greene DATE 1/25193

CONTROLLING DOCUMENT (Title, Number, Revision) QAP-1 7-6, Rev. 0, ACTIVITY EVALUATEDProgram Records Mgmt: Storage, Retrieval & Disposition of Program Records Records Management


1 Review to determine where the requirements for record disposal areidentified. Also, the classification of records (Lifetime or non-permanent)per ASME NQA-1 17 (2.7). Verify these requirements are being met.(Paras. 5.12.3 and 5.13)

*INDICATE RESULTS: SATISFACTORY (SAT), UNSATISFACTORY (UNSAT), NOT APPLICABLE (N/A)

REV. 09/91

I'�_ '_/

I

< I/



SHEET 2 OF 5

AUDITISURVEILLANCE

NO. HQ-93-03

* £ A -' -11 1, , - . .~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


Review Central Records Facility (CRF) storage practices.

a. Records stored by the M&O Contractor. (Para. 5.1.2)b. (CRF) Temporary storage while records are being

microfilmed. (Para. 5.2.2)c. PMC (Program Microfilm Center) temporary storage of

processed silver microfilm awaiting shipment for permanentstorage. (Para. 5.2.3)

d. Restricted-access storage facility for privileged records notin use. (Paras. 5.1.4 and 5.2.4)

e. One-of-a-kind records. (Paras. 5.6 and 5.1.3)f. Electronic Records. (Para. 5.7)

Review CRF verification practices

a. Verify that document records packages and diazo microfilmare reviewed for correctness. This review shall bedocumented. (Para. 5.4.1)


ITEMNO. RESULTS

REV. 11/90

_�' _-2-

I



SHEET 3 OF 5

AUDIT/SURVEILLANCE

NO. HQ-93-03

I I


--4- 4 4

a. Verify that access to records is limrted and that a current listof names of authorized staff members is posted. (Para. 5.8)Also, verify that an authorization list is at the area for storageof completed hard copies. (Para. 5.12.2)

b. Verify that processed microfilm rolls of privileged records areproperly labelled and segregated. (Para. 5.4.2)

Verify that storage of silver master microfilm is accomplished inaccordance with Para. 5.5.1 through 5.5.3.

ITEMNO.

4

5

RESULTS

REV. 11190

'K / I



SHEET 4 OF 5

AUDIT/SURVEILLANCE

NO. H-93-03

9 & A*M q0 I iIII


1 1

Verify that the permanent storage facility has been notified ofpersonnel authorized to have access to permanent storage facilities.(Paras. 5.8.3 and 5.1.5)

Verify that permanent records storage facilities have been audited bythe M&O in accordance with para. 5.2.1. Also verify that the storagefacility for the second copy of the Program record meets therequirements of paras 5.2.1 and 5.2.2

ITEMNO.

6

7

RESULTS

REV. 11(90

" , " i



SHEET 5 OF 5

AUDIT/SURVEILLANCE

NO. HO-93-03

p4 L * 0 EC -.


Verify that the CRF Manager twice yearly reviews hard copy recordsinventory and provides OCRWM with status report of hard copyrecords in storage for six months or longer. (Para. 5.12.3)

Verify that referenced Government and industry standards inprocedures are understood by personnel performing the activity. (e.g.QAP-17-6, paragraphs 5.7.6 and 5.10.1)


4

ITEMNO.

8

9

RESULTS

I- I

REV. 11/90

t4�