AQMS Accreditation Programs Lessons Learned

Company Confidential

Registration Management Committee (RMC)

1

AQMS Accreditation ProgramsLessons Learned

Auditor WorkshopAtlanta, GA

July 22-23, 2010


Atlanta, GAJuly 22-23, 2010

Objectives

•Provide an overview of the NCR’s identified from the witness audits.

•Present in process approach.

•Discuss what to expect when witnessing

2


Atlanta, GAJuly 22-23, 2010 3

•PREVIOUS NCRS:

– Assigning Auditors who either did not have the proper NACE code or AS91XX qualifications or potential conflicts (such as ITAR).

– Client not made aware of the OP assessor.

•AWARENESS:

– Verify auditors are qualified for the full dynamics of the audit.

– Communicate clearly with the client.

– Applicable to both witness audits and office assessments

Pre-Audit Assignment



•PREVIOUS NCRS: 19011, 6.4.1.c

– (4) The audit plan did not ID the organizations functional units and processes to be audited.

•AWARENESS:

– The audit plan ensure the client will have the proper resources available to the audit.

– Clearly demonstrates what processes are intended to be audited.


Pre-Audit Planning



•PREVIOUS NCRS: 17021. 9.3.2.1

– (2) Audit Planning not effective in assuring the AQMS is assessed to the min requirements for surveillance audits.

•AWARENESS:

– Ensure the auditors understand where the full auditing requirements are for surv audits if not clearly ID in the audit plan.


Pre-Audit Planning



•PREVIOUS NCRS: MD 5 and AS9104

– (5) Insufficient Auditor Days in the planning without clear justification.

•AWARENESS:

– If deviation…..JUSTIFY.

– If the audit is being witnessed, provide to ANAB in the pre-audit information.


Pre-Audit Planning



•PREVIOUS NCRS:

– Sampling part of an element and making a conclusion on the whole requirement.

•AWARENESS:

– Audit plan and report accuracy is critical.

On-site Audit Depth of auditing



•PREVIOUS NCRS:

– Auditing to the scope of certification is not adequate.

•AWARENESS:

– Ensuring the scope of certification, scope of audit and audit plan is consistent.

– Validate the exclusions. Is scope consistent with the QM?

– Ensure the address and scope on the Certificate is correct.




•PREVIOUS NCRS:

– Not following audit trails when objective evidence suggest otherwise.

•AWARENESS:

– A plan is a plan. The CB should follow the trail to its natural conclusion if potential findings are evident.

– The CB should keep head on a swivel, don’t ignore clear findings.




•PREVIOUS NCRS:

– Not fully verifying the effectiveness of the actions taken on nonconformities identified during the previous audit.

•AWARENESS:

– Ensure the evidence is more than a record review.




•PREVIOUS NCRS:

– Not fully auditing Outsourcing when applicable.

•AWARENESS

– Standard clearly requires “control of such outsourced processes shall be identified” and therefore subject to audit. Not just a document verification.

On-site Audit Understanding



•PREVIOUS NCRS:

– Changes to the capability listing of the client.

•AWARENESS

– Similar to the scoping discussion. The CB should verify any changes to the capability listing as this could affect the scope of certification or introduce new processes and technology.

On-site Audit Understanding



•PREVIOUS NCRS:

– Calling the Clients CA/PA system effective with ineffective/inadequate correction, root cause or corrective action.

•AWARENESS:

– The CB should deep dive the CA/PA system. Validate the information. If it doesn’t make sense….the CB should document the NCR.

On-site Audit Decisions



•PREVIOUS NCRS:

– Soft Grading

– Clearly stating findings during the audit and not raising the NCR or improper categorization.

– Accepting informal correction during the audit.

•LEARNINGS

– The CB should learn the definitions and follow them. Keep track of the “verbal” findings identified during the audit. Accepting correction to verbal findings without reporting is consulting.

– As an auditor it is not their job to justify an NCR down.




•PREVIOUS NCRS:

– Continuing the audit when the audit objectives are clearly unattainable.

•AWARENESS:

– Only the client can make the decision to continue an audit.

– There should be a clear conclusion that the objectives are unattainable and reported to the client with options.

» At the time this is realized not at the end!!




•PREVIOUS NCRS:

– Closing meeting does not address all of the requirements.

•AWARENESS

– The CB should clearly address the CA follow up and impacts on the existing cert.

– The CB should ensure the CA are presented to the correct requirements.

On-Site Audit Closing



•PREVIOUS NCRS:

– Report not in conformance with AS9014/AS91XX

•AWARENESS

– The report should include:

» Justification / explanation all NA.

» Clear evidence to support the conclusion.

» Clear evidence to support the SCOPE.

» Detailed notes


On Site/Off-Site Audit Reporting



•PREVIOUS NCRS: Improper closing of NCR’s and Improper Certification Decision.

•AWARENESS:

– Clear evidence of Correction, RC and CA.

– Ensure the NCR’s are closed with appropriate evidence PRIOR to the cert decision.

Post Audit NCR Closure



•PREVIOUS NCRS: OASIS data base admin /accuracy

•AWARENESS:

– Must have the CB log in to see OASIS information

– Verify

» Accuracy (reports, scoring, certificates)

» Timing

Office Assessments OASIS



•PREVIOUS NCRS:

– Transfer requirements not met.

•AWARENESS:

– Specific requirements for pre-transfer review

» Validating the certificate

» Collecting audit records for last full system (initial or re-cert) and subsequent surveillances.

» NCR’s should be closed (or closed by new CB before certification)

– If pre-transfer review is unsuccessful full audit must be conducted.

Office Assessments Transfers



•PREVIOUS NCRS:

– Improper qualification /competency of AS cert decision maker.

•AWARENESS:

– Cert decision maker must include someone with aerospace knowledge

– Must be independent of the audit being reviewed (including independent of the corrective action reviews)

Office Assessments Certification Decision Process



•PREVIOUS NCRS:

– Decision made with incomplete or inadequate certification information.

•AWARENESS:

– Records not available (report, CA’s, etc)

– Records not complete or inaccurate

Office Assessments Certification Decision Process



•PREVIOUS NCRS:

– Certificates are incorrect or inaccurate.

•AWARENESS:

– Scope statements not correct or misleading (not specific enough) or incorrect for multiple sites.

– Addresses incorrect or does not include all addresses for multiple sites.

– Text in regards to being accredited under the registration management program.

– Unaccredited certificates not allowed

Office Assessments Certification Documents (e.g. Certificate)



Questions