SMS Model Gap Analysis (３)

SMS GAP ANALYSIS FORM

1. SAFETY POLICY AND OBJECTIVES

1.1 – MANAGEMENT COMMITMENT AND RESPONSIBILITY

1.2 – SAFETY ACCOUNTABILITIES OF MANAGERS

1.3 – APPOINTMENT OF KEY SAFETY PERSONNEL

1.4 – SMS IMPLEMENTATION PLAN

1.5 – COORDINATION OF THE EMERGENCY RESPONSE PLAN

1.6 – DOCUMENTATION

2. SAFETY RISK MANAGEMENT

2.1 – HAZARD IDENTIFICATION PROCESSES

2.2 – RISK ASSESSMENT AND MITIGATION PROCESSES

2.3 – INTERNAL SAFETY INVESTIGATIONS

3. SAFETY ASSURANCE

3.1 – SAFETY PERFORMANCE MONITORING AND MEASUREMENT

3.2 – THE MANAGEMENT OF CHANGE

3.3 – CONTINUOUS IMPROVEMENT OF THE SAFETY SYSTEM

4. SAFETY PROMOTION

4.1 – TRAINING AND EDUCATION

4.2 – SAFETY COMMUNICATION

November 20061

Gap Analysis Model Form

Element Number

ICAO SMS Framework

Response(Yes/No)

If yes, state where the requirement is

addressed.

Describe actions needed to establish SMS compliant element

Person responsible for

action

Estimated completion

DateSAFETY POLICY AND OBJECTIVES

1.1 Is a safety management system with defined components established, maintained and adhered to?Evidence of Completion:

No A SMS document has been drafted but not yet approved. Many parts of the system infrastructure are in place but adherence to requirements is not yet adequate.

Complete all required documents. Establish logical procedure framework. Establish improved accessibility of all procedures for affected departments and their employees

Chen shengSun Li 9/30/2010

Is the safety management system appropriate to the size and complexity of the organization? Evidence of Completion:

No The system that is in place is not yet mature and may be overly complex given the size of the overall operation.

Simplify/consolidate procedures. Identify opportunities to reduce requirements in procedures that may not be absolutely necessary based on real risks.


Is there a safety policy in place?

Yes Policy 4 outlines the basic expectations and RAA’s

No further action needed

Have safety objectives been established?

No Specific objectives have not been identified by LT, have not been communicated and Metrics have not been selected and published.

Establish and communicate safety objectives and metrics that measure progress to achieving each objective.

Andrew Ho (in consultation with LT)

6/30/2010

November 2006 2

Are safety objectives publicized and distributed?

No Cannot be communicated if not yet established by LT

Establish communication schedule and plan for objectives. Create SMS badge card that lists objectives

Chen Sheng 7/30/2010

Is there a formal process to develop a coherent set of safety goals?

No There is no process for planning by LT, nor process for identifying goals based on risk and objectives

Establish SMS goals and include goals on SMS badge card to ensure communication to employees

Andrew Ho (in consultation with LT)

6/30/2010

Is there a formal process to develop and maintain a set of safety performance indicators and safety performance markers?

No Only early states for establishing outcome performance measures for occ. Safety, none for aviation safety or environment. No planning and review a part of Quality and Safety meeting

Formally establish metrics for goals and objectives. Communicate progress to goals and objectives during all hands meetings. Also post metrics performance on employee bulletin boards.


Has the organization based its safety management system on the safety policy?

Partial Policy and procedures are generally aligned, however little thought has gone into the strategy for reliable deployment and ongoing checking by LT. current process is mostly ad hoc

Establish policy and procedure framework that is in harmony of the SMS framework.

Business Management

8/30/2010

Is the safety policy approved by the accountable executive?

Yes Policy 4 coordinated with LT and approved by CEO

No further action required

November 2006 3

Is the safety policy promoted by the accountable executive?

Yes CEO is engaged in ongoing review of safety program elements, issues and corrective actions. Accountability for LT could be improved.


Is the safety policy reviewed periodically?

Yes There is a schedule for review of all safety documents including policies and procedures


Is there a policy in place that ensures that employees are free to report safety deficiencies, hazards or occurrences without being subject to unjust discipline?

PartiallyNo There is only one procedure involved the concept of without bearing the unjust discipline, Pro 4-M13 Employee concern report.Policy is in draft but not yet deployed or communicated

We still need to improve voluntary reporting system, and improve the incentive system.Publish Employee Concerns procedure and deploy process

Chen ShengSun Li 5/15/2010

Does the accountable executive have responsibility for ensuring that the safety management system is properly implemented and performing to requirements in all areas of the organization?

Partially CEO is engaged through weekly meetings and ongoing reviews. However, without objectives and metrics it is difficult for him to determine true performance.

See response to requirements for objectives and goals

Andrew Ho 8/15/2010

November 2006 4

Does the accountable executive have control of the financial and human resources required for the proper execution of their SMS responsibilities?

Yes Overall budget is within the control of the CEO, could be improved by establishing specific safety budget lines in appropriate departments.

May wish to establish a detailed budget for Safety/EHS so that there are discrete budget lines to meet all anticipated spending needs.

Has a qualified person been appointed to oversee the operation of the SMS?

PartiallyNo We have the qualify auditor and safety engineer and F/S person and department safety focus persons to ensure the implementation of safety issue. Assignment is informal at this time.

We haveHire Aviation Safety Engineer to oversee SMS project, but still need to offer more training opportunity to the safety management and safety focus to improve their ability of surveillance.

Does the person overseeing the operation of the SMS fulfill the required job functions and responsibilities?

PartiallyNo Without formal assignment with RAA’s and BG&O’s job functions remain ambiguous

Establish clear RAA’s in SMS procedure under responsibilities section

Are the safety authorities, responsibilities and accountabilities of personnel at all levels of the organization defined and documented?

Partially Continued flux in many organizations so lines of responsibilities and reporting are ambiguous

Establish and publish a stable organization and reporting structure. RAA’s must be integrated into procedures in the responsibilities section.Support periodically safety training for the department safety focus person and other responder

November 2006 5

Do all personnel understand their authorities, responsibilities and accountabilities in regards to all safety management processes, decisions and actions?

No No formal program kickoff and training has yet occurred

Establish training curriculum and information plan for all employees and ensure their roles, authority, responsibilities and accountabilities are communicated and understood.

Does the organization have an emergency response procedure appropriate to the size, nature and complexity of the organization?

Partially A procedure has been published but training, drilling, and communication have not yet occurred

Establish training curriculum and drill schedule.

Have the emergency response procedures been documented, implemented and assigned to a responsible manager?

Partially A responsibility matrix has been created and some communication has occurred, however more training and communication are needed

See above

Have the emergency response procedures been periodically reviewed as part of the management review of the SMS, and after key personnel and organizational change?

PartiallyNo There is no process in place to assure proper maintenance of assignments, no specific person has been assigned to establish the process and maintain it. Though we has release the ＥＲＭ　procedure.

Establish a documented, reliable process that ensures that key role players are maintained even when organization changes and personnel changes occur

November 2006 6

Does the organization have a process to distribute the emergency response procedures and to communicate the content to all personnel?

PartiallyNo There is a lack of clarity in the responsibility for the publishing and communication of new and updated command media.

Establish rational procedure frame work and ensure that RAA is establishing for its maintenance and appropriate department assumes responsibility.

Has the organization conducted drills and exercises with all key personnel at specified intervals?

No No evidence of drill schedule, no table top exercises have been performed, not all drills for all possible scenarios have been planned, scheduled or performed

Establish detailed drill schedule and plans for the performance of drills including after action review process.

Has a documented procedure been established and maintained for identifying applicable regulatory requirements?

No No clear strategy for establishment of procedures, the framework for procedures appears to have some misalignment.

Regulatory review must be integrated into the policy and procedure process to ensure that all procedures incorporated regulatory and legislative changes when procedures undergo their scheduled updates.　

Are regulations, standards and exemptions periodically reviewed to ensure that the most current information is available?

PartiallyYes

All procedures are reviewed to a schedule. There is no specific process established to monitor regulatory changes that might trigger a needed procedure change

Ｔｈｅ　ｐｒｏｃｅｄｕｒｅｓ　ｏｗｎｅｒ　ｓｈｏｕｌｄ　ｂｅ　ｒｅｑｕｉｒ　ｔｏ　ｒｅｇｕｌａｔｏｒｙ　ｒｅｖｉｅｗ　ａｎｄ　ｋｅｅｐ　ｔｈｅ　ｐｒｏｃｅｄｕｒｅ　ａｖａｉｌａｂｌｅ．Ｍｙ　ｂｅ　ｗｅ　ｎｅｅｄ　ａ　ｐｒｏｃｅｄｕｒｅ　ｌｉｋｅ　ｓｔａｎｄａｒｄ　ｍａｎａｇｅｍｅｎｔ　ｐｒｏｃｅｄｕｒｅ．No additional action needed

November 2006 7

Is there consolidated documentation that describes the SMS and the interrelationships between all its components?

PartiallyNo There is ｏｎｌｙ　ｏｎｅ　ｐｒｏｃｅｄｕｒｅ　ｉｎｖｏｌｅｄ　ｔｈｅ　ｐａｒｔｉａｌｌｙ　ｓｍｓelementsThere is no mapping of policies, processes and procedures for all SMS elements

Create a map of all SMS elements showing their relationship to various documentation including policies, procedures, curricula, manuals and online resources. Publish this map on Boeing Shanghai website.

Does this information reside or is it incorporated into approved documentation, such as Company Operations Manual, Maintenance Control/Policy Manual, Airport Operations Manual, as applicable, and where these approved documents are not required by regulation, the organization includes the information in a separate, controlled document?

Yes There are such manuals available.

November 2006 8

Does the organization have a records system that ensures the generation and retention of all records necessary to document and support operational requirements, and is in accordance with applicable regulatory requirements and industry best practices?

No Current records retention and storage is not well established nor is there a clear process owner. Records management is very poor.

Establish a written procedure and master retention schedule of records. Establish an improved records indexing and storage process.

Does the system provide the control processes necessary to ensure appropriate identification, legibility, storage, protection, archiving, retrieval, retention time, and disposition of records?

No No retention matrix exists, the importance of records management has not been communicated, no system for archiving and indexing has been established.

See above

Does the organization have a reactive process or system that provides for the capture of internal information including incidents, accidents and other data relevant to safety and risk management?

Yes The MET process captures on a reactive basis any safety issues and tracks them through completion. The incident reporting process has been established by procedure, compliance is not completely reliable.

November 2006 9

Is the reactive reporting process simple, accessible and commensurate with the size of the organization?

Yes The MET system is easy to access and use. The incident reporting process is a manual paper process that is appropriate for the size of the operation and the frequency of incidents.


Are reactive reports reviewed at the appropriate level of management?

Yes MET items are reviewed by LT monthly. Some improvement is needed for review by Operations


Is there a feed back process to notify contributors that their reports have been received and to share the results of the analysis?

Yes The MET system automatically distributes initial information and updates


Is there a process in place to monitor and analyze trends?

No This is in work Establish ｒｉｓｋtrend analysis ｄａｔｅ　ｂａｓｅ　ａｎｄ　ａｎａｌｙｓｉｓ　ｍｏｄｅｌ／ｓｍｓ　ｍａｎｕａｌ，ｔｈｅｎ　　ｗｅ　ｃａｎ　ｃｏｎｄｕｃｔ　ｔｈｅ　ｒｉｓｋ　ａｎａｌｙｓｉｓ　graphs for monthly review during Quality/Safety meeting.

Are corrective and preventive actions generated in response to event analysis?

Yes MET system requires CAPA, however quality of analysis needs improvement


November 2006 10

Does the organization have a process or system that provides for the capture of internal information including hazard identification, occurrences and other data relevant to safety?

Partial MET captures audit and incident information, but there is no means for capturing employee concerns. There is a process in draft but not yet deployed

Approve employee concerns procedure and deploy process. Establish analysis charts showing trends that require additional action and use this data to create MET items.

Is the proactive reporting process simple, accessible and commensurate with the size of the organization?

No Employee concern report process is in draft but not deployed yet

See above

Is there a structured process for the assessment of risk associated with identified hazards, expressed in terms of severity, and probability of occurrence?

No No systematic Establish a schedule process for performing risk analyses that includes methods, data sources and action thresholds.

Are there criteria for evaluating risk and the acceptable level of risk the organization is willing to accept?

No Current process is completely ad hoc

See above, establish criteria for risk action threshold based on cost, morale, reputation, compliance and business impacts.

Does the organization have risk management strategies that include corrective/ preventive action plans to prevent recurrence of reported occurrences and deficiencies?

No There is no existing proactive risk management process for use of MET

Create connection between risk management process and MET. Document connection in MET manual and other appropriate procedures.

November 2006 11

Are regular and periodic, planned reviews conducted regarding company safety performance, internal audit results, hazard and occurrence investigations, hazard and occurrence analysis results, internal/external feedback analysis/results, status of corrective actions, follow-up actions from previous management reviews, changes that could affect safety, recommendations for improvement and sharing of best practices across the organization?

Yes Audit results, action items, issues, corrective actions and improvement opportunities are reviewed regularly by LT during Quality/Safety meetings


Is there a process to evaluate the effectiveness of corrective actions?

Yes Corrective action effectiveness is a requirement for closure of MET items


Are proactive reports reviewed at the appropriate level of management?

No Employee concern reports will be reviewed during Quality/Safety Meetings once the process is implemented

Implement Employee Concerns Report process

Is there a feedback process to notify contributors that their reports have been received and to share the results of the analysis?

Yes Draft procedure for employee concerns includes provisions for communicating disposition of reports


November 2006 12


No No process for trend analysis of employee concerns has been documented

Establish trend analysis similar to trend analyses of MET items

Has the organization planned self-evaluation processes, such as regularly scheduled reviews, evaluations, surveys, operational audits, assessments, etc.?

No No procedure exists for performance of assessments and audits, nor have a set of audit protocols yet been created.

Draft and approve program assessment process procedure. Create a complete set of audit protocols and post the protocols on the Boeing Shanghai website

Are corrective and preventive actions generated in response to risk analysis?

No No risk analysis process has been established

Include requirements for MET items to be generated when actions are required as a result of negative trends identified from risk analyses

Is a process in place for analyzing changes to operations or key personnel for risks?

No There is no formal process for determining operational impact when work process changes, personnel changes or facility/equipment changes occur

Establish and document an appropriate change management process that includes impact evaluation and mitigation for significant changes to staffing, organization structure, production process changes or changes to facilities.

Are there procedures in place for the conduct of investigations?

Yes The procedure incident notification and investigation has been deployed including MEDA requirements

Additional training may be required to ensure that managers and leads are proficient in the use of investigation methods and tools

Do measures exist that ensure all reported occurrences and deficiencies are investigated?

Yes See above No further action required

November 2006 13

Is there a process to ensure that occurrences and deficiencies reported are analyzed to identify all associated hazards

No There are requirements to do root cause but no standard tools or training have been created

See above action for training in conduct of investigations

Are corrective and preventative actions generated in response to event investigation and risk analysis?

Yes The standard process for corrective action of safety issues is MET, it has a very good process for capturing and reporting corrective actions


Does the organization have a process for evaluating the effectiveness of the corrective/ preventive measures that have been developed?

No There is no systematic method for evaluating corrective actions to determine effectiveness and for assuring a complete and permanent fix.

Establish quality criteria for evaluating corrective action effectiveness related to incident investigations

Are corrective/ preventive actions, including timelines, documented?

Yes MET captures that information, however, use of the system is still undisciplined


Does the organization conduct reviews and audits of its processes, its procedures, analyses, inspections and training?

Yes However, this process is not yet systematic nor is there a complete set of audit protocols aligned to all procedures

See action concerning establishment of assessment process procedure and audit protocols

November 2006 14

Does the organization have a system to monitor the internal reporting process and the associated corrective actions?

Yes Corrective actions in MET are monitored by the LT as part of the Quality/Safety reviews


Is there an operationally independent audit function with the authority required to carry out an effective internal evaluation program?

Yes The audit function is in the Quality /Safety organization which is independent of production and Facilities. Quality reports directly to the CEO.


Does the audit system cover all functions, activities and organizations within the company?

Yes Coverage is based on risk. Audits include LM, HM, Backshop and Facilities


Are there defined audit scope, criteria, frequency and methods?

No The process is not yet systematic and well defined

See action above related to establishing an assessment process procedure

Are there selection/training process to ensure the objectivity and competence of auditors as well as the impartiality of the audit process?

No No selection and training process has been established. There is no method to assess auditor competence. Currently safety staff function as the auditors.

Establish auditor selection and training criteria in the proposed assessment process procedure

Is there a procedure for reporting audit results and maintaining records?

No See answer to records retention question

See above, reporting of results to be included in the procedure

November 2006 15

Is there a procedure outlining requirements for timely corrective and preventive action in response to audit results?

No No time limits or aspirational goals have been established for timeliness of corrective action. Many corrective actions languish unless specifically highlighted during the Quality/Safety Review

See above, include timeliness of corrective actions in the assessment process procedure

Is there a procedure to record verification of action(s) taken and the reporting of verification results?

Yes MET has data fields for verification of actions for CAPA items. However, no specific requirement to attach proof of corrective actions to MET.

Include in MET manual directions for attachment of verification records to MET items

Does the organization perform periodic Management reviews of safety critical functions and relevant safety or quality issues that arise from the internal evaluation program?

No There is no analytical risk assessment model established that would permit such a management review.

Incorporate periodic management reviews for safety critical items into the risk management process.

Are there communication processes in place within the organization that permit the safety management system to function effectively?

Yes Safety bulletins in the newsletter, Quality/Safety Meetings, MET, All Hands meetings, Webpages, internal servers, emails.


November 2006 16

Are communication processes (written, meetings, electronic, etc.) commensurate with the size and scope of the organization?

Yes Communication is relatively simple because facility is in one location and all employees are under one roof, easily accessible.


Is information established and maintained in a suitable medium that provides direction in related documents?

Yes Most information is in an electronic format, readily accessible by all employees.

Improve structure and access to procedures to simplify employee access

Is there a process for the dissemination of safety information throughout the organization and a means of monitoring the effectiveness of this process?

Partial Dissemination processes are based on the audience. Effectiveness is not currently measured.

Establish review process to measure effectiveness of information dissemination and flowdown.


No No metrics or sample methods have been established

Establish metrics that provide visibility of the effectiveness of communication related to safety

Are corrective and preventive actions generated in response to event analysis?

No Data analysis has not yet started, once analysis methods are selected, processes need to be established to make proper use of the data.

See earlier answer above concerning trend analyses. Ensure inclusion of communication effectiveness in these analyses.

November 2006 17

Is there a documented process to identify training requirements so that personnel are competent to perform their duties?

No PTP is in work, but does not address safety, no single training matrix for safety related training has been established.

Establish safety training matrix related to job titles and departments. Establish a regular review schedule to ensure that employees are up to date in their attendance of classes.

Is there a process that measures the effectiveness of training?

No Most safety training does not require an exam

Establish course effectiveness criteria and incorporate into the site training process. May include requirements for pre and post testing or a process for a test out option.

.Is the organization’s safety training incorporated into indoctrination training upon employment?

No There is no formal, standard orientation checklist or materials. Only quick review of safety manual. No shop specific safety orientation.

Improve current process for initial safety orientation for all new employees.

Is there emergency response and response training for affected personnel?

Yes An emergency response course exists for production and production support employees.

Perform needs assessment for training of other employees and establish appropriate course curriculum based on needs.

-----------------------------

November 2006 18

Documents

SMS Model Gap Analysis (３)