SAFETY & RELIABILITY SOCIETY
LESSONS LEARNED FROM 10 YEARS OF USING THE CSM FOR RISK EVALUATION
AND ASSESSMENT (CSMRA)
BYDRAGAN JOVICIC, IVAN LUCIC
AND PETE GRACEY
Slide 1
28TH APRIL 2021
The Safety & Reliability Society is a Licensed Member of the
Engineering Council for CEng and IEng Professional Registration
Slide 2
• Introduction
• The presentations:
• Background – Dragan Jovicic
• Infrastructure Manager – Ivan Lucic
• National Safety Authority – Pete Gracey
• European Rail Agency – Dragan Jovicic
• Q&A session
• Accessing the webinar recording and slides
• Feedback
Note: the Webinar is being recorded
Programme
VIEWING IN FULLSCREEN AND Q&A FACILITY
Fullscreen select buttons
Write message icon
Ask question icon
Click here to move the AV screen to the right
Click here to go to Full Screen/reduce or expand the chat area
To exit the event at the end click here
PRESENTATION 1
DRAGAN JOVICIC
European Rail Agency (ERA)
Slide n° 5Introduction to Reg. 402/2013 on the CSM RA – History, Initial
Objectives, Method – SaRS Webinar, 28 April 2021
Introduction to the EU Regulation 402/2013 on the
CSM for risk evaluation and assessment (CSM RA)
History & Context - Initial Objectives – Method
Safety and Reliability Society (SaRS) Webinar, 28 April 2021
Dragan JOVICIC, EU Agency for Railways
Slide n° 6Introduction to Reg. 402/2013 on the CSM RA – History, Initial
Objectives, Method – SaRS Webinar, 28 April 2021
History
Historicallyfirst EVENT legislator’s REACTION
then
❑ new laws❑ new codes of practice❑ new rules❑ new standards
Every country (27) had national
❑ technical & safety rules❑ standards & operational conceptsConsequences
❑ Differences in principles, technical standards, safety barriers and approaches to safety cultures hindering establishment of international transport by railways
❑ International transport dependent mostly on voluntary multilateral agreements (COTIF, RIV, …) and additional, very often unnecessary, checks, safety demonstrations & authorisations for cross-border operations
❑ To open railway market harmonisation of practices across EU became essential
Slide n° 7Introduction to Reg. 402/2013 on the CSM RA – History, Initial
Objectives, Method – SaRS Webinar, 28 April 2021
EU "New Approach" for creation of a single European railway marketUse of CABs for checking compliance with applicable requirements
Opening of railway market to competition requires the removal of
the historical national barriers
Technical harmonisation through common TSIs
Safety harmonisation through common approaches to safety management
Interoperability Directive 2008/57 → 2016/797
EC Verification of conformity to TSIs
(EU Rules)
Verification of conformity vs. National Rules
MS Certification+ Surveillance
Accred./Recogn.
Assessment of Reg. 402/2013
Accred./Recogn.
Bodies notified to EC & MS (NoBos)
Bodies designatedby MS (DeBos)
NSA/ERA(RUs/IMs)
ECM Certification Body (ECM CB)
AsBos vs. Reg. 402/2013
these are all Conformity Assessment Bodies (CABs)
Safety Directive 2004/49 → 2016/798
AsBos
SMS Certification+ Supervision
"New Approach" requires mutual recognition of independent conformity assessment by CABs
Slide n° 8Introduction to Reg. 402/2013 on the CSM RA – History, Initial
Objectives, Method – SaRS Webinar, 28 April 2021
Safety Directive 2004/49 repealed by Directive 2016/798Introduces concepts of both risk and “proactive risk management”
Safe Operation & Maintenance
(i.e. proactive risk management)
Activities undertaken by RUs, IMs
& ECMs
SMS/MSRisks
Put in place a certified
SMSMS
DO
CHECKACT
PLAN
Major shift in managing safety the railway operations, traffic and maintenance activities
❑ Past: compliance with well-established national rules, standards and legislation (→ technical differences and different approaches to safety among countries)
SMSMS
DO
CHECKACT/
ADJUST
PLAN
❑ New framework (based on concepts of risk):
RUs/IMs/ECMs fully responsible for safe management of their business through a proactive SMS/MS [PDCA]
SMS/MS certified by an acknowledged authority to attest company capability to manage safely all its activities, including the safe management of:
⚫ Technical;⚫ Operational, and;⚫ Organisational
changes
Slide n° 9Introduction to Reg. 402/2013 on the CSM RA – History, Initial
Objectives, Method – SaRS Webinar, 28 April 2021
Risk Manage_
ment
Processes
(Existing)Rules
Procedures
Human Factors
CSM for risk assessment & CSM for monitoring) Cornerstones/pillars of the SMS Risk Management
Risk Monitoring1078/2012
Risk Assessment
402/2013
DO
CHECKACT/
ADJUST
PLAN
Slide n° 10Introduction to Reg. 402/2013 on the CSM RA – History, Initial
Objectives, Method – SaRS Webinar, 28 April 2021
+
–
CSM for monitoring
CSM for risk assessment
(RCM - Risk Control Measures)
Railway System
Order E(p) Command U(p)
Action Plan
Actual System Performance
Predictive Risk Assessment
Operational Monitoring and Preventive/Corrective measures :
REGULATOR
Information on what to monitor & how to monitor it
Measure X (p)
Gap ε (p)
Corrector C(p) System under regulation G(p)
Sensor K(p)
Output S(p)
ACTION:
REACTION
Change
(Objectives)
Controlled Change Management Process based on the combined use of predictive Risk Assessment and Operational Monitoring
Slide n° 11Introduction to Reg. 402/2013 on the CSM RA – History, Initial
Objectives, Method – SaRS Webinar, 28 April 2021
Proactive safety management based on the “concept of risk”Think in terms of «risk» and «proactive safety management»
Compared to historical practices: “react and fix” approach
where existing national railway systems were usually based on compliance with national rules & Standards, and a retrospective review of «bad experiences» from the past.
Safety Directive requires RUs, IMs (ECMs) to set up a documented Safety Management System (a Maintenance System) for controlling and managing risks linked to their activities, looking at both forward and retrospective
New proactive approach based on risk: “Predict and Prevent”
1) What can go wrong? (predict/identify hazards/risks)2) Can it reasonably occur? (likelihood)3) What are the impacts if it happens? (consequences)4) What can I do either to prevent it, or to minimise consequences, and
to manage safely my business? (risk control/mitigation measures)
Key questions to ask in a proactive “risk based approach”:
Slide n° 12Introduction to Reg. 402/2013 on the CSM RA – History, Initial
Objectives, Method – SaRS Webinar, 28 April 2021
Initial objectives of CSM for risk assessment Main considerations
❑ Work with safety experts from EU railways →WG composed of NSAs (15), CER (5), EIM (2), UNIFE (3), UITP (1 – International Association of Public Transport)
❑ A generic process for a harmonised and systematic approach to risk and safety assessments based on existing practices, methods and recognised EU standards:
Recognition of strong railway safety culture → freedom left to each company to continue using existing risk assessment methods/tools/techniques
CSM provides common principles:
⚫ risk assessment process as in CENELEC 50126⚫ allows use of Codes of Practice and Similar Reference Systems, whereas
CENELEC 50126:1999 focused just on explicit risk estimation principle⚫ transparency: documented evidence and use of Hazard Log/Record
→ As CSM requirements can be fulfilled in many ways (not a unique one), CSM does not fix mandatory tools (e.g. FTA, FMECA, etc.)
Advices on risk assessment “tools” given in guidelines developed alongside CSM
Slide n° 13Introduction to Reg. 402/2013 on the CSM RA – History, Initial
Objectives, Method – SaRS Webinar, 28 April 2021
Initial objectives of CSM for risk assessment Main considerations
❑ As railway market opening split former railway companies into many different players → need to emphasise on:
everyone’s roles and responsibilities close cooperation for joint identification & management
of risks shared across interfaces between actors where necessary, exchange safety related information
❑ Equal treatment and transparent decision making for all railway actors →mutual recognition of results from risk assessments (independent check by 2nd pair of eyes)
Incumbent Railway
Company
NSA
RU & IM
ECM
❑ Keepers❑ Manufacturers❑ Suppliers❑ CABs❑ etc.
❑ Flexibility to decide: (concept of SIGNIFICANT change)
when a systematic approach, formalism and witnessingby an independent 2nd pair of eyes (AsBo) is needed
when a simpler formalism is sufficient, with no need for regulating and witnessing by a 2nd pair of eyes
Slide n° 14Introduction to Reg. 402/2013 on the CSM RA – History, Initial
Objectives, Method – SaRS Webinar, 28 April 2021
Iterative Risk Management Process “triggered” by a Significant Change
Short overview of Regulation 402/2013CSM risk assessment process - Flowchart in Annex I
1) Common risk assessment process including:(essential requirements → legally binding)
(a) System definition
(b) Identification of hazards/risks & associated safety measures
(c) Risk analysis based on exiting risk acceptance principles (CoP, Ref. Syst, Explicit Risk Estimation - no priority)
(d) Risk evaluation for checking acceptance of risk(s)
(e) Definition of safety requirements from identified safety measures
2) Demonstration of system compliance with identified safety requirements
3) Requirements for mutual recognition:
(a) Hazard Management via a Hazard Log(b) Independent Assessment (Body)
1(a)
1(b)&1(c)
1(d)
1(e)
2
3(a)3(b)
1(c)(iii)1(c)(ii)1(c)(i)
Slide n° 15Introduction to Reg. 402/2013 on the CSM RA – History, Initial
Objectives, Method – SaRS Webinar, 28 April 2021
²
Short overview of Regulation 402/2013 on the CSM for risk assessment
WHEN should it be applied? → always forTechnical, Operational and Organisational changes
❑ Always at least evaluation of “significance”❑ Risk assessment process in Annex I mandatory
only change is”significant”
NON-SIGNIFICANT safety related changes: risk control mandatory but risk assessment method not regulated/imposed, e.g. process of CENELEC 50126 or ISO 31000 acceptable
²
HOW should it be applied? (no constraints → freedom for details)
❑ Different ways possible - No prescriptions how to comply with “essential requirements”❑ Proposer free to use any tools, instruments or standards - Examples guidelines
WHO shall apply it?→ Proposer
❑ IMs, RUs, ECMs;❑ Actors requested to apply CSM by law (TSIs, directives,…) →Manufacturers
(e.g. an applicant for the Authorisation for placing a vehicle on the market)❑ Other actors when defined through contractual arrangements
WHO decides on significance?
❑ If no NNRs, Proposer decides ❑ NSA does not decide❑ Traceability of changes for
supervision by NSA/ECM CB
Slide n° 16Introduction to Reg. 402/2013 on the CSM RA – History, Initial
Objectives, Method – SaRS Webinar, 28 April 2021
… in order to provide the necessary reasonable assuranceto an Authorising Entity who must take a decision based on the risk assessment results from the application of the CSM
Article 6 appoint an AsBo
❑ gives an expert judgement on the confidence that the change under assessment can fulfil safely the intended objectives,
❑ enables to gain trust in an effective risk management and allows the mutual recognition of the results from the risk assessment
correct application of risk management process in Annex I of CSM, and;
suitability of results from the risk management process;
AsBo is a reliable second pair of eyes who through independent assessment of:
Regulation 402/2013 – Obligations for anindependent safety assessment by an AsBo
Mandatory
Questions? → Send e-mail on: [email protected]
PRESENTATION 2
IVAN LUCIC
Network Rail
19
Applying CSM-RAin real life…
Ivan LucicEastern Region Systems EngineerSRP ChairApril 2021
20
DISCLAIMERThese are my own personal views, not necessarily
the official views of Network Rail.
21
• Data from years 2006-2020. • Numbers are NR wide • Probably doesn’t include every single project (heavy
maintenance for example)• Few projects in dataset were either blank or of ‘pending’
status, these were discounted from below numbers
Total number of projects in dataset = 3152
Total number of significant projects = 561 = 17.8%
Total No of Non-significant = 2591 = 82.2%
Scale of things in NR ….
22
Experience …. Background: selection of similar projects with and without CSMRA, NR vs LU:
• Simple depot upgrades
• Conventional signalling upgrade
• Station upgrade
Summary : whole experience was cheaper, less painful and more efficient in LU
23
Why is it difficult? - LegislationIn principle nothing wrong with the concept…some improvements…
• Roles defined in legislation do not easily fit in the NR organisation and the life cycle change of accountabilities;
• What do we mean by Additionality in practice? In practice Who can manage Additionality in the organisation?
• Safety Justification vs Safety Case (50126);
• Competencies for actors and those who deliver not defined but defined for ASBO (with wrong focus on legislation as opposed on real life application of system safety NOT site safety);
• Focus on What without the guidance on How.
24
Why is it difficult? – NR Implementation• Focus on What without guidance on the How
• ASBOs often doing tick box audits not assessments, overly bureaucratic?
• Most importantly, it is not mandated by legislation to have to have external ASBO, why we can’t do it internally? It works perfectly well in LU.
• System Safety and CDM/Site safetydo not mix well and should not be mixed
• What about projects that are not significant but shall be authorised?
Making sure that people aren’t
hurt while at work
Engineering Safety
(or “System Safety”)
Making sure that people aren’t
hurt as a result of the changes
that we make to the railway
Occupational Health
and Safety
• Plain English
• Simple and Clear guidance on:
What, Where,
How, Who,
When, Why
25
What was wrong with the Yellow Book?
• Examples of what good looks like
• Training and certification available
• Scope includes maintenance as well as projects
26
Conclusions and recommendationsERA:
• Simplify roles so its easer to fit in the generic railway IM organisation;
• Consult those who deliver what are points that need clarification and clarify (CSMRA vs CENELEC; additionality; direct to How …)
UK (NR, ASBOs and ORR):
• Simplify and take bureaucracy out of it please
• There is noting in the Yellow Book that contradicts CSMRA, bring it back as the industry good practice guidance
All involved:
• Please get the
• In doing the above please consult those who actually do it on the ground System Integration Hub
PRESENTATION 3
PETE GRACEY
Office of Rail and Road (ORR)
NSA reflections on 10 years of CSMRAPete GraceyHead of Interoperability & Rail Vehicle EngineeringOffice of Rail and Road
28/04/2021
My CSMRA background (or the second half of my railway career to date)
Pre-CSMRA
• Designated Competent Person
Conformance Assessment Body
• AsBo/NoBo/DeBo Signatory
• Independent Competent Person (Pre-CSMRA ROGS 2006)
Project Delivery
• AsBo/NoBo/DeBo Client
• Proposer of change
National Safety Authority
• Authorisation process and policy
orr.gov.uk 29
The UK transition to CSMRA
The Yellow Book
• Extensive guidance – not always followed
• Pete’s view – CSMRA, it’s a subset of YB and entirely compatible, but …
Safety Verification by Independent Competent Person (ROGS 2006)
• Over application?
• Bias of the individual (or system …)
• Retention issues (NR specific)
Transition to CSMRA
• Not starting from the right point …
• … but adhering to the spirit!
orr.gov.uk 30
Further reflections
As AsBo
• Education, education, education
• Bias (or blinkers) of the client
As Client
• Education, education, education
• Bias (or background) of the assessor
As NSA
• Education
• Communication, communication, communication
orr.gov.uk 31
View from the NSA
• Use the process – it works
• React & fix => predict & prevent
• Supports Authorisation (and specific case decisions)
• Understand the Codes of Practice or reference systems you apply
• Manage your assessment
• Document safety decisions (particularly in early stages of design)
• Utilise the competence of your own team
• Challenge the assessment, but resolve the issues
• Use the process to support (better) investment decisions
• Feeds into through life costing
orr.gov.uk 32
PRESENTATION 4
DRAGAN JOVICIC
European Rail Agency (ERA)
Slide n° 35REX of EU railway sector on Reg. 402/2013 on CSM RA – Experience – Lessons
learned – Next steps – SaRS Webinar, 28 April 2021
Return of EXperience (REX) of European railway sector
with EU Reg. 402/2013 on the CSM for
risk evaluation and assessment (CSM RA)
Experience – Lessons learned – Next steps
Safety and Reliability Society (SaRS) Webinar, 28 April 2021
Dragan JOVICIC, EU Agency for Railways
Slide n° 36REX of EU railway sector on Reg. 402/2013 on CSM RA – Experience – Lessons
learned – Next steps – SaRS Webinar, 28 April 2021
Methodology for gathering EU sector experience withRegulation 402/2013 on the CSM for risk assessment
❑ ERA gather sector experience with method and, where necessary, make recommendations for improvements
❑ Survey carried out in 2018 across whole EU railway sector
applied by❑ RUs, IMs & ECMs through SMS
❑ Manufacturers placing on market
❑ NSAs❑ ECM Certification Bodies❑ AsBos
Reply rate38 over 115
(33%)
ERA web page: more details in report on the "Return of Experience (REX) with theuse of the CSM for risk assessment” [Location: Activities\Common Safety Methods]
Reminders
Slide n° 37REX of EU railway sector on Reg. 402/2013 on CSM RA – Experience – Lessons
learned – Next steps – SaRS Webinar, 28 April 2021
❑ 10 years: 352/2009 → 402/2013 → 2015/1136 (risk assessment process unchanged)
❑ Actual experience with method still very poor -Not used very often because of fears
❑ Why is sector reluctant to use the CSM?
replaces blind application of rules and standards by a proactive analysis of what can go wrong and how to avoid it
requires a independent check by an independent assessment body
❑ CSM Achilles' heel: significant change filter
❑ Misperception of CSM: complicated, boring, requires a lot of unnecessary paper
❑ None likes it and none happy to deal with it
❑ → concept of significant change intentionally misused to avoid using CSM
Sector reluctant to use risk assessment and independent assessorsWrong perception of «risk» and «risk management» concepts
Slide n° 38REX of EU railway sector on Reg. 402/2013 on CSM RA – Experience – Lessons
learned – Next steps – SaRS Webinar, 28 April 2021
Reminder: “use of rules” vs. “think in terms of risk”→ big change in responsibilities and in approach to safety management of railway traffic and railway operations
Sector reluctant to use risk assessment and independent assessorsToo much self-confidence while complexity of railway system increased
❑ Sector too much self-confident that railways are safe because “We are running trains for more than 150
years using well-known rules. We know our business
We do not need being taught what & how to do!”
❑ Underestimation: in 150 years of railway history, risks also changed a lot and new risks arose:
complexity of architecture and technology used in railway system drastically changed
operating speeds and traffic density increased
interactions with human operators (HOF) more complex (design, installation, operation, maintenance)
much higher staff turn over with shorter careers →less time to build a railway safety culture (mindset)
Slide n° 39REX of EU railway sector on Reg. 402/2013 on CSM RA – Experience – Lessons
learned – Next steps – SaRS Webinar, 28 April 2021
❑ Only 2% to 3% (<5%) of changes significant (big companies 800-1000 changes/year)❑ Common/accurate picture per country or stakeholder category not possible❑ Levels of understanding, application and experience vary in function of [Country;
Company in country; project team in company]
(1) a small minority of stakeholders well understands and correctly applies CSM
Not more than “10 % to 15%” of RUs, IMs & ECMs have a positive experience
For manufacturers ratio is probably much higher, because more familiar with the CENELEC 50126 standard and risk management concepts
(2) a large number of stakeholders for whom understanding and application of CSM were challenging but they are “learning by doing” and they are “continually improving” its use
(3) a large number of stakeholders across whole EU still facing serious difficultieswith understanding and correct application of CSM
General trends emerging from analysis of inputs [RUs, IMs, ECMs, Manufacturers]
Many consider a few years’ period of CSM application too short for being able to understand, apply correctly, “learn by doing” and improve experienced deficiencies
Overall picture regarding sector experience withRegulation 402/2013 on the CSM for risk assessment
Slide n° 40REX of EU railway sector on Reg. 402/2013 on CSM RA – Experience – Lessons
learned – Next steps – SaRS Webinar, 28 April 2021
BIG COMPANIES:
1. Size not always the only determining parameter
Have complex organisational structures, higher needs and high ambitions in terms of change management → face different challenges due to organisational and interface complexity
2. Sufficient resources are allocated to risk management (usually a dedicated Safety Department)
3. Higher knowledge of “risk concepts“ and more familiar with “process thinking, and systematic & top-down approach” to risk management → CSM used more proactively way
4. Nevertheless, some big companies are still at a very early stage of implementation of the CSM and need still to improve
Variability of stakeholders’ experience with Regulation 402/2013Big vs. small companies
SMALL COMPANIES:
1. Need more time to react and adapt to changes of legislation
2. Usually smaller needs for changes → less opportunities to “learn by doing” & gain experience to improve
3. More problems regarding resources, as it is carried out by operational staff in charge of many also activities→ usually often subcontracted
4. However, sometimes flexible and able to implement smarter solutions that better capture the essence of CSM
5. In case of deficiencies, tend to solve problems instead of debating on them
Slide n° 41REX of EU railway sector on Reg. 402/2013 on CSM RA – Experience – Lessons
learned – Next steps – SaRS Webinar, 28 April 2021
Former incumbent railway companies:
1. More resistant to change and applica-tion of new methods, in particular if they feel it requires more resources
→ prefer to continue using former processes/solutions
2. More rigid ways of thinking, managing, solving issues, etc. usually based on CENELEC 5012x guidelines
→ consider CSM as unnecessary duplication of CENELEC
3. More resources and railway experience for being able to apply the CSM
Newcomer railway companies:
1. Often very flat organisations with less resources and rarely staff competent in risk assessment
→ often application of CSM is sub-contracted
2. Capable adapting faster to new terminology
3. Flexible and can change easier their way of thinking in a more constructive way.
→ tend to be proactive and apply CSM for staying on market
Variability of stakeholders’ experience with Regulation 402/2013Newcomer vs. incumbent railway companies
Slide n° 42REX of EU railway sector on Reg. 402/2013 on CSM RA – Experience – Lessons
learned – Next steps – SaRS Webinar, 28 April 2021
1. All countries: great lack of experts on market competent in railways, risk asses-sment and risk management → necessity for long term action plans across EU
2. Usually evidence and documentation of risk assessment with poor quality
3. Better understood & more applied for Technical changes (→ better experience)
4. Operational & Organisational changes rather use of standards and CoPs without necessarily formally documenting evidence of proper compliance with CoPs
5. CSM found being complex, not understandable → hardly applicable
6. Perceived as a heavy & costly legal obligation → applied very late at end of project or design when all choices are fixed and too late to change design (just for paper)
7. Usually not used as a proactive tool for risk identification and prevention/controlof risks - Almost never used for cost optimisation and process improvement
8. CSM & AsBo seen useless burden, costly bureaucratic formalism without benefits
9. Obligation to appoint AsBo discourages a lot decisions to apply CSM
10.Roles of AsBo & NSA vs. independent assessment of Op. & Org. changes unclear
Stakeholders’ experience with Regulation 402/2013Major observations
Slide n° 43REX of EU railway sector on Reg. 402/2013 on CSM RA – Experience – Lessons
learned – Next steps – SaRS Webinar, 28 April 2021
1. System Definition poorly described: boundaries, interfaces, environmental conditions, assumptions, limits of risk assessment, etc. → incomplete HAZID
2. Poor or inexistent description of project organisation: unclear roles, responsibilities and competence of experts in charge of risk management
3. Poor interface management:
a) all concerned actors not systematically involved in risk identification and joint risk management
b) many proposers tend to transfer control measures for risks that go beyond scope of responsibility and reasonable domain of control by those actors
4. Non-conformities in area of another actor not always reported to right actor who is capable to take necessary corrective/preventive actions
5. Due to poor system definition and poor cooperation for interface management
a) no assurance that all risks, in particular risks shared across (internal & external) interfaces, are identified and appropriately controlled
b) no assurance that safe integration of sub-systems and in railway system is properly done (knowing also that concept is not well understood either)
Stakeholders’ experience with Regulation 402/2013Major observations
Slide n° 44REX of EU railway sector on Reg. 402/2013 on CSM RA – Experience – Lessons
learned – Next steps – SaRS Webinar, 28 April 2021
Experience with the assessment of the SIGNIFICANCE of changes
1. Many proposers find assessment of significance of changes complex and difficult to arrive at right decision with criteria in Article 4 of Reg. 402/2013. Reason: objective is to conclude that change is non-significant, but not easy without proper risk assessment
2. Decision dependent on experience of people/teams in charge of assessment
3. To avoid applying formal process of CSM and appointment of independent AsBo only 2 to 3% (<5%) of changes considered significant, independently on whether country/company familiar with “rule based” or “risk based thinking”
4. Most of those 2 to 3% (<5%) of significant changes are “TECHNICAL changes”
5. Justifications of non-significance of safety related changes is poor. When existing, no assurance of proper risk identification & risk management of non-significance
6. Many actors wrongly interpret “non-significant” as a “permission to do nothing”
7. NSAs report they do not know how proposers assess and control risks arising from non-significant safety related changes → how do NSAs supervise RU/IM SMS Change Management Process?
Slide n° 45REX of EU railway sector on Reg. 402/2013 on CSM RA – Experience – Lessons
learned – Next steps – SaRS Webinar, 28 April 2021
1. Although preferably used, poor experience with Codes of Practice (CoPs)
2. Used just to limit risk management efforts - Side effect constraining operational measures whereas technical preventive measures would be better
3. Usually railway CoPs or known standards used, relevance not questioned (wrong)
4. Unnecessarily detailed Hazard Identification whereas a low level of detail is largely sufficient + Use of CoP not always registered in Hazard Record
5. Detailed Hazard Checklists misused with CoPs (incompatibles levels of detail)
6. If Similar Reference Systems are used, safety requirements derived without questioning further relevance of Ref. Syst. → point § 2.4.2(a) of Annex I of CSM (“reference system still qualifies for approval in MS”) seems not to be checked
7. When CoPs or Ref. Syst. used proposer complies with all their requirements
8. Deviations vs. CoPs (or vs. Ref. Syst.) assessed using other two principles, including explicit risk estimation
Experience with the use of Codes of Practice (CoP) and Reference Systems
Slide n° 46REX of EU railway sector on Reg. 402/2013 on CSM RA – Experience – Lessons
learned – Next steps – SaRS Webinar, 28 April 2021
Based on positive feedback of 10-15% inputs/actors, it can be concluded
Effectiveness of CSMConclusions from gathering of REX
(b) Objective justifications were not yet identified which would require an urgent amendment or revision of the risk assessment process in CSM
(a) CSM for risk assessment is effective for achieving market access and mutual recognition (objectives in Art. 1 & 15(5) of CSM)
(b) For a big majority of actors, period of application of CSM not long enough to “learn by doing” and improve;
(c) Main problem: lack of understanding of risk based concepts and finding out what can go wrong with a change – Often not aware that some risks are already controlled by existing SMS provisions;
→ Agency did not recommend yet to revise or amend the CSM for risk assessment
Slide n° 47REX of EU railway sector on Reg. 402/2013 on CSM RA – Experience – Lessons
learned – Next steps – SaRS Webinar, 28 April 2021
Next stepsAgency recommendations
(b) As effectiveness of “CSM for risk assessment” depends on correct implementation of “CSM for monitoring”, combine measurement of return of experience of both methods
The lack of effective and complete use of these two CSMs shall not lead to an unreliable picture of the methods and to mislead conclusions that could result in unjustified revisions of CSM RA and CSM MO with additional complexity
(c) Leave sector enough time to “learn and improve by doing” and gain experience with risk management
(d) With support of NSAs, ECM CBs and AsBos, later on measure again effectiveness and experience with CSM RA & CSM MO
(a) In collaboration with NSAs, put in place an appropriate dissemination programmeand training activities to increase sector awareness with “risk management concepts” (i.e. CSM RA and CSM MO) [already ongoing by some NSAs]
Slide n° 48REX of EU railway sector on Reg. 402/2013 on CSM RA – Experience – Lessons
learned – Next steps – SaRS Webinar, 28 April 2021
(e) Build a solid return of experience with clear justifications for necessary improvements or clarifications of legislation/guidance material, based on objective observations of real weaknesses and problems with the CSMs, and better target of any necessary additional dissemination or training activity
Revision of legislation without clear justifications may create additional administrative burden, complexity and misunderstanding of successive
amendments of legislation whereas sector is trying to implement correctly the existing pieces of legislation, to learn and improve by doing and to set up a
proper “risk based” safety culture in a very complex environment
Next stepsAgency recommendations
Slide n° 49REX of EU railway sector on Reg. 402/2013 on CSM RA – Experience – Lessons
learned – Next steps – SaRS Webinar, 28 April 2021
1) Take strategic decisions and initiatives with EC and Member States in terms of education and training in order to:
a) compensate the current lack of expertise on the market in railway risk assessment, and;
b) anticipate the growing loss of railway expertise with a continually increasing number of employees reaching the age of retirement;
2) In scope of implementation of 4th Railway Package, monitor how NSAs supervise the RU/IM SMS and verify that the Change Management Control process of the RU/IM SMS identifies/captures, assesses and properly controls the risks arising from both non-significant and significant changes;
3) ERA to coordinate with EA, national accreditation and recognition bodies and AsBo Cooperation Group in order to harmonise further the requirements, acknowledgement and supervision of AsBo competence across whole EU;
4) Set up verification mechanisms of Recognition Bodies
Next stepsAgency recommendations
Slide n° 50REX of EU railway sector on Reg. 402/2013 on CSM RA – Experience – Lessons
learned – Next steps – SaRS Webinar, 28 April 2021
“It is not because things are difficult that we do not dare, it is
because we do not dare that they are difficult”, Lucius Annaeus SENECA
What to keep in mind concerning Regulation 402/2013 on the CSM for risk assessment?
❑ Risk assessment is a means to an end, not an end in itself - The aim is to keep people safe, not only to have good paperwork
❑ Reluctance to CSM: Do not seek complicated interpretation of requirements – Read what is actually written & use common sense
❑ Not required to forget rules but to know why rules are used
❑ No constraints on how to comply with every step of process. Leaves room/flexibility for an effective risk assessment process that can be targeted to size & complexity of change
❑ It is a method/tool that guides Risk Analyst to do right things and take right decisionsIt does not replace expert judgement of analyst, only one who can take right decisions
❑ Single application of risk assessment and risk management techniques does not guarantee that right hazards/risks are actually identified and thus properly controlled
❑ It is crucial that method is applied by competent staff with a good knowledge of system to assess, experience and knowledge of risk assessment standards (e.g. CENELEC 50126, ICE 61508, etc.)
Questions? → Send e-mail on: [email protected]
QUESTIONS?
Slide 53
• All webinar recordings are archived under the Resources Tab on the SaRS website
• SaRS members can access this archive as a member benefit:
• You can join as a full member, or
• The simplified “Associate of the Society” grade which gives you access to all the SaRS resources including the webinars.
• See www.sars.org.uk
• This webinar recording will be available in a couple of days so keep an eye out if you want to see it again
Accessing the webinar recordings
JOINING SARS
54
If you aren’t a member and have enjoyed this webinar please take a look at our
website, see the benefits of membership and join us.
Safety & Reliability Society is a Licensed Member of the Engineering
Council for CEng and IEng Professional Registration
We accept membership applications from
candidates from all relevant backgrounds –
membership is open to everyone from students
to experienced professionals
More information available at www.sars.org.uk
FEEDBACK
55
• I am now going to initiate a feedback form
• Please can I ask you to fill it in before you exit the webinar
• The information is vital for us to improve our offering
• Please take two minutes to fill it in and click Submit
• Your CPD Certificate will be sent to you if you requested
one at registration
• Thank you very much and STAY SAFE