View
216
Download
2
Category
Preview:
Citation preview
119/10/201019/10/2010 UnclassifiedUnclassified
C.A. Lucas*, A. LyonsC.A. Lucas*, A. Lyons†, W. Glover*, J.S. Cross*†, W. Glover*, J.S. Cross**Agent Oriented Software Limited, †Aero Synergy*Agent Oriented Software Limited, †Aero Synergy
C.A. Lucas*, A. LyonsC.A. Lucas*, A. Lyons†, W. Glover*, J.S. Cross*†, W. Glover*, J.S. Cross**Agent Oriented Software Limited, †Aero Synergy*Agent Oriented Software Limited, †Aero Synergy
IET International System Safety Conference 2010IET International System Safety Conference 2010
Incremental Safety AssuranceIncremental Safety Assuranceof anof an
Autonomous Decision-making System Autonomous Decision-making System for UASfor UAS
Incremental Safety AssuranceIncremental Safety Assuranceof anof an
Autonomous Decision-making System Autonomous Decision-making System for UASfor UAS
OutlineOutline
Role of safety-critical software in autonomous UAS
The challenge of clearing incremental changes to this software
Evolving regulatory environment
Modular safety case
Icing case study
Conclusions
Role of safety-critical software in autonomous UAS
The challenge of clearing incremental changes to this software
Evolving regulatory environment
Modular safety case
Icing case study
Conclusions
2
BackgroundBackground
Modern aircraft and engines rely on software for primary air vehicle control, no manual reversion
Consequently FCS or FADEC software is safety critical
Substantial cost and timescale required to introduce software changes, once safety assured
Modifications incur re-certification costs that approach cost of original clearance
Desirable to have a modular software safety assurance process
relating software re-assurance effort to scale of change introduced
Modern aircraft and engines rely on software for primary air vehicle control, no manual reversion
Consequently FCS or FADEC software is safety critical
Substantial cost and timescale required to introduce software changes, once safety assured
Modifications incur re-certification costs that approach cost of original clearance
Desirable to have a modular software safety assurance process
relating software re-assurance effort to scale of change introduced
3
Implications for Autonomous UASImplications for Autonomous UAS
Have decision-making software that replaces many functions performed by human crew of a manned or remotely piloted vehicle
Scope extends beyond simply flying the vehicle to a wide range of other critical roles, such as:
detect and avoid
mission management and re-routing
power management
failure detection, analysis and resolution, to Prognostic Health Management
centre of gravity and fuel management
ATC and communications
ground handling
Have decision-making software that replaces many functions performed by human crew of a manned or remotely piloted vehicle
Scope extends beyond simply flying the vehicle to a wide range of other critical roles, such as:
detect and avoid
mission management and re-routing
power management
failure detection, analysis and resolution, to Prognostic Health Management
centre of gravity and fuel management
ATC and communications
ground handling
4
ChallengeChallenge
Many successful manned A/C recognised for their versatilitye.g., C130, dH Mosquito, Tornado
Breadth of capability results from inherent human versatility:
take on new missions if correctly briefed
train as necessary for the new mission
Autonomous UAS must be endowed with versatile behavioursIf adapted to new or different missions then on-board Autonomous Decision-making System (ADMS) has to be upgraded with necessary additional or revised behaviours
Challenge – how can these be safely introduced without requiring the re-generation of evidence for parts that remain unchanged?
Many successful manned A/C recognised for their versatilitye.g., C130, dH Mosquito, Tornado
Breadth of capability results from inherent human versatility:
take on new missions if correctly briefed
train as necessary for the new mission
Autonomous UAS must be endowed with versatile behavioursIf adapted to new or different missions then on-board Autonomous Decision-making System (ADMS) has to be upgraded with necessary additional or revised behaviours
Challenge – how can these be safely introduced without requiring the re-generation of evidence for parts that remain unchanged?
5
Strategy to achieve incremental approval of autonomous software systems
Start with clearance of the decision-making software engine (ADMS)
Then incrementally adding behaviours and data to reason with
Critical aspects:
Reasoning engine
Behaviours
Data
Relevant programmes:
SUAV(E) & Taranis
ASTRAEA
Ministry of Defence SSEI
Strategy to achieve incremental approval of autonomous software systems
Start with clearance of the decision-making software engine (ADMS)
Then incrementally adding behaviours and data to reason with
Critical aspects:
Reasoning engine
Behaviours
Data
Relevant programmes:
SUAV(E) & Taranis
ASTRAEA
Ministry of Defence SSEI
Approach used and relevanceApproach used and relevance
6
Evolving Regulatory Environment Evolving Regulatory Environment Civil airworthiness regulators starting to address UAS certification
Most UAS are currently operated in combat zones under war-like conditions
Not cleared to any formal set of regulations
Australia’s CASA one of first regulators to introduce civil UAS regulation
CASR, 1998, Part 101, Subpart F specific to UAS
NATO nations recently adopted STANAG 4671
However 4671 has a number of critical limitations when applied to autonomous vehicles:
Introduction: “These requirements may not be sufficient for the certification of UAV Systems with unconventional, novel or extremely complex features”, and further: “….the following areas are not covered by this airworthiness code:
Airspace integration and segregation of aircraft (including “detect and avoid”),
Non-deterministic flight, in the sense that UAV flight profiles are not pre-determined or UAV actions are not predictable to the UAV crew,…”
Civil airworthiness regulators starting to address UAS certification
Most UAS are currently operated in combat zones under war-like conditions
Not cleared to any formal set of regulations
Australia’s CASA one of first regulators to introduce civil UAS regulation
CASR, 1998, Part 101, Subpart F specific to UAS
NATO nations recently adopted STANAG 4671
However 4671 has a number of critical limitations when applied to autonomous vehicles:
Introduction: “These requirements may not be sufficient for the certification of UAV Systems with unconventional, novel or extremely complex features”, and further: “….the following areas are not covered by this airworthiness code:
Airspace integration and segregation of aircraft (including “detect and avoid”),
Non-deterministic flight, in the sense that UAV flight profiles are not pre-determined or UAV actions are not predictable to the UAV crew,…”
7
UK Regulatory InitiativesUK Regulatory Initiatives2008 CAA updated CAP722 “Unmanned Aircraft System Operations in UK Airspace – Guidance”
Chapter on autonomy introduced for the first time in a regulatory document
2009 CAA first released a draft of AMC UAS.1309: “Guidance material for UAS system safety requirements”:
(a) “This Acceptable Means of Compliance (AMC) is designed to set minimum acceptable UAS systems integrity levels in order to protect persons from collisions with UAS.
(b) This AMC is applicable to the Special Conditions defining the system requirements for the collection of systems that perform the functions usually assigned to an aircraft located pilot, in other words the ‘Synthetic Pilot’ (SP)….”
(c) This AMC, a UAS certification document, has been specifically tailored for use with UAS of all sizes …. to ensure the protection of persons in the air and on the ground from the effects of UAS….
(d) This AMC generally follows the ethos of AMC CS/FAR-25.1309.
(e) The certification basis for UAS will be similar to those for manned aircraft however there will be differences related to the absence of an aircraft-located pilot.”
2008 CAA updated CAP722 “Unmanned Aircraft System Operations in UK Airspace – Guidance”
Chapter on autonomy introduced for the first time in a regulatory document
2009 CAA first released a draft of AMC UAS.1309: “Guidance material for UAS system safety requirements”:
(a) “This Acceptable Means of Compliance (AMC) is designed to set minimum acceptable UAS systems integrity levels in order to protect persons from collisions with UAS.
(b) This AMC is applicable to the Special Conditions defining the system requirements for the collection of systems that perform the functions usually assigned to an aircraft located pilot, in other words the ‘Synthetic Pilot’ (SP)….”
(c) This AMC, a UAS certification document, has been specifically tailored for use with UAS of all sizes …. to ensure the protection of persons in the air and on the ground from the effects of UAS….
(d) This AMC generally follows the ethos of AMC CS/FAR-25.1309.
(e) The certification basis for UAS will be similar to those for manned aircraft however there will be differences related to the absence of an aircraft-located pilot.”
8
ADMS Basis of Certification –In-flight Icing
ADMS Basis of Certification –In-flight Icing
Evaluation of requirements for a UAS to fly into known icing conditions chosen as the basis of the study
This case study then used as the vehicle for the study of incrementally certifiable software system
Evaluation of requirements for a UAS to fly into known icing conditions chosen as the basis of the study
This case study then used as the vehicle for the study of incrementally certifiable software system
9
In-Flight IcingIn-Flight Icing
Comprised of two scenarios – baseline and an incremented case
Baseline is UAS not cleared for flight into known icing conditions, ADMS will include reasoning to:
Detect and identify possible icing conditions
Take necessary actions to avoid these conditions
Ensure effect of the icing conditions is minimised
Continue mission if possible
Comprised of two scenarios – baseline and an incremented case
Baseline is UAS not cleared for flight into known icing conditions, ADMS will include reasoning to:
Detect and identify possible icing conditions
Take necessary actions to avoid these conditions
Ensure effect of the icing conditions is minimised
Continue mission if possible
10
In-Flight Icing – Increment In-Flight Icing – Increment Incremented case describes the changes made so that UAS cleared for flight into known icing conditions
Autonomous software will require modification to:
Manage and maintain anti-icing and de-icing mechanisms
Assess the severity of icing conditions and determine if UAS can fly through them or if it should attempt to exit (cf Rex SAAB 340)
Continually monitor ice sensors to determine level of ice accretion
Determine preferred routes both through and out of icing conditions
Process will be developed to ensure that evidence for core autonomous system does not need to be reproduced
Only modified or new modules affected by the changed functionality
Incremented case describes the changes made so that UAS cleared for flight into known icing conditions
Autonomous software will require modification to:
Manage and maintain anti-icing and de-icing mechanisms
Assess the severity of icing conditions and determine if UAS can fly through them or if it should attempt to exit (cf Rex SAAB 340)
Continually monitor ice sensors to determine level of ice accretion
Determine preferred routes both through and out of icing conditions
Process will be developed to ensure that evidence for core autonomous system does not need to be reproduced
Only modified or new modules affected by the changed functionality
11
Software ApproachSoftware Approach
Approach for incremental certification is: Modularisation of safety argumentation, allowing re-use of base modules (e.g. compiler and Run-time Infrastructure (RTI)) for each new system
Classification of incremental changes leading to different, and potentially limited, needs for re-certification of applications whose base version is itself certified
A language is developed to:simplify modularisation of multi-agent systems and individual agents,
provide support for formal analysis of application
Additional tools help evidence collection suitable for incremental certification (non-regression of unchanged functionality)
Approach for incremental certification is: Modularisation of safety argumentation, allowing re-use of base modules (e.g. compiler and Run-time Infrastructure (RTI)) for each new system
Classification of incremental changes leading to different, and potentially limited, needs for re-certification of applications whose base version is itself certified
A language is developed to:simplify modularisation of multi-agent systems and individual agents,
provide support for formal analysis of application
Additional tools help evidence collection suitable for incremental certification (non-regression of unchanged functionality)
12
Example reasoning
plan
Example reasoning
plan
13
Proposed Safety Assurance ProcessProposed Safety Assurance Process
Establish a basis of certification, agreed with the CAA
Draft a Certification Plan, review with CAA
Functional Hazard Assessment
FMECA
Reliability assessment
Safety targets per AMC (UAS).1309
Establish a basis of certification, agreed with the CAA
Draft a Certification Plan, review with CAA
Functional Hazard Assessment
FMECA
Reliability assessment
Safety targets per AMC (UAS).1309
14
ADMS Basis of Certification – How Established?
ADMS Basis of Certification – How Established?
Existing code clauses applicable without alteration
Clauses wholly inapplicable to UAS
Clauses with a valid safety objective but which need to be re-written
Wholly new clauses unique to UAS
Existing code clauses applicable without alteration
Clauses wholly inapplicable to UAS
Clauses with a valid safety objective but which need to be re-written
Wholly new clauses unique to UAS
15
Establish Basis of Safety AssuranceEstablish Basis of Safety Assurance
Source Documents:CS-23
STANAG 4671
EU-OPS
Part Ops
Part FCL
AMC (UAS)1309
Source Documents:CS-23
STANAG 4671
EU-OPS
Part Ops
Part FCL
AMC (UAS)1309
16
Establish Basis of Safety Assurance- Airframe Codes
Establish Basis of Safety Assurance- Airframe Codes
CS23 – EASA code for manned aeroplanes
Normal, Utility, Aerobatic, Commuter
CS23 – EASA code for manned aeroplanes
Normal, Utility, Aerobatic, Commuter
17
Establish Basis of Safety Assurance- Airframe Codes
Establish Basis of Safety Assurance- Airframe Codes
STANAG 4671NATO code for Unmanned Air Systems
Based upon CS23, but 2 issues:
Failure mode severities, no compatibility to manned codes
Ignores any issue that affects a pilot in the manned code
STANAG 4671NATO code for Unmanned Air Systems
Based upon CS23, but 2 issues:
Failure mode severities, no compatibility to manned codes
Ignores any issue that affects a pilot in the manned code
18
UAS Airframe Codes Issues with the STANAG – Example 1
UAS Airframe Codes Issues with the STANAG – Example 1
CS23.773 Pilot’s View:Clear and undistorted view
Free from Glare
Protected against fog or frost
STANAG says: Not Applicable
CS23.773 Pilot’s View:Clear and undistorted view
Free from Glare
Protected against fog or frost
STANAG says: Not Applicable
19
UAS Airframe Codes Issues with the STANAG – Example 1
UAS Airframe Codes Issues with the STANAG – Example 1
CS(UAS)23.773 Sensor View:Clear and undistorted view of intended field
Free from Glare
Protected against fog or frost
CS(UAS)23.773 Sensor View:Clear and undistorted view of intended field
Free from Glare
Protected against fog or frost
20
UAS Airframe Codes Issues with the STANAG – Example 2
UAS Airframe Codes Issues with the STANAG – Example 2
CS23.775 (h)(i) Windshields and Windows(1) Windshield panes directly in front of the pilot(s) in the normal conduct of their duties, and the supporting structures for these panes must withstand, without penetration, the impact of a 0∙91 kg (2 lb) bird when the velocity of the aeroplane relative to the bird along the aeroplane’s flight path is equal to the aeroplane’s maximum approach flap speed.
CS23.775 (h)(i) Windshields and Windows(1) Windshield panes directly in front of the pilot(s) in the normal conduct of their duties, and the supporting structures for these panes must withstand, without penetration, the impact of a 0∙91 kg (2 lb) bird when the velocity of the aeroplane relative to the bird along the aeroplane’s flight path is equal to the aeroplane’s maximum approach flap speed.
21
UAS Airframe Codes Issues with the STANAG – Example 2
UAS Airframe Codes Issues with the STANAG – Example 2
• USAR 775 says Not applicable
– CS (UAS) 23.775 Clause devised by Aerosynergy:
– (a) Forward-facing structure, covers and lenses, which directly protect systems and equipment essential for detect and avoid and flight control functions must withstand, without penetration, the impact of a 0∙91 kg (2 lb) bird when the velocity of the aeroplane relative to the bird along the aeroplane’s flight path is equal to the aeroplane’s maximum approach flap speed.
• USAR 775 says Not applicable
– CS (UAS) 23.775 Clause devised by Aerosynergy:
– (a) Forward-facing structure, covers and lenses, which directly protect systems and equipment essential for detect and avoid and flight control functions must withstand, without penetration, the impact of a 0∙91 kg (2 lb) bird when the velocity of the aeroplane relative to the bird along the aeroplane’s flight path is equal to the aeroplane’s maximum approach flap speed.
22
What is the Problem with the Existing Codes? - Example
What is the Problem with the Existing Codes? - Example
• AMC 25.1309 7 (a) 5 Classification of Severity of failure mode:
• Catastrophic: Failure Conditions, which would result in multiple fatalities, usually with the loss of the aeroplane. (Note: A “Catastrophic” Failure Condition was defined in previous versions of the rule and the advisory material as a Failure Condition which would prevent continued safe flight and landing.)
• AMC 25.1309 7 (a) 5 Classification of Severity of failure mode:
• Catastrophic: Failure Conditions, which would result in multiple fatalities, usually with the loss of the aeroplane. (Note: A “Catastrophic” Failure Condition was defined in previous versions of the rule and the advisory material as a Failure Condition which would prevent continued safe flight and landing.)
23
HenceHence
• Discussions with CAA
• Regulatory discussions at the JARUS forum, NAAs plus FAA
• Led to:
– CAA document AMC UAS 1309
• Discussions with CAA
• Regulatory discussions at the JARUS forum, NAAs plus FAA
• Led to:
– CAA document AMC UAS 1309
24
AMC UAS 1309 Failure Mode Severities Example:
AMC UAS 1309 Failure Mode Severities Example:
Catastrophic
Failure Conditions that could result in multiple fatalities
Catastrophic
Failure Conditions that could result in multiple fatalities
25
AMC UAS 1309 Main PointsFailure Mode Severities vs. Probability:
AMC UAS 1309 Main PointsFailure Mode Severities vs. Probability:
26
Classification of failure conditions
No safety Effect
Minor Major Hazardous Catastrophic
Allowable qualitative probability
No Probability Requirement
Probable Remote Extremely Remote
Extremely Improbable
Allowable Quantitative probability per Flight Hour on the order of.
No Probability Requirement
<10-3
Note 1<10-5 <10-7 <10-9
Non Detect and Avoid Equipped. Detect and Avoid Equipped. Note 2
Software/CEH Dev Level D
Software/CEH Dev Level C
Software/CEH Dev Level B
Software/CEH Dev Level A
AMC UAS 1309 Main Points:With D & A; Integrity of Airframe and Complex
Flight Mgt System:
AMC UAS 1309 Main Points:With D & A; Integrity of Airframe and Complex
Flight Mgt System:
• AMC UAS 1309 Main Points:• With D & A; Integrity of Airframe and Complex Flight Mgt System:• AMC UAS 1309 Main Points:• With D & A; Integrity of Airframe and Complex Flight Mgt System:
27
Aircraft Category
CFMS Integrity including DA
requirement (Fig 1.0)
Airframe Integrity (analogous CS/FAR
to UAS code) requirement similar
to
Integrity delta
CS/FAR-25 AMC UAS.1309 (10-9) CS/FAR-25.1309 (10-9) No CFMS/ Airframe systems integrity delta
CS/FAR-23 AMC UAS.1309 (10-9) AC-23.1309-1D Increasing CFMS/ airframe systems integrity delta dependent on class
CS/FAR-23 AMC UAS.1309 (10-7) Mitigated case
AC-23.1309-1D Only allowed if mitigation is accepted by the authorities
CS/FAR-29 AMC UAS.1309 (10-9) CS/FAR-29.1309 Medium CFMS/ airframe systems integrity delta
CS/FAR-27 AMC UAS.1309 (10-9) CS/FAR-27.1309 Large CFMS / airframe systems integrity delta
CS/FAR-27 AMC UAS.1309 (10-7) Mitigated case
CS/FAR-27.1309 Only allowed if mitigation is accepted by the authorities
CS-VLA / VLR and others
AMC UAS.1309 (10-9) CS-VLA / VLR.1309 Very large CFMS / airframe systems integrity delta
CS-VLA / VLR and others
AMC UAS.1309 (10-7) Mitigated case
CS-VLA / VLR.1309 Only allowed if mitigation is accepted by the authorities
Other Codes:Other Codes:
• EU-OPS Part Ops
– Operational requirements
• Part FCL
– Looking at requirements for Flight Crew Licensing to understand what the human pilot is expected to do
• EU-OPS Part Ops
– Operational requirements
• Part FCL
– Looking at requirements for Flight Crew Licensing to understand what the human pilot is expected to do
28
UAS Airframe Codes Issues with the STANAG - Example 3
UAS Airframe Codes Issues with the STANAG - Example 3
• USAR1772 Part time data
• USAR1725 Powerplant data etc..
• Puts a lot of emphasis on the UAS Pilot, and this is a problem because:
– Estimated Reliability of the data link is 10-3
– Not sufficient for critical functions
• USAR1772 Part time data
• USAR1725 Powerplant data etc..
• Puts a lot of emphasis on the UAS Pilot, and this is a problem because:
– Estimated Reliability of the data link is 10-3
– Not sufficient for critical functions
29
Firescout IncidentFirescout Incident
30
“The MQ-8 Fire Scout experienced lost link and proceeded 23 miles north/northwest out of Pax River, still about 40 miles south of DC area in
northern St. Mary's county, Maryland, into National Capital Region airspace,” he said. “The operator team shifted to other Ground Control
Station, restoring link and successfully commanding vehicle to recover at Webster Field. The aircraft returned to Webster Field safely without
injuries, and without damage to the aircraft or vessel.”
According to Capt. Tim Dunigan, Fire Scout According to Capt. Tim Dunigan, Fire Scout program manager, the helicopter was at 1,700 feet program manager, the helicopter was at 1,700 feet
AGL 75 minutes into a test flight from its base, AGL 75 minutes into a test flight from its base, Naval Air Station Pax River in Southern Maryland, Naval Air Station Pax River in Southern Maryland, when operators temporarily lost communications when operators temporarily lost communications
with the unmanned rotary aircraft.with the unmanned rotary aircraft.
Firescout IncidentFirescout Incident
• He added that the MQ-8B Fire Scout has flown more than 1,000 flight hours since December 2006. i.e. 10-3
• Conclusions:
– The integrity of the UAS must be on board the UAV
– The relevance of the pilot is limited
– Autonomous logic is necessary...
– ...including ‘interrupts’ to prevent pilot error
• He added that the MQ-8B Fire Scout has flown more than 1,000 flight hours since December 2006. i.e. 10-3
• Conclusions:
– The integrity of the UAS must be on board the UAV
– The relevance of the pilot is limited
– Autonomous logic is necessary...
– ...including ‘interrupts’ to prevent pilot error
31
• Firescout incident highlights the risks
• This paper proposes a means for dealing with autonomous UAS regulation and ADMS safety assurance
• Cross fertilisation of results from SSEI Programme to ASTRAEA, with opportunity to flight test ADMS behaviours on Cranfield A/C
• Flow through to civil and military UAS programmes worldwide
• Civil regulators, CAA and FAA, developing regulations with industry cooperation
• What will MoD and the MAA do?
• Firescout incident highlights the risks
• This paper proposes a means for dealing with autonomous UAS regulation and ADMS safety assurance
• Cross fertilisation of results from SSEI Programme to ASTRAEA, with opportunity to flight test ADMS behaviours on Cranfield A/C
• Flow through to civil and military UAS programmes worldwide
• Civil regulators, CAA and FAA, developing regulations with industry cooperation
• What will MoD and the MAA do?
ConcludingConcluding
32
Recommended