Upload
vonhu
View
329
Download
18
Embed Size (px)
Citation preview
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2011 – Vance Hilderman Slide 1
By Vance Hilderman, Principal Founder, HighRely Inc.
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2004 -2011 – Vance HildermanSlide 2
• “DO-178 is the worst standard in the world; except for all the others …” (Winston Churchill Paraphrased)
• “The School Of “Avionics Wishful Thinking” has many students, but no graduates …” (Vance Hilderman)
Almost Famous Quotes
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2004 - 2011 – Vance HildermanSlide 3
Overview
• Aviation Safety Framework
• Aircraft, Systems, Hardware and Software
• DO-178B and DO-254: History
• Scope and Application
• DO-178B Overview
• DO-178C Overview
• DO-178C Implications on
Verification & Certification
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2005-2010 Slide 4
About AtegoHighRely
• North America’s Largest Avionics Services Company
– 30% Avionics Software Engineering
– 20% Avionics Systems Engineering
– 20% Avionics Software/Hardware Testing
– 10% DO-178/254 Training & Project Management
– 10% Strategy, Gap Analysis, JumpCert-178
– 10% DER’s/Certification
• Partnerships with the top tool vendor in all five major product categories
• Largest repository of DO-178B/DO-254 Whitepapers
One Stop Supplier for all your avionics development needs.
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2004 -2011 – Vance Hilderman Slide 5
Safety, System, Software & Hardware
Software
DO-178B
Hardware
DO-254
System Development
ARP 4754
Safety
Assessment
ARP 4761
• Criticality Level
• Architectural
Inputs
SW Rqmts HW Rqmts
Tests Tests
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2004 -2011 – Vance Hilderman Slide 6
• Certification standards for airborne equipment– DO-178 => Software
– DO-254 => Hardware
• Regulated by the FAA
• Required if target aircraft flies in commercial U.S. airspace
• Covers full engineering lifecycle:– Planning (CM, QA, Development, Testing)
– Development (Requirements/Design/Implementation)
– Testing/Verification
– Certification
What are DO-178 and DO-254?
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2011 – Vance Hilderman Slide 7
“Guidance” (“Considerations”)
• “Considerations”, not
requirements
• Developed via committee: all
things to all people
• Applies to huge and tiny
systems alike
• Applies to very critical Level A
systems, and also Level D
• Complaints by industry that
compliance with 178/254 is
vague and too expensive; true?
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2005-2010 Slide 8
History of DO-178 and DO-254
• RTCA DO-178B: “Software Considerations in Airborne Systems and Equipment Certification”
• Developed 1980 – 1992 via 100+ Industry and Government personnel
• Many compromises to satisfy different goals
• Not a recipe book or “How To” guide
• “Discussion” flow for guidance; able to accommodate many different development approaches
• Lawyers versus Software Engineers; who wins?
• In practice: The Golden Rule …
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2004 - 2011 – Vance Hilderman Slide 9
DO-178: Evolution History
Version Year Basis Themes
DO-178 1980 -
1982
498 &
2167A
Artifacts, documents, traceability,
testing
DO-178A 1985 DO-178 Processes, testing to improve
quality, components, four
criticality levels, reviews, waterfall
methodology
DO-178B 1992 DO-178A Integration, transition criteria,
diverse development methods,
data (not documents), verification
to assess quality, tools
DO-178C 2008?
(underway)
DO-178B Reducing subjectivity; Address
modeling, OOT, improved tools,
formal methods
HighRely: Rely On Us 6h
2009-4-VH
Slide 10
Avionics Safety History: 1946 - 2008
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2004 - 2011 – Vance HildermanSlide 11
DO-178B Document Layout
1. Planning
2. Development
3. Correctness
1. Overview
2. System Aspects
3. Lifecycle
4. Planning Process
5. Development Process
6. Verification
7. Configuration Mgmt
8. Quality Assurance
9. Certification Liaison
10. Overview of Aircraft And Engine
Certification
11. Data & Considerations
A. Objectives by Cert Level
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2004 - 2011 – Vance HildermanSlide 12
Three Key Processes(same for DO-178 and DO-254)
• Planning Process – Occurs first
• Development Process – Follows Planning
• Correctness Process – Continuous Throughout
Project
1. Planning
Process
2. Development
Process
3. Correctness Process
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2004 -2011 – Vance Hilderman Slide 13
Optimal DO-178 & 254
Engineering RouteSafety
Assessment
& RqmtsSystems
Rqmts
Develop Plans,
Stnds, Chklsts
Develop
Traceability
Implement
CM
High-Level
Rqmts
Start QA
Low-Level
Rqmts
Design
Code &
Logic
Verification & Validation
Time (Planning Phase)
Time (Development & Correctness Phases)
Integration
Conformity
Review
SOI
#1
SOI
#2
SOI
#3
SOI
#4
Cert
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2004 - 2011 – Vance Hilderman Slide 14
DO-178 and DO-254 Key Attributes(similar for DO-178B and DO-254)
1. Detailed planning
2. Five Criticality Levels (A, B, C, D, E)
3. Consistency & Determinism
4. Traceability: top-to-bottom, and back
5. Independence (especially Levels A/B)
6. Path testing
7. Proven Tools (“Qualification”)
8. Up to 20 artifact types and 66 objectives
9. “Guilty Until Proven Innocent”
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2004 -2011 – Vance Hilderman Slide 15
DO-178B Objectives by Level
• Level A: 66 Objectives (25 with independence)
• Level B: 65 Objectives (14 with independence)
• Level C: 57 Objectives (no mandatory independence, but independent reviews recommended)
• Level D: 28 Objectives (no mandatory independence)
• Level E: No Objectives
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2004 -2011 – Vance Hilderman Slide 16
DO-178B Five Key Plans
PSAC: Plan for Software Aspects of Certification
SQAP: Software Quality Assurance Plan
SCMP: Software Configuration Management Plan
SWDP: Software Development Plan
SWVP: Software Verification Plan*** Plus 3 Standards: Requirements, Design and Coding
1.
PSAC
2.
SQAP
3.
SCMP
4.
SWDP
5.
SWVP
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2004 -2011 – Vance Hilderman Slide 17
Scope of DO-178B & DO-254?
PLD
ASIC
FPGA
CPU
RTOS
BSP
Math
APP SW
Drivers
DO-178B
DO-254
Typical Avionics LRU
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2004 -2011 – Vance Hilderman Slide 18
Criticality Levels
Criticality Level Pyramid
A
B
C
D
E
• Level A: Catastrophic
• Level B: Hazardous/Severe
• Level C: Major
• Level D: Minor
• Level E: No Effect
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2005-2010 Slide 19
Criticality Levels
• “Software whose anomalous behavior, as shown by the system safety assessment process, would cause or contribute to a failure of system functions…
A. …resulting in a catastrophic failure condition for the aircraft.” Level A = <1E-09
B. …resulting in a hazardous/severe-major failure condition for the aircraft.” Level B <1E-07
C. …resulting in a major failure condition for the aircraft.” Level C <1E-05
D. …resulting in a minor failure condition for the aircraft.” Level D > 1E-05
E. …with no effect on aircraft operational capability or pilot workload.” Level E = No further application of 178/254 required.
• 4
Level A
<1E-09
Level B
<1E-07
Level C
<1E-05
Level D
>1E-09
Level E
NA
Level E
NA
Level D
>1E-05
Level C
<1E-05
Level B
<1E-07
Level A
<1E-09
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2004 - 2011 – Vance HildermanSlide 20
Typical System Criticality Level
DAL examples of Systems
DO178B Level Sample of historical Systems
A Flight controls, Engine Controllers,
Primary Displays
B FMS, Many Radios and communication
systems. Many navigation Systems
C Back up Displays, back up
communication systems (SATCOM),
D Maintenance systems, Monitoring
Systems (Engine Vibration Monitors,
etc.)
E In Flight entertainment Systems (May be
Level D), Video Systems. Coffee makers
and galley services.
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2011 – Vance Hilderman Slide 21
Why Different Criticality Levels?
• Why Does 178/254 Have Different
Criticality Levels?
– Who were major 178/254 contributors?
– What were their major concerns?
• Schedule
• Cost
• Safety, but with reasonableness
Level A
<1E-09
Level B
<1E-07
Level C
<1E-05
Level D
>1E-05
Level E
NA
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2004 - 2011 – Vance Hilderman Slide 22
DO-178B Criticality Level Comparison
(NOT for DO-254; See DO-254 Whitepaper!)
DO178B Aspect Level A Level B Level C Level D
Independence Level High Medium Low Very Low
Necessity of Low-Level
RequirementsYes Yes Yes No
Statement Structural Coverage Yes Yes Yes No
Decision/Condition Structural
CoverageYes Yes No No
MCDC Structural Coverage Yes No No No
Configuration Management Tight Tight Medium Low
Source to Binary Correlation Yes No No No
Requirements Correlate to
Target processorYes Yes No No
Architecture & Algorithms
VerificationYes Yes Yes No
Code Reviews Yes Yes Yes No
SQA Transition Criteria Yes Yes Yes No
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2004 - 2011 – Vance Hilderman Slide 23
Special Terminology
• “Certified”: the entire “system” is Certified for flight, while
components (LRUs) may have different certification Levels
• “Certifiable”: a component (LRU) within a system achieving its
highest certification status prior to certifying it with a “certified”
system
• “Compliant”: certification via an entity other than the
FAA (e.g. Military or non-commercial avionics)
• “Qualified”: formal approval of a tool which (since it does not “fly”)
does not require “certification”
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2004 -2011 – Vance
Hilderman
Slide 24
178/254 For Military?
• Since 2001, worldwide militaries adopting 178/254.
Why?– Improved subcontractor consistency
– Improved re-usability
– Improved schedule
– Improved cost
– Improved reliability
– Safety requirements
• “Military” 178 Examples:
– JSF, C-17, C-130, A400M
– Global Hawk
– T50A
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2004 -2011 – Vance
Hilderman
Slide 25
DO-178’s Verification Equation
V = R + T
• Verification = Reviews + Tests
• What is Reviewed?
– Virtually Everything (for Levels A, B, & C)
• What is Tested?
– All Requirements & All Code (for Levels A, B, & C)
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2005-2010 Slide 26
Avionics Testing
Four Categories of Tests:
1. Functional Tests
– All Requirements
2. Normal Range Tests
– “Sunny Day” conditions
3. Robustness Tests
– “Rainy Day” conditions
4. Structural Coverage Tests
– Cover all code
Test
Functional
Tests
Normal Range
Tests
Robustness
Tests
Structural
Coverage
Analysis
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2004 - 2011 – Vance Hilderman Slide 27
Software Testing
• Four Categories of Tests:
– One Black Box
– Three White Box
– Mind the Overlap
– Note the Relative Sizes: to Scale
SW
Test
Functional
Tests
Normal Range
Tests
Robustness
Tests
Structural
Coverage
Analysis
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2004 - 2011 – Vance Hilderman Slide 28
Black Box Vs. White BoxSW
Test
Functional
Tests
Normal Range
Tests
Robustness
Tests
Structural
Coverage Tests
Black Box Vs. White Box
What is the difference?
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2011 – Vance Hilderman Slide 29
Weaknesses of DO-178B Verification
• Overly dependent upon subjective:
– Requirement granularity
– Design/Coding standard review criteria
• Complex asynchronous interactions not necessarily
verified at functional/system level
• Reduced benefit of model based development (MBD)
and advance development tools
Which leads us to DO-178C …
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2011 – Vance HildermanSlide 30
DO-178C Preview
• Almost 20 years since DO-178B released
• Software landscape has changed ...
• Advancements in:
– Tools & Automation
– Modeling & Object Oriented Technology
– Formal Methodologies
• Commercial world has embraced the
above; Avionics has slowly followed …
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2011 – Vance Hilderman Slide 31
DO-178C Preview
• Since 2005, committees have met to
discuss, and update, DO-178B
• Like 178B, includes Industry & Agencies
• Unlike 178B, more Tool Vendors
– Obvious focus on “acceptability” of certain
types of tools, particularly “theirs”
• Predominantly America & Europe, nearly
equal; quarterly meetings.
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2011 – Vance Hilderman Slide 32
DO-178C Preview
• Seven “Sub-Groups” (SG’s):
1. SG1: Document Integration
2. SG2: Issues & Rationale
3. SG3: Tool Qualification
4. SG4: Model Based Design (MBD) & Verification
5. SG5: Object Oriented (OO) Technology
6. SG6: Formal Methods (FM)
7. SG7: Safety Related Considerations (and
ground-based systems)
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2011 – Vance HildermanSlide 33
DO-178C Preview
• Unlike the DO-178A to DO-178B update, the “core”
update to 178C is modest
• Instead, changes are handled via four “Supplements”,
which “clarify”:
• A. Tools Supplement
• B. MBD Supplement
• C. OO Supplement
• D. FM Supplement
• Reduced subjectivity for Testing
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2011 – Vance Hilderman Slide 34
Tool Qualification
DO-178B:
• Two Criteria:
1. Development
2. Verification
DO-178C:
• Three Criteria:1. Development
2. Verification & Augments other
development or verification
activities
3. Verification only
• Five Tool Qual Levels:
1. For Level A
2. For Level B
3. For Level C
4. Tool Operational Rqmts (TOR),
Arch, Additional Verification
5. TOR Verification
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2011 – Vance Hilderman Slide 35
MBD & OO (Continued)
DO-178B:• No Explicit Provisions
• Assumes “structured design”
• OO acceptance, but user-defined (subjective)
• Maximize Determinism & Visibility
• Weak on OO and MBD traceability
• Weak on structural coverage application to OO & Models
DO-178C:
• Allow controlled modeling & OO
• Bound MBD & OO acceptability
• Emphasize traceability
• Address memory management &
exception handling
• Verify “type consistency” (verify
substitutes,
• Each subclass passes all tests
applicable to parent
• Verify all callable methods for each
invocation
• Emphasize detailed MBD & OO design
standards
• Allow defined generics
• Acceptable Virtualization (“code” versus
“data”)
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2011 – Vance Hilderman Slide 36
Formal Methods (for Verification)
DO-178B:• No Explicit
Provisions
• (But commonly
applied, subjectively,
in Europe via ED-
12B)
DO-178C:• Recognize acceptance of formal
methods for:
– Requirements correctness,
consistency, and reviews
– Source code reviews, particularly
autocode generation from models
(low level requirements)
– Test cases covering low level
requirements
– Replacement of some forms of
testing via formal method-based
reviews
– “Potential” to reduce testing via
code analysis
HighRely: Rely On Us 6h
2009-4-VH
Copyright 2011 – Vance Hilderman Slide 37
• More information? Just Email:
[email protected]: ask for:
- “Military Certification” Whitepaper
- “DO-178 Costs Versus Benefits” Whitepaper
• Private DO-178B Training
(over 7,500 trained by Vance;
2 or 3-day customized sessions
at your site.)