Embed Size (px)
Citation preview
June 2007 1
Software Research and Technology Infusion
June 2007
Presented by
Pavan [email protected]
June 2007 2
2007 Infusion Team
Representation from GSFC, MSFC, JSC, JPL, and the NASA IV&V Facility– Allen Nikora (JPL) - [email protected] – Sally Godfrey (GSFC) - [email protected] – Ben Kobler (GSFC) - [email protected] – Ken Chen (JSC) - [email protected] – Andy Young (MSFC) - [email protected]– Wesley Deadrick (IVVF) – [email protected]– Pavan Rajagopal (IVVF) – [email protected]
June 2007 3
Outline Background
– Goal & Approach– Collaboration concept– Funding for Collaboration– Selected Technologies– Collaboration Roles
Selected software engineering and assurance technologies Perspective Based Inspections Requirements Assistant Reactis SAVE Toolset Software Cost Reduction
SW Developers Assistant SPACE CodeSurfer/CodeSonar Klockwork Inspect SpecTRM
June 2007 4
Infusing Software Research and Technologies
Goal: Transfer mature Software Engineering and Software Assurance technologies into practice– Provide a reduced risk approach to evaluating:
Technologies derived from NASA-sponsored research Other new and innovative SE/SA tools and technologies
Approach– Present selected SW technologies to the
NASA software development/assurance community
– Encourage and support collaborationsbetween the technology providers andNASA software developers and assurance personnel.
June 2007 5
Collaborations Initiated by a individual involved with SW
development/assurance who wants to bring on board a candidate technology
Purpose – benefit the software development project– validate the technology– generate empirical data to assess adoption– Not: further develop the research
Funding available for—– training and consulting in the use of the technology– license fees in the case of commercial technologies– applying the technology– collecting & analyzing data– reporting results
June 2007 6
Funding for Collaborations
Funding for 5 - 7 collaborations available via the Software Assurance Research Program (SARP).– History: 15+ projects in the range $15K - $45K
– Competition for SARP funds is among the NASA Centers and JPL. Proposals must come from a civil servant or a contractor who has a contractual vehicle in place with NASA.
Scope and POP of contract must be able to support the collaboration
NO NEW CONTRACTS WILL BE AWARDED
– Proposal template and instructions on the Research Infusion website
www.nasa.gov/centers/ivv/research/research_infusion_index.html
– Proposals Due: Monday, July 23 5:00 PM EDT
– Collaborations Start: Oct - Dec 2007
June 2007 7
Funding for Collaborations (cont.)
Mechanization
– The Principal Investigator (PI) represents the organization which plans to apply the new technology. PI can be a civil servant or contractor.
– Proposals must identify a NASA CS Point of Contact (POC) responsible for managing the collaboration
If PI is a contractor, often the POC is the COTR or technical manager on the PI’s contract
POC is responsible for coordinating the mechanization of the funding
– Either the PI or the POC can pay the technology provider
– In-kind funding is welcome!
June 2007 8
Selected Technologies Identified from
– NASA-sponsored software engineering and assurance research
– Leading edge commercial tools– Research in DoD, institutes and industry
suggested by NASA projects. Reviewed by researchers
experienced in tech transfer of software engineering research
Send us suggestions for next time.– SE & SA development problem areas– SE & SA technologies– Send suggestions to
June 2007 9
Selected Technologies (continued)
Technology Selection Criteria– Focus on Software Development or Software Assurance– Address a known need/requirement– Robust and mature with good user documentation- Demonstrated successes outside of a single domain or
application- Not currently in widespread use within the agency- Promise of support from technology providers
June 2007 10
Collaboration Roles
Principal Investigator – During proposal preparation:
Work with technology provider to plan collaboration and select suitable application
– Must have buy in from the technology provider Write and submit the proposal
– If proposal is selected: Coordinate training course with developer Identify software artifacts to which the technology will be applied Apply the technology (may require multiple iterations) Collect data & evaluate performance Write final report
June 2007 11
Collaboration Roles (continued)
Technology provider :– During proposal preparation
Help plan collaboration, including assisting in the selection of a suitable application
– If Principal Investigator’s proposal is accepted Provide any necessary training course (preferably on-site) Provide tutorial and other user documentation Provide customer support throughout collaboration
June 2007 13
Perspective Based Inspections (PBI)Dr. Forrest Shull, Fraunhofer Center - [email protected], 301-403-8970
What is it?– An inspection technique that improves the cost-effectiveness of
inspections (and teams' participation) by enabling reviewers to focus on topics most relevant to their expertise.
Features– Each participant looks at the work product from a focused
perspective, more related to their expertise or job needs.– Can be applied to requirements, design, code or other types of
documents. (E.g. test plans) Benefits
– Identifies defects (missed, incomplete, or wrong information) early
– Has been shown to be more effective than other inspection approaches
June 2007 14
Perspective Based Inspections (PBI)
Successes– At GSFC:
Dr. Shull worked with development team members to tailor the perspective-based approach for their requirements inspections.
The approach was both piloted and reviewed by key personnel. The results of both were very positive.
The final technique was written up for inclusion in branch-level standards.
– At JSC: Dr. Shull worked with a team at United Space Alliance to tailor the approach for their
specific requirements and code, and provided training in the technique. Defects were found during practice inspections, in reused software that was thought to
be defect free. PBI also found a major defect which had escaped previous inspections. Less inspection time was required per inspector than usual.
Contexts for best use– The only challenges found so far for the technique have been with very small
development teams (i.e. 3 to 4 people on the entire project team). However, a tailoring kit is being developed to make insertion easier in these contexts.
June 2007 15
Requirements Assistant (RA)Herman Driessen, Sunny Hills [email protected] Jones, IV&V [email protected]
What is it?– Toolset designed to help ensure that requirements are complete,
consistent, feasible, and unambiguous, using text in natural language as input.
Features– Runs on desktop– Employs a lessons learned knowledge base gathered from analysis of
numerous requirements reviews.– Uses natural language parsing, checklists, templates– Can be customized to specific domains.
Benefits– Reduces much of the tedium involved in document reviews.– Supports continuous improvement by enhancing lessons learned data
base.
June 2007 16
Requirements Assistant (RA)
Successes– NASA IV&V Analysts performed an evaluation of several
Requirements Analysis tools and RA was judged the best.
Contexts for best use– during requirements and design reviews– during requirements generation– as an input check before requirements are put into a
traceability matrix – tool can also be used on use cases, test specifications, and
contracts.
June 2007 17
ReactisPoC: Steve Sims, CTO, Reactive [email protected] 703-534-6458
What is it– Tools to automate white-box test case generation and validation of
Simulink and Stateflow models. Features
– Runs on Desktop.– User can specify structural-coverage criteria (such as branch and
MC/DC, as required by FAA DO-178B) in selecting test data.– Generates test data automatically from models to achieve specified
coverage. – Check whether model violates requirements.– Advanced debugging: breakpoints, reverse execution, track model
coverage metrics. Benefits
– Construct better models more quickly through requirements checking and debugging, and better code through coverage testing.
June 2007 18
Reactis
Successes – Reported 25 – 75% reduction in testing costs in automotive industry.
Contexts Best Used– Any project using Simulink/Stateflow in a model-based design
process to develop software Use Reactis to debug models Use Reactis to compare code against model
– Scales well to large models Automatic test-generation even for very big models Manual hand-tuning possible to enhance automatically generated tests
June 2007 19
Software Architecture Visualization and Evaluation (SAVE)Dr. Mikael Lindvall – Fraunhofer Center [email protected], 301-403-8972
What is it?– Toolset and process that allows a user to capture and compare a
planned software (SW) architecture with the actual SW architecture evidenced in source code (e.g. C/C++, Java, ADA, Fortan)
Features– Analyzes source code to enable visualization and high-level
understanding of the architecture– Identifies architecture changes and potential improvements – Supports bringing architecture into alignment with plan
Benefits– Used to identify architecture violations– Used to simplify complex architectures
June 2007 20
Software Architecture Visualization and Evaluation (SAVE)
Successes– Successful application of tool set and process with:
Common Ground SW at JHU/APL Space Network Access System at NASA Digital Camera at Ricoh Engine Control System at Hitachi
Contexts for best use– Systems that are developed, maintained by different people– Systems that are difficult to understand, costly to maintain– Systems that need to be restructured/reengineered
June 2007 21
Software Cost Reduction (SCR)Connie Heitmeyer, Naval Research [email protected], 202-767-3596
What is it?– Toolset that facilitates the application of formal methods - model
checking, theorem proving, simulation and automated test case generation – for defining and validating requirements specifications.
Features– Runs on desktop– Enables conversion of natural language requirements into lightweight
formal specifications– Permits simulation/animation (i.e., symbolic execution) of the system
based on the specifications – Identifies incompleteness (e.g., missing cases) and inconsistencies
(e.g., unwanted non-determinism) in specs– Generates tests satisfying various coverage criteria; Criteria include
Branch Coverage and MCDC– Scalable to large systems
June 2007 22
Software Cost Reduction (SCR)
Successes– Lockheed Martin
User of tools since 1999 Tools used to specify autopilot logic, flight navigation, flight
control/management, and airborne traffic/collision avoidance Tools currently being used on the Joint Strike Fighter
– ISS Fault Detection, Isolation Recovery (FDIR) of Thermal Radiator Recovery Subsystem at IV&V Facility.
Error detected in the original requirements documents Implementation bias in the requirements exposed
– ISS Bus FDIR at IV&V Facility Consistency checker detected missing and incorrect requirements
June 2007 23
Contexts for best use– System/software with following characteristics:
Systems with modes Systems with complex control logic Safety-critical, Fault-tolerant systems
– Prerequisites for effective use of the SCR method: Precise knowledge of the system/software requirements Domain experts who can answer questions about the
requirements
Software Cost Reduction (SCR)
June 2007 24
Software Developer’s Assistant (SDA)Stewart Bush: [email protected], 281.224.4408
What is it?– Process enactment platform that guides NASA software teams through
their project-specific standards, processes, and procedures. Features
– Web based toolset
– Helps ensure process tasks are completed in the correct sequence.
– Provides all of the tools, instructions, reference materials, and supportive artifacts to enable process compliance.
– Supports NASA Flight Software Development Standard
– Tailorable and Customizable
– Flexible, Robust, Extensible
– Provides real time status reporting of multiple projects
June 2007 25
Software Developer’s Assistant (SDA)
Benefits– Enhances productivity by shortening process learning curve– Automates tedious aspects of project management and process
compliance– Reduces effort for CMMI compliance– Supports continuous improvement by sharing best practices and
enabling comparison between projects. – Makes process and data more easily visible and accessible.
Successes– Tietronix received SBIR Phase II grant to mature the tool for
multiple NASA organizations. Phase III expected in July. – SDA currently implements NASA/JSC Engineering Flight Software
process
June 2007 26
Software Developer’s Assistant (SDA)
Contexts for best use– Currently able to support implementation of NPR 7150.2 and
RUP.– Also may soon be able to support IEEE 12207, OpenUP,
SCRUM
June 2007 27
Software Process Assurance for Complex Electronics (SPACE)Richard Plastow : [email protected], 216-433-3431
What is it?– Overall approach that includes process and product assurance activities
that can be used on complex electronics.
– Provides examples on how to use Fault Tree Analysis. Traceability Analysis, etc. on complex electronic devices.
Features– Guide implemented via a web-based interface.
– Assists in defining, planning and managing the assurance process.
– Includes document templates, techniques, and checklists Benefits
– Provides a process checklist for each phase of the life cycle
– Includes Best Practices from NASA and Industry
June 2007 28
Software Process Assurance for Complex Electronics (SPACE)
Successes– Checklists used to do spot checks on several projects– Currently being used by several projects in their initial
stages of development– Currently being evaluated by multiple projects for use in
development of their FPGAs
Contexts for best use– Gives the best results when used starting with the initial
project development but provides value to projects as far along as the test phase.
June 2007 29
CodeSurferPoC: Mark Zarins, GrammaTech, [email protected], 408-246-9100
What is it: Intelligent C/C++ source code browser for:– Program understanding
– Manual code inspection
– Safety/Security auditing Features
– Commercially supported product
– Code navigation
– Graphical reports (including call graphs)
– Advanced control flow and dataflow analysis
– Powerful searching
June 2007 30
CodeSurfer
Benefits– CodeSurfer makes it easy to analyze and understand code.
Successes– Used by NASA, Mitre, MIT, Thales, Airbus, and many other organizations
– Successful 2004 Research Infusion collaboration at JSC: “Our group analyzes many mission-critical software projects to
reduce defects and ensure that the software meets requirements. We conducted a formal study to see if CodeSurfer could improve our software inspections. We found that CodeSurfer reduced inspection time by an average of 30%. In addition, when using CodeSurfer the number of defects found more than doubled.“
Contexts for best use– Manual code inspection/reviews– Works on compilable C source code– Best applied smaller applications (< 250K LOC).
June 2007 31
CodeSonarPoC: Mark Zarins, GrammaTech, [email protected], 408-688-1243
What is it:– Automated code analysis tool that identifies serious defects in C/C++ code.
– Can be used on a whole program or program fragments. Features
– Commercially supported product
– Finds over 50 types of problems, including buffer overruns, null pointer dereferences, uninitialized variables, and memory/resource leaks.
– Users can extend the analysis with custom checks.
– Some checkers enforce coding recommendations for software reliability and safety developed by Gerard Holzmann’s group at NASA JPL.
– HTML reports show details of bugs and paths upon which they occur.
June 2007 32
CodeSonar
Benefits– Identification of many types of defects, including interface defects.
– More coverage than test suites alone.
– Easy to apply; no user input is required. Successes
– Many commercial customers, including Smiths Aerospace/GE: "CodeSonar does a better job of finding the more serious problems,
which are often buried deep in the code and sometimes hidden by unusual programming constructs that are hard for other static analysis tools to parse. CodeSonar is not easily fooled."
Contexts for best use– Need compilable C source code; build application with a standard C/C++
compiler; CodeSonar monitors build and captures information to identify defects.
– Can handle both small and huge (millions of lines of code) applications.
June 2007 33
Klocwork InspectPoC: Tim Oliver, Klocwork, [email protected], 770.529.7111
What is it?– Automated source code analysis tool for detecting logical code problems, metric
violations and architectural violations in C and C++, and Java code. Features
– Detects over 125 pre-defined defect rules out of the box– Customizable to include user-defined defect rules– Can filter to view defect by severity, status, type, and scope – Can identify new defects and fixed defects in current build – Can be integrated into organization’s build & issue-reporting process– Relatively high accuracy (identifies real defects with relatively few false positives
or false negatives) compared to most other source code analysis tools– Includes trend analysis, architectural analysis, and metrics.
Benefits– Identification of defects early in the development lifecycle reduced cost to fix.– High accuracy less time spent on filtering false alarms
June 2007 34
Klocwork Inspect
Successes– Positive evaluation by Gerard Holzmann at JPL on several example
applications including 400 K non-comment source code flight application.
– Used by HP and other large software developers. (Largest application in use is over 60 MLOC).
– Also used by NSA, DOD, DISA, DOA, IRS, SPAWARS, US Missile Defense Command, LMCO, Boeing, Motorola, Sysco and others.
Contexts for best use– Integrated into the development environment. Best used at the
developer desktop– Target applications should follow good programming practices
guidelines
June 2007 35
SpecTRM (Specification Tools and Requirements Methodology)PoC: Grady Lee, Safeware Engineering [email protected], 206-328-4880
What is it – Environment for creating & analyzing state-machine based
requirements models, with particular emphasis on mission-critical and safety-critical systems.
Features– Emphasis on construction of software requirements models that
can be easily read, reviewed, simulated visually, traced and analyzed
– Direct support for capturing “intent specifications” to document rationale—not only what and how but also why.
– Tool support for completeness in hazard analysis
June 2007 36
SpecTRM
Benefits– Find consistency/completeness errors at the requirements level, where
resolving errors is least costly and most effective– Easily-learned notation enables use by domain experts– Assist in satisfying new NASA safety standard.
Successes– Derived from Prof. Nancy Leveson’s work on TCAS II with NASA and
FAA support– Adopted and used by Japanese Manned Space Systems Corporation – SpecTRM-based services provided to automotive, aerospace and
medical devices industry by Safeware Contexts for best use
– Software-intensive, mission-critical and safety-critical systems.– Software with complex decision-making algorithms, such as mode and
state transition logic, benefit more than systems where complexity is in numerical calculations.
June 2007 37
Next Step
If you’re interested in a collaboration involving a Research Infusion technology, check out the collaboration proposal process at
http://www.nasa.gov/centers/ivv/research/research_infusion_proposal.html
We will help broker matches oftechnology and softwaredevelopers.
