Conducting FMEA over the software development process

Conducting FMEA over the Software Development Process Konstantina Georgieva

University of Magdeburg, Dept. of Computer Science, P.O. Box 4120, D-39016 Magdeburg, Germany,

< [email protected] >

Abstract: Failure mode and effects analysis (FMEA) is one of the well-known analysis methods with an established position in the traditional reliability analysis. It is widely used but even so, it is not popular in software engineering. To address this gap, in this paper we propose an application of the FMEA method for the software development process after which we visualize our approach with a tool that makes the application of SFMEA very easy. Our tool is available in our virtual Software Measurement Laboratory (SML@b) at

http:// www.smlab.de/webapplcations.html. 1. Introduction and related work

A Failure Mode and Effect Analysis (FMEA) is a well known method for failure analysis in the manufacturing industries in various phases of the product life cycle and is now also very popular in the service industry. The method was introduced in the late 1940’s for military usage by the US army and later on found big usage in the aerospace and rocket development to avoid errors and following fatal outcomes. With the years it becomes more and more popular and is used in different big industrial companies. Although that it is widely accepted and well-known there is relatively little information published on the use of FMEA for information systems (Banerjee 1995) and even less published about the usage in the software development process. Goddard notes the lack of techniques for applying FMEA during the design of software systems (Goddard 2000). He describes two types of Software FMEA – system software FMEA and detailed software FMEA. System Software FMEA analyzes the effectiveness of the software architecture. And detailed software FMEA validates that the software has been planned and constructed to reach the right and safe requirements from the beginning. Banerjee provides a useful insight that a good working software product requires teamwork and the knowledge of all team members during the entire software development process (Banerjee 1995). Luke highlighted the role of FMEA for software (Luke 1995). He discussed the early identification of potential software failure modes as a critical and excellent practice in software development process because it gives the possibility to verify the presence of correct code. Stamatis proposed the use of FMEA in the information systems (Stamatis 1995). He claimed that computer industry failures may result from software development process problems, coding, systems analysis, systems integration, software errors, and tying errors and that all these failures can origin from the work of the people in the concrete development process.

There are few examples which apply FMEA in the software development over concrete programs: (Lauritsen & Stålhane 2005), (Mackel 2002) and are very practically oriented but a methodological application of the failure method over the software development process have not been explicitly formulated. Because of this our work puts a new milestone in the use of FMEA in the software industry.

2. Conducting an FMEA In order to conduct an effective FMEA one must follow a constructive approach. The method brakes down the process into basic sub-process and examines potential problems in every step. The FMEA can be generally summarized into four major parts, which are shown on the following figure: Discover failure modes; Recognize the causes for them; Assign a RPN and Proposing mitigation strategies.

Figure 1. FMEA steps In order to achieve problem solving FMEA needs to be conducted strictly, consecutively and constructively in the following 8 steps: Step 1. Gather a team and review the process or product.

Step 2. Brainstorm unknown risks.

Step 3. Assign different effects caused by the failures.

Step 4. Prioritize – assign severity, occurrence and detection rankings for each failure mode.

Step 5. Calculate the RPN number.

Step 6. Collect data, analyze and measure the failure modes for each action.

Step 7. Apply methods to reduce high-priority/high-risk failures.

Step 8. After performing actions evaluate the performance of the system again.

ACM SIGSOFT Software Engineering Notes Page 1 May 2010 Volume 35 Number 3

DOI: 10.1145/1764810.1764819 http://doi.acm.org/10.1145/10.1145/1764810.1764819

http://www.smlab.de/webapplcations.html�

3. Our approach for applying FMEA during the Software Development Process

Our main idea is to apply FMEA over each phase from the software development process. Because of this we have taken ISO 12207 as the standard that describes the software development process and after this we have applied the well-known technique of FMEA over the different development stages.

3.1. International Standard ISO 12207

The international standard ISO/IEC 12207 (ISO/IEC 1995) defines a common framework where all software developers, programmers and everybody involved in the software creating process “can speak a common language”. The standard describes the whole conception from the ideas through implementation and coding to retirement. The development process described in the ISO 12207 consists of five main steps: Acquisition; Supply; Development; Operation and Maintenance Processes which can be additionally broken into sub-steps over which we will apply the FMEA method. The development of software requires different stages – requirements analysis, design, implementation, testing, configuration management, quality assurance, support, etc. Software managers should choose the right model which is good enough for the concrete team, risk level and the area of activity and knowledge. It is also essential to choose the right tools to support the process. Taking the model of software development prescribed by the ISO standard and involving our own observations and experience we end with the following 8 steps of the software development process that we analyze with the FMEA method for possible failure modes. According to the methodology in the FMEA method we need to break each main process into sub-processes:

Figure2. FMEA Principe

And then having this done we can proceed with explaining the failure modes in every development step (sub-process). 3.2 Failure modes during every step

STEP 1 - Planning and managing the project – process implementation The first step includes project planning and scheduling by examining the needed activities to plan and manage a software development process. The main concepts are – schedule estimation, risk management, project planning, team organization and costs planning. Probable Failure Modes:

I. The chosen software development model is not appropriate to the scope, magnitude and the complexity of the defined task.

II. The chosen model does not fit the complexity of requirements for the project at the very beginning.

III. The documentation and results obtained by the chosen software model does not reach the concerned management, development or maintenance team.

IV. Specifications for a certain component of the software project are not clearly defined or not correctly documented.

V. The information and documentation about the selected standards, methods, tools and programming language is not enough to have the required tasks completed.

VI. The required assignments are not achievable because of the limitations of the chosen framework or programming tools.

VII. There are “wholes” in the system which makes it vulnerable to external attacks.

VIII. There is no clearly defined hierarchy model for the different type of people using the system.

IX. During the development process the schedule made at the beginning fails to determine the needed time for all activities from the beginning to the end and also the cost seems inaccurate.

X. The project fails because of not enough experienced engineers and insufficient abilities of the team members to cope with the different problems.

XI. Cost estimation of the projects is not correctively realized.

Step 1. Planning and managing the project – process implementation.

Step 2. Capturing the software requirements.

Step 3. Designing the system – architectural and detailed design.

Step 4. Software implementation and coding.

Step 5. Testing the programs and the system in-tegration.

Step 6. Qualification testing and software deli-very.

Step 7. Software installation and maintenance.

Step 8. Software acceptance and support.



STEP 2 - Capturing the Software Requirements At this step of the software life cycle process the team focuses on capturing the system requirements. The main goal is to make as clear as possible which is the problem that the system is intended to solve; specific behaviors of safety and reliability, human-factors engineering and interfaces should be described.

Probable Failure Modes: I. Mistakes made during the requirements process cause additional

problems later in the software life-cycle because some requirements are not well understood.

II. The chosen techniques are not suitable to complete the concrete requirements.

III. A change in one requirement for an element is not supported by another related element. This leads to incompatibility and no possible further development.

IV. Possible failures of interfaces external to the software system.

V. The worst scenario is having a planned software design which is not feasible for implementation.

VI. The requirements are not back traceable from the system design.

VII. Anomalies in data definitions and database requirements.

VIII. Failures concerning human-factors engineering specifications, including those related to manual operations, human-equipment interactions, areas needing concentrated human attention which are sensitive to human errors and training.

STEP 3 - Designing the system – architectural and detailed design At this step a transformation from the requirements for the software into architecture is performed. An important part is the assurance that all the software requirements connected to the software components are allocated and correctly assigned in order to be provided further facilitation of detailed design. Probable Failure Modes:

I. The design fails to meet the low-level concepts where the high coupling and low cohesion make the design incomprehensible and not easily maintainable.

II. During the design phase important considerations are not made which leads to changes in the implementation and coding step. Usually, this is cost effective, time consuming and sometimes even not achievable.

III. The components in the design are not correctly combined, not well documented and there are components that affect other ones in unexpected way.

IV. The detailed design of the database does not satisfy the Normal Form as specified in the system requirements analysis.

V. Later in the development data redundancy occurs due to bad designed database.

VI. The detailed design is not feasible on the framework provided by the software architecture.

VII. Detailed design usually consists of software components. It is possible that some of these components can’t be broken into logical module units in order to be coded and compiled individually.

VIII. The developed design interfaces for external software fail to integrate correctly or reject communication, which results in a loss of data or unrealized communication.

IX. Improper inter-communication design in components fails due to wrong procedures.

X. The detailed design for a module is not back traceable to its requirements.

XI. There is no internal consistency between software components and software modules.

XII. The testing process defined by the detailed design process is not feasible at some level.

XIII. Feasibility of operation and maintenance can’t be realized at some point because of structure defined by the detailed design.

STEP 4 - Software implementation and coding At this step the main objective stands in front of the software programmers – implementing the design conceptions into high quality code. Probable Failure Modes:

I. There is a set of three main failures during the coding phase that are usual for most of the projects:

⇒ First – the designers have failed to address the specific connections between the platform and the programming environment, between the structures and relationships which are easy to be described with charts and tables. This makes difficult the following phase of coding.

⇒ Second – the written code is not understandable later for other developers and even for the author later.

⇒ Third – the programmers didn’t take advantage and experience of design’s characteristics, third part data structures and programming language’s constructions which are easy to be reused.

II. Sometimes the development team chooses for reusing not the correct component.

III. Relationship between design and implementation is not consistent.

IV. Coding process fails because of lack of coordination and cooperation. There are misunderstandings between the



programmers for what they have written.

V. The documentation for a reusable component is misleading and incorrect.

VI. Possibly functionality that programmers need for the system is missing of a reusable component.

VII. Possible failures may occur when the team tries to determine how to fit the reusable component into the design of the system.

STEP 5 - Testing the programs and the system integration Testing is not the only point where faults and errors are found, requirements and design phases also have reviews in finding out problems. But something important here is that testing is specifically focused on finding faults, not failures. Probable Failure Modes:

I. The tester fails to identify the type of fault and to classify it.

II. The testing team cannot achieve optimal results because it is using traditional testing techniques on object-oriented systems.

III. In many instances the system is tested in stages. Testers did not take in mind the different system configurations that are being developed. The result is high risk and errors.

IV. The software aggregates fail to integrate with the hardware components and configuration items.

V. Upon testing, the software modules and database give incorrect or no results when specific input data is entered.

VI. Software integration and testing are not feasible based on the final components design.

VII. Some of the components of the integration plan (test requirements, procedures, data and responsibilities) fail to meet the requirements of the software configuration item.

VIII. The test development section search for correctness of the system instead of focusing on the faults.

STEP 6 - Qualification testing and software delivery

At this stage an important step has to be made – transferring the software system from the developer to the customer. The need for documentation and training is essential. Probable Failure Modes:

I. During the design of the system helping documentation and specific descriptions are not made, or are not well described and clear for the users. The accompanying documentation is not logically structured and separated for the different kind of users.

II. Training does not provide the entire necessary information and is not available at any time, only when the system is delivered.

III. The system does not support training for novice users and absolute beginners.

IV. The results of qualification testing in accordance with the qualification requirements do not confirm the expected results.

V. Most of the software engineers did not take seriously this step, which results in incorrectly using of the system by the side of the users and they may be not satisfied of the project. Users can not be high productive and effective which makes the whole system unusable and a waste of time and money.

STEP 7 - Software installation and maintenance Software maintenance is the modification of the software product in order to be corrected errors or faults, to improve performance, to be added more capabilities to the system or to adapt the product to the fast changing environment. Probable Failure Modes:

I. A change in the system is necessary but the impacts analysis fails to evaluate the effects of the change in one component on the rest of the system.

II. The system needs changes, it becomes more unstable, slow for the current loading and some serious security measures must be taken because of the presence of new threats. The team fails to implement the right changes because of not good enough knowledge on the project.

III. The team fails with the task of measuring the maintainability of the system.

STEP 8 - Software acceptance and support Software acceptance and support is provided from almost any company guaranteeing high-quality services. At this stage additional information, answers and trainings can be provided for the customers in order to achieve smooth work and errorless conditions. Probable Failure Modes:

I. Customers can’t reach the support team of the software product due to lack of communication lines.

II. Some general working problems with the software occur during normal working state, because of bugs which prevent achieving results.

3.2. Our tool

We have developed a tool for conducting FMEA analysis over the software process in which we can create a new project or select an existing one and select the development phases that we have in the moment in order to observe their criticality. We can even change some of the fault modes; add new failures and new recommendations so that we can match the whole process of analysis to our specific project. This tool should be very helpful in



the process of preparing our software development process because we are able to see all the phases inside with all probable faults and the recommendations for them. Below are two screenshots from our tool that give a visual idea about it: Figures 3 and 4.

Figure 3. Screenshot 1 of the SFMEA tool

Figure 4. Screenshot 2 of the SFMEA tool 4. Conclusion

We have proposed a SFMEA method for analysis of the failure modes in the software development cycle. Using the SFMEA method the developer can identify possible failure modes and include source code or other components in the system to prevent the negative consequences. This helps a lot not only in recognizing failures in sub-systems/sub-process but also in the whole system level. The main advantage of the method comes from focusing only on one failure at a time.

FMEA is used successfully many times on a hardware level, where the potential problems are well known and the task is to analyze the changes they cause. In the last years FMEA is adapted for the software. A problem that occurs in the software FMEA is that the technology and the environment are very dynamic and additional considerations and limitations should be taken in mind. It is considered that FMEA method can be applied only to a limited level. FMEA is potentially suitable for conduction in any of the software phases, but the biggest advantage comes when it is used in the early stages of the software development where failures in the structure of the system can be recovered and a lot of resources can be saved. Here are the main conditions which should be fulfilled in order to execute a successful SFMEA process: Defining the system model. This task, which defines the

hierarchy model of all system functions, helps in the FMEA process to identify the main stages and requirements of the customer and the user.

Identifying potential failures. The core of the process conducted in the early stages of the software development transforms the requirements into safer implementation.

Team work during the entire process. When design experts, testers, developers and all the rest work together from the very beginning this guarantees considerable improvements.

Bibliography Banerjee, N 1995, 'Utilization of FMEA concept in software

lifecycle management', Proceedings of Conference on Software Quality Management.

Goddard, P 2000, 'Software FMEA techni-ques.', 2000 Proceedings Annual Reliability and Main-tainability Symposium.

ISO/IEC 1995, 'ISO/IEC 12207 '.

Lauritsen, T & Stålhane, T 2005, Safety Methods in Software Process Improvement , Springer Berlin / Heidelberg.

Luke, S 1995, 'Failure mode, effects and criticality analysis (FMECA) for software', 5th Fleet Maintenance Symposium.

Mackel, O 2002, 'Software FMEA Opportunities and benefits of FMEA in the development process of software-intensive technical systems', In Proceedings of 5th International Symposium on Programmable Electronic Systems in Safety Related Applications.

Stamatis, DH 1995, Failure mode and effectanalysis: FMEA from theory to execution, WI: ASQC Quality Press, Milwaukee.



Documents

Conducting FMEA over the software development process