1

Interoperable Toolchain for Requirements-DrivenModel-Based Development

Jan Steffen Becker∗, Vincent Bertram†, Tom Bienmuller‡, Udo Brockmeyer‡, Heiko Dorr§,Thomas Peikenkamp∗, and Tino Teige‡

∗ OFFIS - Institute for Information Technology, Oldenburg, Germany† Daimler AG - Group Research & MBC Development, Ulm, Germany

‡ BTC Embedded Systems AG, Oldenburg, Germany§ Model Engineering Solutions GmbH, Berlin, Germany

Abstract—This paper introduces a toolchain for requirements-driven model-based development of embedded software as used inthe automotive industry. Development usually starts with textualfunctional requirements written in natural language. Verificationof functional requirements required in safety critical systemsneeds traceability on system level and on implementation level.Therefore, the formalization of the provided textual requirementsis of vital importance. This however is a challenging task ingeneral, which we approach using an intuitive and graphicalformalization language, namely simplified universal pattern. Hav-ing the requirements formalized, as a second step an analysis isdone to ensure that the requirements are in a consistent state.This is important as within agile development, functionalities areevolving over time and textual requirements are continuouslyenhanced. To keep track of the implementation, an aggregationof model changes wrt., e.g., consistency, model test status, formalrequirement coverage, or modeling guideline conformance duringproject runtime is done, while all information is visualized insidea single dashboard. An expressive running example implementedas Simulink R© model will be used to show the formalization andverification workflow using the provided toolchain.

Index Terms—Requirements Formalization, Verification,Traceability, Interoperability, Consistency Analysis, ModelComplexity, Model Quality

I. INTRODUCTION

The requirement-driven design of large systems usuallyfollows a hierarchical approach: The process starts with aninitial description of the system that is broken down tofunctional (and non-functional) requirements. In parallel, thesystem is broken down into components and sub-componentswhich leads to models of different granularity including designand implementation models. During the development processdifferent challenges have to be solved. In this paper wepresent an integrated toolchain that covers a major part ofthis process involving natural language requirements, formalrequirements and executable models. The toolchain tacklesthe following keypoints that are not only considered goodpractice in software development, but also required by variousstandards for safety critical systems such as ISO 26262 [1]and IEC 61508 [2].

a) Completeness, Correctness and Consistency: For highdevelopment quality, completeness, correctness, and consis-tency [3] must be ensured between all levels of the design

The work has been partially funded by the German Ministry for Educationand Research (BMBF) under the funding ID 01IS15031H (ASSUME).

process and within the artifacts on each level. Requirementson one level of the design hierarchy must be consistent witheach other and complete with respect to the requirements onhigher levels.

Following the design flow, the design and implementationmodels must be complete and correct with respect to therequirements, and the implementation must be complete andcorrect with respect to the models.

As correctness and completeness are transitive relations, en-suring them between consecutive levels in the design hierarchyimplies a correct and complete implementation with respect tothe textual specification.

b) Traceability: Quality assurance needs traceabilityon system level and implementation level. The presentedtoolchain provides traceability between natural language re-quirements, their formal representation, model elements andtest cases.

c) Model quality: The (non-functional) quality of designand implementation models is essential for effective andhigh quality development. In order to achieve understandableand maintainable models, model complexity and guidelinecompliance must be tracked.

d) Overall quality monitoring: Since the developmentprocess is accompanied with a quality process, quality mon-itoring for the complete product development is required.Concise and comprehensive information about the status ofdevelopment and product quality must be available at any time.For effective quality assurance, a centralized view on qualitydata is necessary that includes measurements for complete-ness, correctness and consistency of requirements, design andimplementation as described above.

A. Toolchain needs

Functional requirements are checked using different toolsand technologies such as model in the loop (MIL), softwarein the loop (SIL), static analysis, or tests. How a functionalrequirement is checked depends on the complexity of thisrequirement and the amount of the tools used. While thecollaboration of “specialized tools” instead of one “universaltool” leads to more effective and efficient analysis results aseach tool can show its strength on the other hand an deepintegration of tools is necessary.

2

In order to enable validation of completeness, correctnessand consistency we propose to use formal methods in thedesign process. For an effective, maintainable, and high-quality use of formal methods, a simple, but yet powerful,formalism together with a methodology to formalize thetextual requirements is needed. Formal requirements have un-ambiguous, machine-consumable semantics and thereby allowformal reasoning.

The check procedure is getting more expensive the morehuman interaction and the more specialized software productsare used. The input formats to specify functional requirementsand verification procedures differ significantly between toolsand used technologies. Therefore, it is expensive or evenpractically impossible to move a check procedure for onerequirement from one tool or technology to another even ifthis would save effort in the verification phase. To tackle thisproblem, it is necessary to have an interoperable toolchain forformalization of functional requirements, formal verification,evaluation and model quality. From our view, the main aspectof interoperability is that requirements and models have thesame semantics among all the tools in a chain. Becausetranslating between different notations is error-prone, the sameset of formal requirements and the same model files shall beused for the different verification, validation and generationtasks. For the same reason, results must be exchanged betweentools without manual editing.

To show a comprehensive quality trend for the wholeproject, quality metrics are required. The tools need to exportquantitative measurements that abstract from the detailed re-sults of verification and validation. Also the status of validationactivities, such as test coverage, must be reflected.

B. Achievements in this work

In this paper we present a sample toolchain that providessolutions for ensuring consistency, correctness, model quality,centralized quality monitoring, and partially traceability. Wepresent each tool in the chain and the interplay betweenthe tools based on a running example from automotive in-dustry. The running example is initially developed in theSPES XT [4] project and is now extended with code genera-tion to fit the needs of the ASSUME [5] project in which thecollaborative toolchain was developed. Although the runningexample has low safety critically, the methods for analysis andtest demonstrated in the toolchain are usually applied to safetycritical systems. They are recommended in ISO 26262 for anyelement that needs compliance with the standard [1].

II. RUNNING EXAMPLE: CORNERING LIGHT

The running example represents an automotive adaptivelight system (AL) which contains functionality like adaptivehigh and low beam, turn signaling, cornering lightning and am-bient lightning. The functionality which is chosen to elaboratethe interoperable toolchain is the cornering light. Corneringlight is illuminating the area ahead and to the side of thevehicle to take a look around the bend. This functionalityis activated when the indicator or turn the steering wheel isoperated during night-time driving. For more details see [6].

MES Model-based Platform

Automotive Development

Process

Design-Models

Environment Models

BTC

Embedded Platform® (EP)

OFFIS Research Results OFFIS Consistency

Analysis

BTC EP Requirement Based Testing

BTC EP Formal Verification

MES Model Examiner® (MXAM)

MES M-XRAY® Model Metrics

Textual Requirements

Formalized Requirements

V&V Results

BTC EP Formal Specification

Implementation

MES Quality Commander®

Build Information

Quality ensured over tim

e

Fig. 1: Overview Interoperable Toolchain for Requirements-Driven Model-Based Development

In general, the functionality represented by the runningexample is highly distributed on various ECUs (ElectronicControl Units) that communicate via the car’s vehicle busses,such as CAN, LIN, and Automotive-Ethernet in an AUTOSARenvironment. The running example abstracts this environment,as in context of the interoperable toolchain, the textual (in-formal) requirements and model representation are in focus.Containing 113 selected functional and non-functional require-ments it is still big enough to show up realistic challanges. Theimplementation is done using the model-based developmenttools Simulink R© and TargetLink R© where TargetLink R© isused to generate ANSI-C source code. The following excerptintroduces three requirements describing the cornering lightfunctionality as part of the adaptive light system and willbe used to examplify the interoperable toolchain in the nextsections.

AL-43 If the low beam headlights are activated, direc-tion blinking is requested and when the vehicledrives slower than 10 km/h the cornering light isactivated. 10 seconds after passing the corner (i.e.the direction blinking is not active any more for10 seconds), the cornering light is faded out in aduration of 1 second.

AL-122 With subvoltage the cornering light is not avail-able.

AL-139 With activated darkness switch (only armored ve-hicles) the cornering light is not activated.

According to textual requirement AL-122 a subvoltage ispresent if the voltage in the vehicles electrical system is lessthan 8.5V. The darkness switch mentioned in textual require-ment AL-139 disables additional adaptive light functiionalitywith the push of a button.

III. COLLABORATIVE TOOLCHAIN FOR IMPROVED V&V

An overview of the collaborative toolchain is providedin Figure 1. The workflow starts with textual requirementsdescribing the cornering light. Implementation (Simulink R©)and build information to generate ANSI-C code are initiallypresent in our running example. In a real-world workflow,design- and environment models would be used as well. In the

3

BTC EP Matlab Plugin MATLAB ® (incl. Simulink/TargetLink)

MES MXRAY ®

MES MXAM ®

«XML file exchange»

«XML file exchange»

MES Quality Commander ®

OFFIS Consistency Analysis

BTC EP Requirement Based Testing

BTC EP Formal Specification

BTC EmbeddedPlatform ®

Fig. 2: Interfaces between the tools

example we start directly with an implementation model. Sincethe example system has little interaction with the environ-ment, we include the most important environment constraints(e.g. value ranges) in the specification but do not modelthe environment’s behavior. As we do not present designand implementation models here, they have been greyed-out in the figure. The textual requirements are formalizedwithin BTC EmbeddedPlatform R© (BTC EP) by using thegraphical specification language simplified universal pattern(SUP, cf. Figure 3 in subsection III-B) [7]. Then the formalrequirements are checked for consistency, and afterwards theimplementation is verified against them using requirements-based testing of BTC EP. Since we omit a design model inour example, we run MIL and SIL tests directly against theimplementation model. The inputs of the system are fullydefined in the testcases, therefor we can run the tests withoutimplementing an environment model. Using generated test-cases, these are correct by construction with respect to theformal requirements. In a complete workflow, the correctnessof design-models could be proven by the formal verificationfeature of BTC EP.

Besides checking correctness and consistency, we evaluatethe model quality using guidelines and metrics in MES ModelExaminer R© and MES M-XRAY R©. Results from all steps arecollected and displayed with the help of the MES QualityCommander R© which tracks quality over all revisions of the de-velopment cycles. The different tools used in the collaborativetoolchain are integrated so that they exchange their informationfor a better analysis and presentation of test results.

The component diagram [8] in Figure 2 shows which toolshave interfaces to each other. The data between tools isexchanged as XML files as depicted by the lollypop notationin the diagram: The consistency analysis consumes XML filesfrom BTC EP; MES Quality Commander R© consumes XMLfiles from all the other tools in the chain. This reduces theoverall manual effort of data maintainance to a one-click fileexport and import. Formal specification and requirement basedtesting are part of one tool, BTC EP, and hence share the same

database. The BTC EP integrates with Simulink/TargetLink viaa plugin in the MATLAB environment—the communicationbetween BTC EP and MATLAB is fully automated and hiddenfrom the user. There is no need for an interface betweenBTC EP and MES MXAM/M-XRAY R© since the model qual-ity analysis is independent from formal testing.

A. Model Quality Analysis

Today a mature set of metrics is used to evaluate the non-functional quality of a software system. These metrics measurevarious complexities, guideline conformances and anti-patternoccurrences on code level in order to rate the maturity andmaintainability of a (future) product. The increasing use ofmodel based development tools and code generators movesthe non-functional quality concerns from code to model level.The used metric set has to follow the shift and also providecomparable statements about maintainability and maturity onmodel level. Overall quality statements for products withmodel based development and without shall be comparable.

The static analysis tools used in the collaborative toolchainprovide mechanisms to measure complexity metrics, guidelineconformances and anti-pattern occurrences for models fromSimulink R©, Stateflow R© and TargetLink R© as well as mech-anisms to derive aggregated metrics which indicate maturityand maintainability of a complete set of models for a product.

During the design flow, the OFFIS consistency analysisand the requirement-based testing feature of BTC Embed-dedPlatform R© evaluate and export quantitative measurementsfor completeness, correctness and consistency. Particularlythese are consistency status, test status, formal requirementstatus, and formal requirement coverage. These values abstractfrom the actual results of consistency analysis, test generation,MIL, and SIL testing and yield a comprehensive and com-parable view on the project status: A consistency status of100% means that no inconsistencies have been detected. Highvalues for test status and formal requirement status togetherwith a high formal requirement coverage ensure correctnessand completeness of the design model with respect to therequirements.

B. Formal Specification with BTC EmbeddedPlatform R©

The formal specification feature of BTC Embed-dedPlatform R© (BTC EP) deals with creating and managingsemi-formal and formal requirements for safety-criticalprojects. While a semi-formal and formal notation ofrequirements is recommended in standards like ISO 26262-8:2011 [1] and IEC 61508-3:2010 [2], most formal notationlanguages require a high level of expert knowledge to bothread and write them. Instead of having to create formalrequirements from scratch, BTC EP provides an intuitivemethod to derive comprehensible formal requirements fromnatural language without expert knowledge. As naturallanguage typically leaves some room for interpretation, theone important benefit of a semi-formal or formal notation isthat informal requirements can be transformed into a clear,unambiguous, and machine-readable representation, thus

4

Fig. 3: Formal specification of natural-language requirement AL-43 using the SUP editor of BTC EmbeddedPlatform R©.

improving their quality and making them much more valuablefor the following steps in the development workflow.

There are mainly two primary reasons why formal notationshave not yet gained more acceptance in embedded develop-ment projects. Formal languages like LTL or CTL [9] areoften considered to be too difficult and do not provide enoughtraceability towards existing informal requirements. To addressboth issues efficiently, BTC EP provides an intuitive andgraphical specification formalism called simplified universalpattern (SUP), cf. Figure 3. An SUP exploits the intrinsicnature of functional requirements by directly specifying atrigger/action relation and thus reflects explicitly the essentialartifacts of the natural-language requirement.

A typical formalization workflow starts with identify-ing these textual artifacts of the informal requirement asso called macros. For an example, consider the identi-fied macros in the first sentence of the textual require-ment AL-43 as shown in Figure 3 where, e.g., the textualartifact “low beam headlights are activated” is identifiedas macro $lowBeamHeadlightActivated. Thereafter,these macros are structured to define their relationship and tim-ing behavior by using the graphical SUP editor, and moreoverserve for traceability as it establishes a connection between thetextual and formal requirement. Concerning requirement AL-43, the SUP formalization and in particular the definitionof the trigger and action part can be easily derived fromthe (well-written) textual description. More details about theexpressiveness of the SUP language and its semantics canbe found in [7], [10]. Regarding usability, the SUP editorinitially shows a very simple structure, i.e. not all SUP artifactsare visible, while more artifacts can be activated on demand.That means, the SUP is kept as simple as possible and

as complex as needed and thus facilitates understandability,documentation, and reviewing. After having achieved sucha semi-formal representation of the requirement, it remainsto map all macros to real interface objects of the systemunder test such that the requirement becomes formal andmachine-readable. This action further improves tracebility asthe informal and formal requirements are additionally linkedto the corresponding interface objects of the system undertest. The finalization of the formalization process then allowsfor further and fully automatic use cases like formal testing,requirement-based coverage measurement & test case gener-ation, formal verification [7] as well as consistency analysis,as being explained in subsection III-C, establishing the addedvalue of formalizing requirements.

C. Consistency Analysis by OFFISThe consistency analysis tool is a research prototype devel-

oped by OFFIS. It demonstrates the use of formal methodsin order to verify that a set of requirements is free ofcontradictions. This is named internal and external consistencyin ISO 26262-8:2011 [1] and is a required property for anyset of requirements.

The tool validates consistency by symbolic execution offormal requirements. The basic idea is to find an executiontrace for the system – a sequence of steps where a concretevalue is assigned to each macro – that contains a completeobservation of every requirement. This is called existentialconsistency and explained in detail in [11]. If the tool finds asatisfying trace it is displayed to the user. If no trace is found,the analysis tries to find either a maximum consistent or mini-mum inconsistent subset of requirements which helps the userto locate faulty requirements. However, existential consistency

5

is not sufficient to find all cases of inconsistency. For example,the contradiction between requirements AL-43 and AL-139as introduced in Section II is not found: Requirement AL-43 states that direction blinking activates the cornering lightbut AL-139 prohibits activation of the cornering light withactivated darkness switch. If the driver uses direction blinkingwith activated darkness switch, the requirements contradict.However, AL-43 and AL-139 are existential consistent. Ina satisfying trace, the darkness switch could be activatedafter the cornering light fades out. Therefor the tool supportsanother consistency notion called partial consistency: Here,all subsets of the requirements are checked for existentialconsistency under the additional assumption that their trig-gers occur synchronously (provided that the triggers do notcontradict). Now the tool checks exactly for the describedcase – direction blinking with activated darkness switch –and displays the inconsistency. Figure 4 shows a screenshotof the tool displaying the inconsistency between AL-43 andAL-139. The top part shows the observation status of therequirements (yellow: triggered, green: successful, red: failed).The bottom part shows macro traces. We read the figure asfollows: At step 1 the right turn indicator is switched of whilelow beam lights and darkness switch are on which triggersthe formal requirement $F_CorneringLightOffRight,indicated by a yellow bar starting in step 1. This is part of theformalization of AL-43 saying that the cornering light stayson for at least 10s. To satisfy AL-43 the cornering light is onin step 1, which in turn violates AL-139, indicated by a redbar.

Fig. 4: Screenshot of the consistency analysis.

In the current workflow, the user exports the formalizationfrom the BTC EP in an exchange format and imports the fileinto the analysis tool. Although the migration is a manualstep, the tools are integrated in our sense: The user nevertouches the file contents, so he/she will not corrupt data duringthe process. Furthermore both tools use the SUP and namesof requirements and macros are the same. The export ofconsistency metrics for visualization in MQC works the same

Fig. 5: Requirements-based testing with BTC Embed-dedPlatform R©.

as from BTC EP. The exported metrics from the consistencyanalysis and BTC EP use the same IDs to identify componentsso the data in MQC gets related.

After finding the inconsistency, the user returns tothe BTC EP and modifies the formalization of AL-43. For our running example we add an exception to$F_CorneringLightOffRight that honors an activateddarkness switch. The process is repeated until all inconsisten-cies have been removed.

D. Requirements-Based Testing with BTC Embed-dedPlatform R©

BTC EmbeddedPlatform R© supports manual and automatedrequirements-based testing facilities. In requirements-basedtesting, the correct implementation of functional requirementscan be verified using dedicated test cases, namely against themodel implemented in Simulink R© and the production codegenerated using TargetLink R©. This process can be performedin a traditional way, in which functional test cases are manuallylinked to informal requirements and specified using dedicatedtest specification techniques.

Besides the traditional approach, BTC EP offers to auto-mate this traditional manual process by taking advantage offormalized requirements using the SUP language as describedin subsection III-B. The SUP formalism comes along with anintuitive requirement coverage notion, cf. [10, Section 2.2] formore details. This means that using formalized requirements,achieved detailed coverage of test cases wrt functional require-ments can be automatically measured. Additionally, by havingthis measurable coverage notion available, automatic test casegeneration for functional requirements can be applied [10,Section 3.2]

Figure 5 gives an impression of how these requirements-based test results are integrated into BTC EP: while the y-axis corresponds to the concrete architecture of the systemunder test, e.g. model in the loop (MIL) and software inthe loop (SIL), the x-axis shows results for the differenttest metrics based on the available (manually created orautomatically generated) test cases. The test status reflects thetraditional requirements-based test where the simulated testcases are compared to their expectations given, e.g., by sometest verdict functions like absolute or relative tolerances. The

6

formal requirement status shows the relative number of formalrequirements which are definitely satisfied by some test cases.Using color coding and additional textual information, it isimmediately indicated whether requirement violations wererevealed during test execution. The column formal requirementcoverage measures the number of covered formal require-ments. A requirement is classified as covered if its actualcoverage exceeds some user-defined threshold. For coveragemetrics of single requirements, we refer the reader to [10,Section 2.2].

In early stages of the development process, the systemunder test may not have sufficient quality as, e.g., indicatedin Figure 5 where some requirements are not fully coveredor, even worse, some requirements are actually violated. Tosupport the system engineer in enhancing the test qualityor in understanding and eliminating the malfunction of thesystem under test, BTC EP provides very detailed debuggingpossibilities. For instance, the uncovered parts of a formalrequirement are revealed or system executions violating aformal requirement are illustrated in a very lucid and com-prehensible way, linking each execution step to the currentstatus of the SUP as shown in Figure 6. BTC EP additionallyprovides the possibility of exporting test environments of thesystem under test, given as Simulink R© or TargetLink R© modelor as production code, which then allows for simulating anddebugging the abnormal system behavior for the correspondingcritical scenarios.

To track the progress of such iterations in a developmentproject wrt requirements-based test status in tools like MESQuality Commander R©, as described in subsection III-G, adedicated plugin was developed to export the current statusof the requirements-based test, cf. the button “MQC Export”in the screenshot of Figure 5.

E. Analysis on best modelling practices with MES ModelExaminer R© (MXAM)

Models which are used as source for the generation ofcode shall follow best practices for such kind of models.Otherwise, code generation may lead to insufficient results oreven fail. Similar to the well-known MISRA-C [12] guidelines,there is a comprehensive set of guidelines on good modelingpractice, which facilitate the creation of models being sourcefor the generation of code. One of the earliest set of guidelineshas been defined by the Mathworks Automotive AdvisoryBoard (MAAB) targeting at readability of models which isan essential precondition to successful maintenance of modelsand team work [13]. MISRA has also compiled a set ofguidelines dedicated to the creation of robust models [14].Additional sets of guidelines are supporting the use of codegenerators to produce code from the Simulink model. As formany compilation steps, also the generation of code is notfeasible for arbitrary models. The supported language subsetmust be checked in order to achieve a successful generation ofcode [15]. Furthermore, problems known from user experiencewith code generators are regularly collected by the providerof the generator in order to inform the user community aboutpotential risks in generation of code. The existence of those

scenarios can also be automatically checked, and the developerwill be able to create a robust model, suited for generationof code. In each run of MXAM, a predefined or customizedset of guidelines is defined a scope of analysis for a singleor multiple models. The MES Model Examiner R© analysisthe model(s) and determines violations of the guidelineswhich may occur at any part of a model. In general, severalfindings with individual criticality are reported. The developeror quality engineer can assess the report and decide on thetreatment of findings. All decisions are covered in the analysisreport. For the running example, a number of guidelines havebeen selected and applied to the model using the providedautomatic analysis. The findings have been assessed and areport showing the analysis results has been created. Basedon the findings, the developer is guided to modify the modelin order to improve the generation of code from the model.As the result also provides information on the quality of themodel under development, the MES Model Examiner R© hasalso been incorporated into the interoperable toolchain. Theintegration is realized by a particular adapter which is able toparse the proprietary report file structure used for persistentstorage of results of the analysis. Whenever an analysis hasbeen executed, the respective report is going to be generatedand captured by the toolchain.

F. Complexity Analysis with MES M-XRAY R©

The assessment of complexity of a development artifact isan important means to manage a development project and toachieve a sustainable product. Firstly, from the complexityof a model, one can e.g. estimate the amount of expectedreview or validation efforts improving the accuracy of resourceplanning. Secondly, detailed information on the complexitydistribution within a model gives a valuable indication onthe quality of the model: unequally balanced model structuremay lead to overhead in the management of software units.MES M-XRAY R© has been applied to the running exampleand the distribution of complexity has been reported. Highlycomplex parts of the model have been identified and should befactored in order to improve the overall quality of the model.Again, similar to the integration of MES Model Examiner R©,a particular adapter captures the results of the analysis byparsing the result file.

G. Quality Monitoring with the MES Quality Commander R©

The interoperable toolchain provides a comprehensive set ofanalysis tools all contributing to a better understanding of thequality of development artifacts. The large number of analysistools however has the risk that important information get lostand will not be recognized early enough. Project managementmust be able to react immediately in order to avoid delaysof development due to late observation of issues. Therefore,during development, it is mandatory to track the evolution ofquality over time based on the analyses executed. That infor-mation on quality can mandate immediate action whenever therisk of missing quality reaches certain thresholds. Because thedata exported by the tools contains raw quantitative data (e.g.

7

Fig. 6: Tabular representation of a system execution violating a formal requirement.

Fig. 7: Quality Monitoring with the MES Quality Commander R©

number of passed and failed test cases), the predefined metricsare configurable. It is even possible to define new metrics.

Keeping track with the quality during development is thekey feature of the MES Quality Commander R©. It collectsany results from quality assurance activities and serves as asingle source for quality data over time. Since the number ofdate easily exceeds tractable size, great importance has beenlaid onto methods for data aggregation. A highly configurablequality model allows structuring the information on quality ina hierarchical manner. At the top level, the quality of the wholedevelopment project is shown, being decomposed into variousaspects like requirements quality, model quality, or codequality, down to individual analyses. A second aggregationhierarchy is established based on the composition structure ofthe product. Consequently, root causes on missing quality caneasily be determined and an early warning system is available.For the running example, the result data of the analyses havebeen collected and the type of information has been gatheredin a quality model. The example in Figure 7 shows that LocalComplexity Metric has not been changed significantly, thecompliance to modeling guidelines has been achieved overtime and that the “Test Status” has been heavily improved inline with “Formal Requirements Coverage” and “ConsistencyStatus”.

IV. OVERALL RESULTS AND FUTURE WORK

In the following we show some results from applyingthe toolchain to the running example. Detailed quantitativemeasurements on process improvements are future work to bedone in the ASSUME project.

The textual requirements AL-43 and AL-139 describingthe cornering light functionality (explained in Section I) areimplemented as a total of 11 formal requirements using theprovided SUP pattern. In addition to the formal requirementscreated, the running example consists of 13 environmentalassumptions that are used for the definition of e.g. the ignitionkey, light switch or engine status. Environmental assumptionscontain general conditions that are mentioned by multipletextual requirements. Therfore they are also assigned to morethan one formal requirement. This feature helps to keep theindividual SUP small and understandable as it is focused onits individual feature mentioned in the textual requirement.

For a successful testcase generation additional four formalrequirements were generated to realize the rotary light switchand turn signaling which are needed as precondition for thecornering light functionality. Two formal requirements addresssubvoltage and darkness switch which are mentioned in textualrequirements AL-122 and AL-139.

The running example contains five model calibration values.They are used to enable or disable specific model functionalitydepending on the vehicle configuration. Therefor the func-tionalities “cornering light” and “darkness switch” have tobe parameterized as they are an optional feature inside theadaptive light system. This is covered by the SUP. A total of44 individual macros were defined. These are specific textualartifacts of the informal requirements (see Section III-B). Likethe environmental assumptions they can be used in morethan one formalized requirement for an easier formalizationprocess. The formalized requirements use 26 inports and 25outports of the corresponding components.

8

In model revision 6 (see Figure 7) 13 successful generatedtestcases (see Figure 8) exists, which were executed in MILand SIL using BTC EP.

We found three scaling incompatibilities during SIL con-cerning the scaling settings of the running example which werenot found during MIL tests. The effort required to set up andemploy an analysis using formalized requirements are reducedas the formalization step using the SUP pattern is much easierto understand for the developer compared to a direct specifi-cation using LTL or CTL. Automated testcase generation is anadditional important feature during the development process.

The consistency analysis by OFFIS can handle a newerror classes, the detection of “Partial Inconsistency of Re-quirements” and the BTC EmbeddedPlatform R© reports now“Missing Requirements Coverage”. As the consistency analy-sis explicitly shows inconsistency this helps the developer tofind inaccurate requirements at an early stage. This results inefficiency gains for requirements engineering and testing inthe development process (see Section III-C). One requirementinconsistency was found inside the running example (betweenAL-43 and AL-139).

From the diagram shown in the left section of Figure 7one can determine that the quality of the model has signif-icantly improved with respect to the adherence of modelingguidelines. From an initial level of around 20% of quality,modifications and improvements of the model have achieveda compliance of nearly 90%. In addition, one can derive fromthe trend information on local complexity measured by MESM-XRAY R© that the modifications to the model have not beenthat significant. The quality aspect of local complexity hasnot dropped below 90%. Both trend information indicate, thatthe fixes of findings found by modelling guidelines had beenmostly of local and small impact.

The definition, application and visualization of consistentquality assessment over time ensures that the developed func-tionality is on the right way (cf. Figure 7). The aggregationof individual results and the precision of the reported resultshelps to identify concrete sources of actual problems. Therequirement coverage can be measured with greater preci-sion. The integrated quality dashboard with its iterative datacollection and trend representation of: requirements, coveragedata, guideline conformance and complexity metrics helps toidentify problems in an early development stage. An improvedquality assessment including project history is given by evalu-

Fig. 8: Testcases created by BTC EmbeddedPlatform R©.

ating the joined data inside MES Quality Commander R©. Themeasurements done are able to handle the six revisions of therunning example, but scale for more revisions.

During our evaluation, we developed new ideas for theconsistency analysis and data integration. A variety of differentdefinitions exist that describe if a set of requirements isconsistent (see Section V). It turned out that partial consistencyfinds more practical relevant inconsistencies than existentialconsistency that was applied to pattern-based requirementswith similar semantic in previous work (see [11]). However,there are still intuitive inconsistencies that are not foundbecause the interactions between requirements may be morecomplex. Though the trigger/action scheme in the SUP is usedfor defining and analyzing partial consistency, ongoing andfuture work tries to use more information from the SUP toform an even more complete but efficient consistency analysis.The consistency analysis tool is currently a research prototypeonly. If it is adopted as a feature in a commercial tool, i.e.the BTC EmbeddedPlatform R©, in future, it will be qualifiedaccording to a safety standard.

Though the SUP specification language received very pos-itive feedback from industrial users1, in particular due to itsintuitive and graphical nature, and though a vast majority ofreal-world (safety-critical) requirements can be formalized asSUP, there recently is a rising industrial need of enhancing theexpressiveness of SUP, which will be addressed in future work.By way of example, some requirements involve a whole seriesof trigger and action phases, e.g. to describe signal curves.Another request deals with user-defined timers within thespecification, e.g. to directly limit the duration between startingthe trigger and finishing the action. Another very expressivefeature would be to support user-defined functions within thespecification in order to use very complex computations in aneasy way.

The static analysis of compliance to modelling guidelinesis well established and mature, so is the computation of com-plexity. For both cases, though, repair guidance and supportis highly required. In particular for large models of highcomplexity, proposals for refactoring steps are needed in orderto reduce the complexity to a acceptable limit. At present, theimpact and benefit of a particular refactoring, e.g. by meansof restructuring single subsystems or by replacing structuralelements, cannot be anticipated prior to the modification. Tothe contrary, developers have to use a trial-and-error approachto find appropriate refactorings. In future, therefore, moreeffort has to be spent on patterns and refactoring operationswhich lead to tangible results.

The power of the collection and aggregation of qualitymeasures highly depends on the availability of data. Forthat purpose, additional adapters have to be provided suchthat further analysis tools will be connected to MES QualityCommander R©. In order to facilitate the integration of resultdata, a generic XML schema has been defined. That schemacan now serve for several modes of interaction, depending

1There are some hundred industrial users of BTC EP employing formalmethods like formal specification and formal verification in industrial projects.For some success stories confer https://www.btc-es.de/en/company/references.html

9

on the core technology used to establish a tool chain. Thesimplest mode of interaction may be exchange of files, i.e.an XML-file will be generated by the analysis tool andimported into MES Quality Commander R©. In case a toolchain is using OSLC as interaction technology, the XMLschema will serve for the definition of an OSLC domain. Oneparticular quality analysis will be of additional importance forMES Quality Commander R©. Since the tool already accessesnumerous analysis tools, it will be of high value to also assessthe quality of traces between the various tools and the artifactsbeing processed by the tools. However, quality of traces hasnot been fully assessed, but the mere trace management is stillchallenging. Once, a single source of trace information hasbecome operational, additional effort will have to be spentfor the quality analysis of trace information. The results ofthat analysis will become valuable input to MES QualityCommander R© being an overall platform for integration andvisualization of the overall quality state of a development.

At present, the integration of analysis results provides animportant benefit to developers and project managers. Thesemain stakeholders are able to

1) investigate into the current stage of quality,2) retrieve information on the past, and3) extrapolate into the future.

In order to further improve the benefits from collecting theresults of an analysis, counter actions shall be determinedfor findings of an analysis. Based on the detailed findings,developers as well as project managers may derive a list ofactions from the present state of quality. Furthermore weightedrequirements could improve the representation of the actualproject status.

V. RELATED WORK

Concerning formal specification, in the past, several graph-ical formalization languages have been proposed. Symbolictiming diagrams [16], [17] or life sequence charts (LSCs) [18]are prominent examples. Such formalisms offer tremendousexpressiveness but this expressiveness comes along withneeded expert knowledge to apply them. Approaches likethose presented in [19] (graphical pattern templates) or [20](textual pattern templates) limit expressiveness to gain betterreadability, but also benefit from reduced complexity fortableaux generation for formal verification. Though the lattertwo approaches use terms and notions to better understand theformalism, none of them inherently bases on the nature of aninformal functional requirement which impose another hurdleto requirement engineers.

The SUP language [7] used in this paper instead directlyrelates the formalism to the intrinsic composition of informalfunctional requirements by using a simple trigger/action rela-tion. Furthermore, end users are not overwhelmed with exorbi-tant expressiveness, which will also lead to better performancewhen reusing the artifacts later within formal techniques suchas formal verification or consistency analysis.

Besides the informal definitions that a set of requirements isconsistent if it is free of contradictions [1], or if there exists animplementation [21], formal definitions of consistency vary.

For example, in [21] the formal definition of consistencydepends on the underlying formalism of the requirements.In [11], [22], [23] consistency is based on the existence ofsatisfying traces. While rt-consistency [23] requires that everyvalid finite trace can be extended to an infinite one, Aicherniget al. [22] define consistency as the feasibility to produce avalid output to every input. The partial consistency introducedas new consistency notion in this toolchain circumvents boththe distinction between inputs and outputs as required by[22] and the computational complexity in [22] and [23]. Itis based on existing work [11]. Because a simple analysissetup without environment information or design models is astrength of the tool, no explicit distinction between inputs andoutputs is made. On the other hand, partial consistency doesnot guarantee liveness as in [22] and [23], wherefore we checkbounded triggered existential consistency [11] separately.

Data integration has been a long lasting topic in tool chainoptimization. Model repositories like [24] tend to collectrelevant engineering data in a single data base which becomesthe master source of information. Analysis tools will have toadapt to that central data source. Since analysis tools stronglymake use of efficient representation of model information, thecapabilities of centralized data source will not suffice, but theanalysis tools will have to replicate the centralized data andenrich it for analysis purposes. This leads to multiplication ofsources leading to the well-known issues of redundancy andconsistency in tool chains.

OSLC [25] overcomes this challenge of centralized datastorage, but provides a standardized interface to distributedengineering data. So, an intermediate access layer is providedwhich implements a distributed storage of engineering data.Any analysis tool may retrieve model data via the OSLCaccess layer. Still – and similar to the centralized data storage–, the analysis tools have to extend the model information bydedicated supplementary information facilitating the executionof analysis. In future, InterOperability Specifications (IOS)may be defined on top of the model interchange formats inorder to enable the reuse of enriched information betweenanalysis tools.

The ToolNet framework has been an earlier approach to theimplementation of a traceability middleware which focusedon the decentralized distribution of engineering data [26],[27]. Similar to OSLC, data were addressable by URLs. Bymeans of a centralized trace repository, navigation betweenand remote presentation of engineering data were realized. [28]used the framework in order to collect data for the creation ofsafety cases.

The SQaRE family of standards (see ISO / IEC 250xx [29])on Software Product Quality provides a comprehensive frame-work for the systematic collection and compilation of qualityproperties. Individual results of analysis, like the consistencyanalysis on requirements serve as basic measurements. Thesemeasurements require an interpretation in order to derive astatement on quality. The MES Quality Commander adoptsand implements the methodology given by ISO / IEC 250xx.

In [30], [31] integrated toolchains have been presented thatcover tracing of requirements to model elements and test casegeneration. The integration of result data of analysis tools for

10

quality monitoring, as presented in this contribution, has notbeen covered in any related toolchain known to the authors.Instead data integration has been in the focus. The toolchainin this paper currently relies on file-based data exchangeinstead of OSLC resp. ModelBus which is used in relatedwork [30], [31]. Future versions of the toolchain will integratethe consistency analysis as a plugin to the BTC Embed-dedPlatform R©2 and may replace the XML file exchange toMES Quality Commander R© by some middleware technology.This will further tighten the integration and traceability.

REFERENCES

[1] ISO 26262-8:2011: Road vehicles — Functional safety — Part 8:Supporting processes.

[2] IEC 61508-3:2010: Functional safety of electrical/electronic/pro-grammable electronic safety-related systems - Part 3: Software require-ments, Edition 2.0.

[3] D. Zowghi and V. Gervasi, “On the interplay between consistency,completeness, and correctness in requirements evolution,” Information& Software Technology, vol. 45, no. 14, pp. 993–1009, 2003. [Online].Available: https://doi.org/10.1016/S0950-5849(03)00100-9

[4] SPES XT - Software Plattform Embedded Systems XT, 2014 (lastaccessed June 12, 2017). [Online]. Available: http://spes2020.informatik.tu-muenchen.de/spes xt-home.html

[5] ASSUME - Affordable Safe & Secure Mobility Evolution, 2017 (lastaccessed June 12, 2017). [Online]. Available: https://itea3.org/project/assume.html

[6] Mercedes-Benz TechCenter, 2016 (last accessed June 12, 2017).[Online]. Available: http://techcenter.mercedes-benz.com/en/corneringlight/detail.html

[7] T. Teige, T. Bienmuller, and H. J. Holberg, “Universal pattern: For-malization, testing, coverage, verification, and test case generation forsafety-critical requirements,” in MBMV, 2016, pp. 6–9.

[8] J. Rumbaugh, I. Jacobson, and G. Booch, The Unified Modeling Lan-guage reference manual, 2nd ed. Boston: Addison-Wesley, 2005.

[9] J. van Leeuwen, Ed., Handbook of Theoretical Computer Science (Vol.B): Formal Models and Semantics. Cambridge, MA, USA: MIT Press,1990.

[10] T. Bienmuller, T. Teige, A. Eggers, and M. Stasch, “Modeling re-quirements for quantitative consistency analysis and automatic test casegeneration,” in Workshop on Formal and Model-Driven Techniques forDeveloping Trustworthy Systems at 18th International Conference onFormal Engineering Methods, 2016.

[11] C. Ellen, S. Sieverding, and H. Hungar, “Detecting consistencies andinconsistencies of pattern-based functional requirements,” in FormalMethods for Industrial Critical Systems - 19th International Conference,FMICS 2014, Florence, Italy, September 11-12, 2014. Proceedings,2014, pp. 155–169.

[12] MIRA, Guidelines for the Use of the C Language in Critical Systems,2013.

[13] Control Algorithm Modeling Guidelines Using MATLAB, Simulink, andStateflow, 2012 (last accessed June 12, 2017). [Online]. Available:https://de.mathworks.com/content/dam/mathworks/mathworks-dot-com/solutions/automotive/standards/docs/MAAB Style GuidelineVersion3p00 pdf.zip

[14] MIRA, MISRA AC SLSF: Modelling design and style guidelines for theapplication of Simulink and Stateflow, 2009.

2This has been presented in [11] for previous versions of the tools.

[15] dSPACE Modeling Guidelines for TargetLink, 2017 (last accessedJune 12, 2017). [Online]. Available: http://www.dspace.com/shared/TargetLink/files?dl=TL ModelingGuidelines403.pdf

[16] R. Schlor and W. Damm, “Specification and verification of system-levelhardware designs using timing diagrams,” in European Conference onDesign Automation. IEEE, 1993, pp. 518–524.

[17] H. Wittke, “An environment for compositional specification verificationof complex embedded systems,” Ph.D. dissertation, University of Old-enburg, 2005.

[18] W. Damm and D. Harel, “LSCs: Breathing life into message sequencecharts,” FMSD, vol. 19, no. 1, pp. 45–80, 2001.

[19] T. Bienmuller, W. Damm, and H. Wittke, “The STATEMATE verificationenvironment - making it real,” in CAV, ser. LNCS, vol. 1855. Springer,2000, pp. 561–567.

[20] M. B. Dwyer, G. S. Avrunin, and J. C. Corbett, “Patterns in propertyspecifications for finite-state verification,” in ICSE. ACM, 1999, pp.411–420.

[21] A. Benveniste et al., “Contracts for system design,” INRIA ResearchCentre, Tech. Rep. 8147, 2012.

[22] B. K. Aichernig, K. Hormaier, F. Lorber, D. Nickovic, and S. Tiran,“Require, test and trace IT,” in Formal Methods for Industrial CriticalSystems - 20th International Workshop, FMICS 2015, Oslo, Norway,June 22-23, 2015 Proceedings, 2015, pp. 113–127.

[23] A. Post, J. Hoenicke, and A. Podelski, “rt-inconsistency: A new propertyfor real-time requirements,” in Fundamental Approaches to SoftwareEngineering - 14th International Conference, FASE 2011, Held asPart of the Joint European Conferences on Theory and Practice ofSoftware, ETAPS 2011, Saarbrucken, Germany, March 26-April 3, 2011.Proceedings, 2011, pp. 34–49.

[24] ModelBus R© Tool Integration Framework, 2014 (last accessedJune 12, 2017). [Online]. Available: https://www.modelbus.org/en/modelbusoverview.html

[25] M. Biehl, W. Gu, and F. Loiret, “Model-based service discovery andorchestration for oslc services in tool chains,” in Web Engineering: 12thInternational Conference, ICWE 2012, Berlin, Germany, July 23-27,2012. Proceedings, M. Brambilla, T. Tokuda, and R. Tolksdorf, Eds.Berlin, Heidelberg: Springer Berlin Heidelberg, 2012, pp. 283–290.

[26] F. Altheide, H. Dorr, and A. Schurr, “Requirements to a frameworkfor sustainable integration of system development tools,” in EuSEC ’02,2002, pp. 53—-57.

[27] R. Freude and A. Konigs, “Tool integration with consistency relationsand their visualisation,” in Proceedings of Satellite Workshop on theIntegration in System Development of ESEC/FSE 2003, A. Schurr andH. Dorr, Eds.

[28] W. Ridderhof, H.-G. Gross, and H. Doerr, “Establishing evidence forsafety cases in automotive systems – a case study,” Delft University ofTechnology, Software Engineering Research Group, Technical ReportSeries, Tech. Rep.

[29] ISO / IEC 250xx:2011: Systems and Software Quality Requirements andEvaluation (SQuaRE).

[30] B. K. Aichernig, K. Hormaier, F. Lorber, D. Nickovic, R. Schlick,D. Simoneau, and S. Tiran, “Integration of requirements engineeringand test-case generation via oslc,” in Quality Software (QSIC), 201414th International Conference on. IEEE, 2014, pp. 117–126.

[31] E. Armengaud, M. Biehl, Q. Bourrouilh, M. Breunig, S. Farfeleder,C. Hein, M. Oertel, A. Wallner, and M. Zoier, “Integrated tool-chain forimproving traceability during the development of automotive systems,”in Proceedings of the 2012 Embedded Real Time Software and SystemsConference, 2012.