COWP-9104106—4-Dtaf t

011370

,-r: AMERICAN POWER CONFERENCE PAPER

OR VERIFICATION AND VALIDATION CF WJCIZ.-..7 zr.:nR PLANT::::TROL SYSTEM SOFTWARE7'

ABSTRACT

The ::ii:wing guideiir.es are proposed for verification and validation (V&V) of:v::lsar power plant control system software: (a) Use risk management to decide..-.-.at ar.d hew much vsv is needed; (b) Classify each software application using aerr.erne that reflects what type and how much V&V is needed; (c) Maintain a set,f reference documents with current information about each application; (d) UseProgram Inspection as the initial basic verification method; and (e) Establish- deficiencies log for each software application. The following additionalpractices are strongly recommended: (a) Use a computer-based contiguration-.ar.agement system " track all aspects of development and maintenance; (b)Uitasiish reference case-ir.es :f -he software, associated reference documents,.;-.:: -.svelcpmer.t -. :;is :-.- regular intervals during development; (c) Use.^;~ct-:rientea aesigr. ar.o. programming to promote greater software reliability2.r.a reuse; (d) Provide a copy cf tha software development environment as part:f -he package ;f deliverables; and (e) Initiate an effort to use formalr.ethcds for preparation cf Technical Specifications. The paper providesbackground information and reasons for the guidelines and recommendations.

INTRODUCTION

The subject of this paper is verification and validation (V&V) cf digitalcontrol system software. The work to be described here has been funded by theICE Advanced Liquid .Metal Reactor (ALMR) Program through the Advanced ControlsProgram in the Instrumentation and Controls (ISC) Division at Oak Ridge:3-i;r.al Laboratory (ORKL) , the DOE Office of New Production Reactors, and the:.i:;ri: =;wer Researcn Institute lEPRI).

?:".= rrilcwmg guidelines are proposed:

• Use risk management as the basis for deciding what and how much VSV isr.eeaed as well as for other management decisions for software development.

• Classify ail new and existing software using a scheme consistent with the-ypes and levels of \'&V to be used.

• Maintain a basic reference set cf project documents on-line in a formeasily accessible for printing selected sections and for doing automated•teyword searches.

• Use Program Inspection as the intial basic verification method.• Set up and maintain a deficiencies log for each software application.

In addition to the guidelines above, the following additional practices arectrongiy recommended:

• Use a computer-based configuration management system to track ail aspects:i development and maintenance.

• Define and establish reference baselines of the software at regularintervals during development.

• Include copies of ail current versions cf on-line documents and

MASTERDISTRIBUTION OF THIS DOCUMENT JS UNLIMITED

DISCLAIMER

This report was prepared as an account of work sponsored by an agency of the United StalesGovernment. Neither the United States Government nor any agency thereof, nor any of theiremployees, makes any warranty, express or implied, or assumes any legal liability or responsi-bility for the accuracy, completeness, or usefulness of any information, apparatus, product, orprocess disclosed, or represents that its use would not infringe privately owned rights. Refer-ence herein to any specific commercial product, process, or service by trade n«me, trademark,manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recom-mendation, or favoring by the United States Government or any agency thereof. The viewsand opinions of authors expressed herein do not necessarily state or reflect those of theUnited States Government or any agency thereof.

. «i r.te2 I^si-r. ir.d ? roc: ri "'„•?, ir1.- *'scr.r.icjuws ".z -or.isve Greater

* ?revise s cccv cf the software ievelcc.T.ent tools uses ir. tr.e cackaoe cf.-jrscles "3 provide continuity, consistency, a::a greater reliability in the.citior. from deveicpmer.t to maintenance.

.^;: ;f tr.is presentation paper) previses background and reasons for the~jr.5r.-3ticr.s rr.ase accve.

rirre going further, a couple cf definitions and reasons for some distinctions._! i5 mace. Tefinitions most ocmmoniy used today for validation and-rifioation =re these given in .-.NSI/IZEE Std "29-ISS3, as .follows:

L-.:.:.T::::: The proeess .: -rvaluati.-.g software at the end of the software

:?,:~".-.T;;::: The crocks: :f determining •..'.-.•atr.er or r.ot the products of a given,;i; : f tr.e ^cftware -ev»_;pment r.ycie fulfill the requirements srstaolished.-..•.•-• •:."•; crevieus phase.

distinction must be made between reactor protection system software and plant:r.trol system software when discussing verification and validation because of.e requirement for very high reliability in safety-critical control systems.:cn valuable experience to date with digital control systems for the nuclear.sustry world-wide r.as come from protection systems or the safety-critical•stem ^ide :f plant operations. The qualification process has r.ot been smooth: -iasy. Efforts required to qualify protection system software are still:r.siderea to be very high compared to most applications where reliability is:uoial. Some nuclear power advocates feel they are too high. Experience;rir.r the past decade with the qualification process for digital protection;-.;-s feeas roncerr.s acout the level cf resources that will be needed to•rify i.~d validate plant control system software, both for new designs and for;grao.= s to existing plants, as the transition proceeds from use of analog

.? ::SIT.-.L CC>2T?.CL SYSTEMS ::; NUCLEAR POWER =LANTS

•::i. -echr.ciogy is being used in replacements of existing analog systems,:. ::r ir.aividuai modules and for control of major plant subsystems; toens the lives of existing plants coming up for license renewal; and ini r.s :f future plants cased on the next generation reactor technology.

_:g system replacements use embedded microprocessors to duplicate the.ctior.ality of what is being replaced. Instructions executed by thecroprccesscrs are fixea and designed to handle a small number of well-defined.'.S. Verification, testing, and validation of replacement units arel---efir.ed activities a.-.d straightforward to do. The area cf concern with: --cnr.clogy is possible appearance of unanticipated behaviors that mightour it i higner Level iue :: the interoiav of manv such modules oouoied

. ii :"ir.":»,*.5- ir. severs- capers it last year' z ;cr.£erer.ce, cr.e cf v.'nich,;-i':i55d :,".; rc_e ;;; ir.strumer.tatien inc. centre! systems with rescect to:i"iC.i;r.ir.3 =r.o. rr.air.tsir.m- eiar.t desior. basis documentation *Addisc£0] .••.!tr.:u:r.*: tr.e focus ef the paper was en reactor protection systems, the examples:resented are =11 common to oiar.t control systems. The author illustrated very.:fective>y .r.e complexity :f the many functions that control software must be:.cle to r.ar.ale reliably. Most ;f the technical challenges remain the same for:rots:tior. and control* systems, with ocvicus differences in the level of effort:; verify, zest, and validate the system software.

rcr.trcl system designs favored by potential venders of Advanced Light Water:=actcrs ,.-.lWRs) use distributed microprocessors that communicate across data•.ishwavs. Some c£ these systems, shown in Table 1, were described at last;ear's" conference [RossSO]* [DavAll90], '3ruVij50], [Wilkir.90] . A wide range•f thinking exists about what kind and hew much verification, testing, and•s.iiaticr."=re necessary to have an appropriate level of reliability and to.stair, recru-atcry approval without a let cf additional effort and delay.••."3R.-.?H: irew reactor sesicns usir.cr sistricuted microprocessors and data

•,'3?..~.r H ; Hi — r.li-r.t- *. f ^uvanceo. reactor concects

Ith-sr cnailenges to the use of digital technology in plant control than thoseusually identified (as in the references cited above) must not be overlooked.Integration cf new technology to improve performance and reliability as well as:o avoid obsolesence must be considered, riant lifetimes are expected tosxceed 40-50 years while lifetimes for specific computer technologies are now-easured in rr.cntns or a few years at most. Reductions in plant maintenancerests and increased reliability will be possible using smart sensors andrr.-lir.e data bases to assist with monitoring equipment performance or history:f repairs as well as to track maintenance activities. Such improvements in:iar.t" cperatien a-nd control depend on a wide range cf types cf software that•••ill have :: be-verified, tested, and validated to varying degrees.

rr.e r.eeas tc which a program cf verification and validation must respond arefirst :f nil reliability and availability, then items such as, improved:erfcrmar.ee, enhanced capability, and greater flexibility. Reliability is a:rimary goal. It is important to different people for different reasons, asi.':cwr. m Table I.«:»'esas -

Reliable software (Table 2}Satisfy regulators

Appropriate attention t- puoiic safety and principles z± 3oundmanagement

Satisfy investors3ood financial investment

ySafe cperation ana lew electric rates

Satisfy plant managersHigh plant avaiiabliiity and easy, reliable maintenance

Enhanced capability =nd flexibiiity»

EXPERIENCE WITH SOFTWARE VERIFICATION AND VALIDATION

Darlington Protection Systaxn

• re-cent sxserisr.ee, -ver. thcugn it pertains tc a digital protection system, is•; "Try v^luacle reference point -rcm whicn tc draw lessons for verification and• = _idatier. *7SV) tf ;ontroi system software. Atomic Energy cf Canada LimitedAECL. ana Ontario Hydro encountered problems several years ago with the AtomicEnergy Control Scard :AZC3) when they attempted to obtain a license for the .ir — ir.ctcn ^iuc~ear Cer.eratmc station protection svstem. IVeccrto of the events J:;.at followed have generated further concerns about how much =nd what kind of />^•:7 will be required tc satisfy regulators when the time comes tc obtain . / •-icenses for plant control systems.

-. nnef lock at icme highlights from the Darlington protection system willprevise = basis for evaluating tne proposed YSV guidelines for control system

'ir.asa'i: Atomic Energy Control Hoard !ASC2) required extensive revisions not,..y : o t.-.s ?ret = cti:r. system control software, hut also cf rsny aspects of.:;•,.• _t ••'•as trcaucec The initial iscrcacr. -sea sv AECL was baseu .n the "best":;t-..-are ^evelcpment tscnmques isvccated at the time ;early i:-:Csi . A.?_.csrat•= -rfftrt '-'as ™.ao.e to use i rsasonablv rautious ana conservative\pproacn which included such practices as:

• defining a software structure that reflects the actual processes to becontrolled (based on concepts of Jackson [Camero86])f• Employing a deterministic scheduling algorithm that eliminates concerns aboutprogram execution behavior associated with interrupts or multi-tasking systems,• Writing structured psueco-coae for the detailed design descriptions,• ".'sing redundancy to increase reliability,• Enforcing diversity (design groups, computers, compilers, applicationlanguages, processor chips, board layouts),• Vsir.g independence to avoid common design errors.

Approval of the protection system software was given subject tc preparingiccitiona. cooumer.tation and modify some existing documents. Concerns remained-i.~ut c.-.e acility of AECL and Ontario Hydro to sustain a reliable maintenance.recram. The concerns expressed by AEC3 were

• Code was difficult to review by a third party,• Cocumentation was difficult to understand,• reviewers were r.et confident that code could meet requirements.

AECL ::.: Cntaric Hydro were required to rewrite the protection system software£0 tr.at _t would be maintainable,

AECL and Ontario Hydro jointly addressed AEC3 concerns by making modifications.0 tne software development process. The biggest change affected the level ofeffort and attention needed at the conclusion cf the Requirements rhase andtargets problems (e.g., ambiguity) associated with use of natural language insr.y form m any phase beyond the Requirements Fhase. A major new elementfocused attention on preparation cf a Software Requirements Description, which.;.-.tair.s a detailed, mathematical interpretation of the Functional-"•ie-eificaticn expressed with

• Tables, and

• r. =i" ." ::-i. :. ;£t i ;r.s . : . ; . ' ; - ; ~r.-= :~™,c_'its _i;t :z items :.-seams attention was• :i~ 5 - r.a!

• Ili.T.ir.ate use :f structured pseuac-c:ae '.it reflects ambiguity :f natural. ir.guacre statements cf requirements) for software design,• ."ivslos and implement imorcved ccair.2 --icisiir.es,• Imcreve error handing techniques,• l.Txrive reverses cf cestir.3,• .'is :f rar.dcm testing to establish tr.e level -f reliability of the software,

• ?er::rr. formal verification {done by Ontario Hydro).

r'r.e lariir.gtcr. experience shows hew coca intentions can fall prey to rapid.r.sr.ges ir. technology. Software development standards and guidelines have not•.*pt ;-3ce with the deir.ar.a to have access to vastly expanded capabilities and.^rf:r-ance offered cy digital technology. In the time needed" to incorporateur.-. r.sw technology ir.tc r.sw designs and operating systems, large gaps can.-••.'?..? ::<3tweer. .r.e rr.gir.eerms =racti:es ;f the builder ana the resuirements

-agelec P20 Program

::uc_=cr.ios '.veek i-"an LI, 1391) reported that Cegeiec is encountering.:iiiiru-ties with its -ontrocioo ?20 decentralized control system. "Failure toior.ieve sufficient capacity to process the mass of acquired reactor data withthe :riginai Cor.trebloc ?20 architecture ... had led to "increasingly complex=;ftware programs' with 'increasingly numerous interactions between subsystems';i the processing structure, which is basea on a cluster approach."l.-:pettations expressed in the growing list cf requirements resulted in a system:cr.figuraticn management task that went out of control. Again, difficulties;re tied to products of the requirements phase and the need to constrainexpectations by doing requirements validation relative to the goals establishedr. -..".=• r^r.ceptuai Analysis phase.

: ::-:?.-.i TCNTROI SYSTEM VERIFICATION AND VALUATION ACTIVITIES AT CRNL

A ~=j:r cbjective cf V5V activities in the I5C Division at ORNL has been toprovide an environment that will stimulate transfer of ideas and experiences-..-.at will benefit ail parties. Other objectives are to• document current VSV practices within the nuclear industry,• revelep = broader perspective of the VSV needs and problems affecting the

• Evaluate automated tools and testing software for appropriateness andrifectiveness in accomplishing V&V tasks,• Identify those VSV methods and tools that are most appropriate for the•^ricus types cf software applications needed, and• Jrrccse r.ew metnoas and tools that would enhance the effectiveness of VSVactivities.•VGr.APH: Objectives of the ORNL V&V effort.

:".g--re 1 summarizes accomplishments and the range cf activities of work in-regress at 0?.NL. Ail tr.e initial literature review was done through the-.avar.oea Controls Program. For tr.e past eight months most of the effort has.:eer. directed toward developing VSV guidelines and ccmoiiino a collection of

.. r.etr.cas *,: 25 ;r,;: a rataic::, ihert-term projects :;: tr.e "2E "ffice ofv.v : rcsucticr. .".eactcrs ::i??,) r.ave beer, interspersed in response to requests•. r .vert*.; a re ViV i-slitea help, rcmmer.ts an,:; recommendations provided to the.'?? .-.sve already influenced £:rmuiaticr. cf software QA policies and guidelines:. :r.e :."??. program. Analysis cf difficulties faced by the :>?R program have in:rr. confirmee tne value :f choices r.ade in preparing the E??.I VSV guidelines.-.5 rr.a;cr :c;e«ivs is being r.et..'3-.---: Vse vugrapn prepared f:r the VlV workshop.

STANDARDS FDR HUCLZAR FACILITY SOFTWARE APPLICATIONS

"GR.-.=H: i'hew lists cf the various series cf standards. 'Jse material from V&V

>A:;3I. ANS ;eries»

•A^SI.' I'ZEi seri55»

v, i;" " rui«^ 1 i n e c •*>

MODUS IF THE SOFTWARE DEVELOPMENT LIFECYCLE PROCESS

::::;v5re development today for almost any application intended for wide use isa rr>ajcr undertaking isecause to do useful and powerful things conveniently andreliably by human standards requires execution of many complex computational:as.-:s. >2cst cf the challenge in developing and maintaining software is.earning new to manage complexity in ail its manifestations. Experience from-anaging software development at the project level has been formalized in terms:f a number cf process models, shown in Table 3. These models are compared in:^rms :f principle benefits and limitations. Their limits of applicability are

."se :f the Spiral Model proposed recently by Boehm [3oehm88] is increasing and

.3 considered generally to be superior to the ubiquitous Waterfall Model.:;enrr. attempted to address the greatest shortcomings of the Waterfall Model:

• ::o place fcr crctctypes fail system requirements must be known and.ncerstcod from the beginning),

• Very limited iteration allowed, i.e., only between successive phases, and• Too strong emphasis en documentation (document preparation schedules were

irivir.g tne development process).

:/ providing a staged, iterative approach, the Spiral Model allows a manager to:ind cut much sooner in the development process what the greatest difficultiesishow stoppers) are and to be able to terminate the development process (ifmat becomes necessary) before committing large sums of money and major company:esc'urces to a large scale effort. This approach is more effective in showing..-.at iirectiens can be taken tc assure success in higher risk development

.';er.~'s Spiral ^oaei cper.ei -he way t; use cf risk management ;£cer.m91] rather:r.ar. iccumer.t preparation ;r some other development activity to drive the.;cr--.vare oeveiopment process. This framework can be extended ir. a natural way-. o plan and to manage verification, testing, and validation aspects of software.-•.svslopment. Leveson and Harvey ;LevKar33] developed a framework for-rvaluatir.g hazards associated with safety-critical require.T.er.ts. Their work,-ar. ce adapted no evaluation of risks associated with mission-critical or any:ther such category cf requirements. Methods can thus be identified andrefined for analyzing many kinds of risks associated with ail aspects ofsoftware development. .-. formal basis can thus be laid out for determining what_evels of effort are appropriate (perhaps also needed) for verification,resting, and validation to achieve a desired level of reliability in thedelivered software.

":r.fi?uraticn control is the primary means for managing complexity at the;r;--=;t level, during envelopment, especially en large presets, many:::v:::es ire slways ?:ir.3 :n it the same time, Managers must

• Coordinate reiatea tasks,• Track the status :f many parallel activities,• Identify and define when a development phase is complete, and• Establish and preserve important document, design, software, or

development environment baselines at regular intervals during development.

Effective verification must include extensive integration into theconfiguration control system. The importance of this is illustrated in Fig. 2,which shows schematically what a mapping cf a set cf requirements onto theproducts cf following phases looks like. This diagram is useful forillustrating many important issues associated with producing reliable software.If verification activities fail to identify a serious deficiency in therequirements until the coding begins, the consequences can be quite costly. Intr.is figure a set of bold lines is used to trace back from a code module (box=2£i to the design items (boxes *12 and #13), and from the design items toiheir parent technical specifications, etc."73?..-.?H: Jse requirements tracking diagram frcm MacFlow report.

If a serious deficiency is found at the requirements level, there may be rippleeffects (look at boxes «11 or #14 at the technical specifications level) back-own tr.rough phases in which much work has already been done. There may beserious problems in some of the intermediate reference baselines so that newbaselines must be defined and access to the deficient baseline limited torarefuily controlled uses.

Verification methods may have to be chosen or adjusted according to the type ofapplication for which the software is being written (e.g., real'time process;er.troi vs. data legging or trend analyses). Although this will all bedocumented in a Software Verification and Validation Plan, a reliable means oftracking implementation of the plan and communicating status informationaccurately and clearly is needed. Results of verification activities for eachphase must be reported ar.d these reports accessible to those working on laterpr.ases *her. questions arise. If enough information is not available,"ambiguities arise and lead to confusion.

Figure 3 shews a diagram cf a typical development lifecyle and most of the

. *.i:ea items. Ail information ar.s activities implied by s'izr. % diagram must- .-.rcrprrateo. i.-.t: -.-,5 :;-:'ii-ursti:r. management' system. All configuration

•••ar.a-ement information =cout tr.e seveicpmer.t cf 3 software trosuot is;: oter.tis-ly very va.uasle for tr.csa wno must maintain it. This information isiir: :f tr.e software documentation, most likely maintained in ar. on-line data:;ase in a computer system. It has r.ot been considered part of the software-.e.iveracle package, cut snouis be, especially for safety-critical software.It is r.eeced to provide continuity between the mitiai development phases andthe maintenance pnase. Improvement in reliability in both operation andmaintenance is a possible consequence, as well.

Figure 3 shows the top chart of a hierarchy of charts constructed using the"acFlow™ ;5erTerS9] software on a Macintosh™. It appears to be adequate to..anale configuration management for projects cf modest size. XacFlcw is:lowcnarting software intended originally for use in documenting code. It hasTaxabilities :hat ro far beyond that to support many other aspects cf software-.evelopnent • it is highly adaptable. :-!acFlcw is a good example cf inexpensive:cftware tr.at can be used as a multimedia information management tool::ur.rc*0! . A ?C version cf this software is scheduled to be available later

"•;?.-.•••: Vse software development lifecyo-e uiagram from MacFl:;; report.

A classification scheme is needed in order to decide what VSV methods areappropriate and how much effort is required in applying them. Three obviouscategories are safety-critical, mission-critical, and general operationssupport. Current regulatory requirements and engineering practice specify thatail safety-critical functions must be handled by a protection system that isir.aependent cf the plant control system. There may be safety-related functionsm the plant control system, but not safety-critical. Plant control systemsoftware is mission-critical, reflecting needs such as maximum plantavailability and ease and reliability of software maintenance. Software usedto prepare reports and to support everyday business operations falls into the~ **. 1 r c class

Furt.-.er subdivisions may be desirable, such as real-time process control andeverything else, particularly for safety-critical applications and likely alsofor mission-critical applications. Whether to subdivide depends en howdevelopment is managed and what methods are used for things like

• Formal Specifications• Programming languages• Verification• Testing and Validation

The specifics cf each development project (goals, regulatory environment,licensing constraints) must be analyzed to determine the probable costs andcor.eduiir.g impacts cf introducing further distinctions.

A further classification by application type is being used by DOE for the NPRprogram. The types and their descriptions are listed below.

Scientific/Engineering Codes: Software used to define, analyze, evaluate:r verify functional and physical characteristics cf a proposed facility.

Engineering/Design: Software classified as computer aided design (CAD),computer aided engineering (CAE), or computer aided manufacturing (CAM), or:tr.er sucn software that is used to design and /or engineer a plant.

~ rroeccior. s:";2 control system software.

latasase management applications systems: .Software '-sea as a controlled. :ur:s -f ir.f rrmsticr.; software affecting a technical baseline cr configuration

rcrr.putenoed models used :o design, test, operate, cr maintain any.;irs,::r.ai software within a plant.

Taw Software

_.issif ication ccr.err.es s.re easiest to acsiv *: r.ew software, Classification,,--_i-ly sr.cuid ::r;r during tr.e requirements Phase cf software development as-.-- i ::«»:::.: ".."= i^ftware ;.-. 51ar. ar.a iiftware Verification ar.a Validation

l::i.sring Softwar*

I::iitir.a software .-.seas " ie rlissifisci initially the same way as new,iitware. It r.eeas •:: e'further :lassified according to whether it will or..•ill r.ot ce modified. Software verification and validation in some form must:s required when any existing software is modified. If existing software is toce modified, it rust be further classified according to use, whether it is part:£ a real-time control system cr r.ot. If it is, it must be treated as newsoftware when a modification is made. Modifications to real time control -software can very easily introduce unintended and unnoticed side effects. Sorertair.iy for safety-critical and probably for mission-critical software, more=:-:tensive verification, testing, and validation is needed for other than realtime control types cf applications. For these ether types, testing and••slication cf the modules where modifications are made is probably sufficient."ireful •sxamir.aticn cf use and evaluation cf risks should provide adequate.-uiaance for wr.at and hew much VSV is appropriate.

Tare must ie taken with existing software to assure it is nor. being used in a.:or.text :r under conditions for which it was not originally designed. Any-rxisting software, whether modified cr r.ot, used for purposes different from-:.;; ..as cnoir.ally inter.sed, r.eeas to re treated as new software (e.g., a..••ell-developed prototype* for purposes cf qualification.

verification and validation methods needed for existing software may bedifferent frcm what is needed for r.ew software. There is not much experience.•et upen whicn to draw -z propose guidelines other than what are recommendedior r.ew software.

VERIFICATI:::, TESTING, AND VALIDATION GUIDELINES

«'3enerai requirements for each Qualification Category*"ecer.t experiences by the Canadians and French show very forcefully that•rrificaticn, testing, and validation methods for all classes cf controlsoftware r.eed ro be given more attention. More support for research and-.-rveiepment in software engineering is needed, but especially in the areas of

."~i:.;i;r,. TT£ will prccsciv r.sve ~; ;s ".".s ^ame ™r. its l'<s?, ciccram.

-rv^rsl s.reas with cirsct imoacts cr. YSV "ire sicr.ificant sr.s ieserve more

• L.braries cf tasted and qualified software modules have been proposed as.••.sar.i: :: enhance the quality cf new control systems, while at the same time-icueing develop costs ar.d times, -eveicpment and use of such libraries has..ser. cart cf the long term expectations of the DOD Ada language suyjport-rcgiam. Interest is r.cw growing rapidly ir. developing collections of commonly_cea class methods fcr software developed using principles of Object Orientedr-iiirr.. Object Oriented rrogramming appears to offer strong support forcre cueing reliable, reusable software.

• r'r.e r.uciear industry (including DCS laboratories) needs to establish closerr-r.cacts with Computer Science departments in the universities where software•••srification and validation problems and needs are analysed and studied. At-.-.- federal level, some parts cf the military services have been doing this for-ir.v years. Cr.e result :f this is the Software Engineering Institute at",.;r.*2i«-K9llon "diversity iCKU) . VJestir.gnouse has links also -.?. Z'AU.

' • urcue University ana .j-rorgia Institute ;f Technology have a cooperative^rc-rarr, ;rer] supported by the department '.f Defense "iDCD) to T2D «Suinmariz«*;:-,rc:se». Their ~£f:rt r.\s prcaucea ar. automated testing system [ref] (MOTHRA):as«i ;n Mutation Analysis.

• Research in the use cf Formal Specifications [ref] appears still to belargely confined to the academic community. Those working in this area arevery interested in the problems faced by the nuclear industry and welcome the:ppcrtunity to work-in collaborative efforts both to solve real world problems=r.d to find ways to transfer their knowledge and experience to the nuclearengineering community. In principle, software maintenance and reliabilitywould benefit immensely if all deficiencies could be fixed and all enhancementsT.ade by modifying only the formal technical specifications, thereby eliminating;-rver=l time consuming development phases entirely.

• r.apia developments _r. computing technology during the pasc decade have-rcught powerful capabilities to every engineer's desktop. Convenient access-; increasingly powerful and flexible applications stimulates development of.'.=•.> ideas and concepts that quickly exceed cur resources to produce forisfety-criticai applications. The same will probably happen"fcr.••.ission-critical applications, too. New forms of obsolescence will appear.Ideas and tools are needed to help manage expectations that are expressed in-cr.tinuaily changing requirements (part of theCSGELEC experience on P20) aswell as to avoid the worst effects of obsolescence.

^Program Inspection. Try to illustrate the method as much as possible using:.:.:= crams and tables. Review materials prepared by Fagen for ideas.»Vntil benefits from the areas described above appear, some vital first steps:r.ust be taken. The focus now is strictly on plant control system software.T3D• ::eed to shift emphasis from code and test to developing and understandingrequirements and to more formal statements cf technical specifications.• ::eea a record cf experience - deficiencies log

.r^ir.icaticr. ;r Ir.specti:r. Taam»- I".p_5rT'.sr.~i;ion ~f rr.scecticr. rrccess»

lieficiencies Log

£=crir.g up and maintaining a deficiencies log is a very important part of andirective V5V program. Its uses include

• Identifyir.g problem prone software modules,• Evaluating effectiver.ess cf verification, testing, and validation program,• Supplying basic data needed to i.T.prcve V5V methods,• locating system design problems (poor designs, places needing improved

.-.esisr.s),• Providing a base of experience for evaluating the level of VSV needed for

~2ir.ter.ance and system modifications to maintain the desired level of system.e-iabiiity, sT.d

• retermir.ir.c when major upgrades or replacements are needed.

'. ricier.cies log must include a description of the deficiency and where itii first identified. ":r =sch deficiency information should be provided

.aer.tif yir.g its source, b:th in project documents and with respect to;5ve_opmen^ phase in which ic was introduced, and what must be" done to correct... .-. clsssificatic-n scheme should be set up to assist with determining the•iricusnass of each deficiency and indicating a level of urgency for gettingattention. A record of how each deficiency is resolved must be kept and shouldinclude the name of those responsible for a successful resolution."

On-Line Refer«nc* Documentation

.".pies of =11 reference documents need to be easily accessible on-line. This_s an important way to support an effective VSV program. It can be integrated.•ery naturally in"to the configuration management system. Copies of the basic-^ference documents need to be part of each reference baseiine. The current. ccy :f -he reference set r.ay consist cf drafts cf the next version to be-•e.easea. .-. minimum set of documents should include

• Software £A Plan• Statement of Requirements (needed to design tests and for validation)• Tunctionai Description and Technical Specifications• Software Verification and Validation Plan• Software Test Plan• User's Manual• Source Code Listings

::ost current text editors include keyword search capabilities which can be usedt: find very quickiy those sections cf reference documents needed to do aparticular task. Automated searches can enhance the effectiveness with which atask is accomplished. This also can improve the effectiveness cf VSV efforts.

Validation of Software Requirements

Validation cf Requirements is essential for controlling expectations and.~ar.aging change requests. This validation activity is often omitted. Again,whether to include it or not should be evaluated in terms cf the risks

-;.'.'.-_v 2. ••"-. Resuirsrr.er. ts '.'all dsticn r.sview ?,cpcrt r.ee»s -.: ::cc'-:r.er.t "he.;;:!:: ;; tr.e review ;;r future reference, r.eassr.s f:r soticr.s -2..>;en need toi ir.o-ucec in t ~e -^ccrt Cw s i"2cc-d exists cf exciansticns z~ initial

-""ir.iii zrcals. >ot *n,.v will this sis ir. rr.ar.aoinci chancre :5cuss*s, but also-rrviie useful juidar.ce ir. maintaining the final produce.

Validation of Control Software

•--luati:.-. cf ;cr.trrl iys:s.T, aritware st -he conclusion ci the system-r.tsrraricr. phase 15 usually the cniy validation done. Methods and techniques-sea are usually depende.it en the particular application and r.ust ce definedf:r -iach appiicatirn.

New Software

Nearly all bocks ;ref list] ana references [res list] en the subject of•~rif-cati~n, testing, sr.d validation assurr.e the reader is concerned about new,;v-.are. Help ir. this ;rsa ~z ve-at-vsiv -issv ~,z find.

Prototype Software

.:r:t:type software receives little attention with regard to verification,testing, and validation. :.i rr.cst cases it is viewed as "throw-sway" softwaredeveloped to examine feasibility of new concepts or to understand systemperformance issues associated with choices of particular architectures fordifferent parts cf a complex application. Often such software is not thrownaway, but eventually incorporated into a functioning demonstration facilitythat is operated to do useful things. Such software ir.ay':even have been takenthrough a limited V5V process, resulting in a more reliable product than someexisting software, not developed to be a prototype, used iby the nuclearindustry. This class cf software has not received much attention to date, but_5 identified here because it may become a class that needs to be considered.

Existing Software

;---=lp with verifying, testing, and validating existing software is difficult tofir.a :r often dees not exist. 2ne way to fit it into existing schemes is totreat it as some form of prototype.

SUMMARY

r.-.e following guidelines have teen proposed:

• Vse risk management as the basis for deciding what and how much V&V isr.sesed as well as for ether management decisions for software development.Effective use cf this approach will require spending more time on "up front"planning activities than most managers are used to doing in order to assembleenough information uo be able to assign values cf some sort to the risks_nvolv»d. Assignment cf risk to alternate approaches tc V&V and many othersoftware development activities may be very difficult initially due to lack of3 basis of experience.

• Classify ail new and existing software using a scheme consistent with the

• .lair.cair. s casie reference set "f prefect uecumer.ts jr.-lir.e m a lormlly accessible ::r printing selected sections ana for uoir.g automatedv:rs searches. This sr.culd not -r.iy help productivity, but car. potentiallyatly -rr.r.sr.ce qualities important -.:. Vi'/ such ^s consistency, completeness,

• Vse ircsrs" Inspection is the basic verification method. This method isesed as a startir.3 ccint if r.o verification activities are currently

=tr.;cs rr.ay ce rcuncL whicr. are ~ore appropriate or effective. "Jse themr.steac. If other verification methods are already integrated into each phase; i software ieveicp.T.ent lifecycle and work well, there ~ay ce no need toi.ir. e •:; use cf the rrcgra.T. Inspection rr.ethca.

• Tet up ana -air.tsir. -=. deficiencies log nor each software application. A-. 1-c 1 sr.r.ed deficiencies locfcir.2 svstem is !-:ev to estabiishin- s docurnentcd

; -y.cen-ir.rs :::r ?.__ ispectc :: ?. verification. -.'Sstir.a, \T.Z. validation

. ^diti:n t: tr.e juidelir.es sieve, the followma additional practices have--:•:. ;trcng_y recc.T.T.er.ded:

• "Jse a corr.puter-casea configuration management system to track all aspects: development and maintenance. This enhances consistency and completeness,remotes clarity by providing a current record of who did what and when, androvides a level of confidence in the project information as long asopropriate access is maintained. All these contribute to greater reliabilityf" the final product.

• refine and establish reference baselines of the software at regular•.tervals during development. This helps to reduce confusion,^ur.oerstar.air.g, =nd misinterpretation.

• Include :ccies of all current versions of on-line documents andT-.-e.opr.er.t tccis with each archived baseline of the software. This alsoremotes clarity and consistency, important goals of any VSV effort.

• "Jse Cbject-Iriented Design and Programming techniques to achieve greater:;tware reliability and reuse. Currently, Object-Oriented anything is very:pular, so this subject area needs to be approached with due caution.rvertheiess this point of view emoodies many concepts that address many of the-quirements for reliable control systems very well.

• Provide a copy cf m e software development tools used in the package ofr.iverables to provide continuity, consistency, ana greater reliability in therar.sition from development to maintenance.

• Initiate an effort to use formal methods in preparing technical:ecifications. This will likely require building links to the academicimmunity to get assistance. This initiative especially should be a very goodsy both to promote desirable technology transfer and to prevent gaps fromsvelopir.g between engineering practice and new knowledge vital to warding efficnnoioaicai cbsoiescer.ee.

ADVANCED REACTOR SYSTEMS INCORPORATING DISTRIBUTED MICROPROCESSORS FOR CONTROL

REACTOR SYSTEM

SYSTEM 80+™(NUPLEX80+)

ABWR

SBWR

MHTGR

AP600

L _

REACTOR TYPE

PWR

BWR

BWR

GCR

PWR

VENDOR

ABB Combustion Engineering

General Electric

General Electric

GA Technologies

Westinghouse Electric Corporation

WHO NEEDS RELIABLE CONTROL SYSTEM SOFTWARE AND WHY

GROUPS OF PEOPLE REASONS

Regulators

Investors

Plant Managers

Public

Responsibility to protect the public

Establish policies for sound management of resources

Obtain a reasonable return on investments

Achieve high plant availability

Have easy and reliable maintenance

Assurance of safe plant operation

Low electric rates

TABLE 3.1 COMPARISON OF MODELS FOR SOFTWARE LIFECYCLE DEVELOPMENT PROCESS

MODEL

Code-and-Fix

EvolutionaryDevelopment

Spiral

Stage-Wise

Transform

Waterfall

DESCRIPTION

Write code and modify ituntil it works

Software application isenhanced in incrementsas requirements evolve

Repeat sequence ofdevelopment phasesto refine results of eachphase as required

Forerunner to the waterfallmodel: specified use ofsuccessive stages forsoftware development

Use a high level specifi-cation language whichis later translated tolow level code

Single pass through asequence of phases

PRINCIPAL BENEFIT

Quick results

Gives users an initialoperational capabilityquickly

Buiid on experienceControl risks

Provides a structureddevelopment frame-work necessary formanaging large scaleprojects

Enforces a consistentrepresentation ofrequirements

Maintains consistentformat

Provides a structureddevelopment frame-work necessary formanaging large scaleprojects

PROBLEMS

Poorly structured codeCode hard to maintainUnreliable software

Poorly structured codeCode hard to maintainUnreliable software

Careful managementplanning and coordin-ation required

No provision for makingrefinements to require-ments or design duringdevelopment

Does not accommodatereuse of software well

Limited provision foriterative refinementsacross multiple phases

Development processbecomes documentdriven

WHEN USED

Small projects

Projects with poorlyknown requirements

Prototyping, redesign

Large projects

Can accommodateevolving requirements

Software applications forwhich requirements arewell understood andcomplete

Formal proofs of correct-ness are required

Software applications forwhich requirements arewell understood andcomplete

Irii- Ot'vPioTiib rt.iw

ORNL/EPRI Plant Control System SoftwareVerification and Validation Project

LITERATURE REVIEW

J. K. Munro, Jr., "Verification and Validation of Digital Control SystemSoftware for Nuclear Power Plants: A Literature Review of CurrentPractice, Current R&D, and Future Alternatives," (DRAFT 3).

PREPARATION OF V&V GUIDELINES

J. K. Munro, Jr., "Guidelines for Verification and Validation of NuclearPower Plant Control System Software," (DRAFT 3).

APPLICATION OF GUIDELINES TO DEVELOPMENT OF A CONTROLSYSTEM

Workshop (to be held later in 1991) to identify projects that will useand evaluate the V&V guidelines.

CATALOG OF V&V METHODS

J. K. Munro, Jr., "A Catalog of Methods for Verification and Validationof Digital Control System Software for Muciear Power Plants," (DRAFT1).

PROVIDE V&V CONSULTATION AND DOCUMENT REVIEW SUPPORTTO NPR PROGRAM

J. K. Munro, Jr., "On the Use of MacFlow™ to Manage Development ofthe HWR Training Simulator Software", ORNL/NPR-90/49, December1990.

Review software QA plans, software development plans, software V&Vplans for HWR

i

S ^

ill

"5? J

XI V V»

HUIK SIMUI««R toriwuM ICMUIFMCNT PUN

Ultcycit Phut* tchtlultni

VALIDATION

<MCtPr»MCf \TISTAHO \

OCUVUTO /«urtM/