Embed Size (px)
Citation preview
FromLegacy to Component:SoftwareRenovationin ThreeSteps
Arie vanDeursenCWI
P.O.Box94079,1090GBAmsterdam,TheNetherlands
http://www.cwi.nl/˜arie/
BenElsingaProgramDirectorCBDandSecurity, CAPGemini
P.O.Box2575,3500GNUtrecht
PaulKlintCWIandUniversity of Amsterdam
P.O.Box94079,1090GBAmsterdam,TheNetherlands
RonTolidoCorporateTechnology Officer, CAPGemini
P.O.Box2575,3500GNUtrecht
Abstract
The major challengefor futurebusinessoperationsis to align changingbusinessgoalsandchangingtechnologies,while preservingtheassetsthatarehiddenin thelegacy systemssupportingtoday’s businessoperations.In thispaperwe formulatethemainquestionssystemrenovation hasto solve, andwe give a comprehensiveoverview of techniquesand approaches.We discussthe analysisand transfor-mationof legacy systemsandalsodescribehow domainengineeringcanhelp toidentify reusabledomainknowledge. By stressingthe needfor cooperationbe-tweenrenovatedsoftware and new software we naturally arrive at the needforcomponent-basedapproachesandcoordinationarchitecturesthatdefinethecoop-erationbetweenold andnew components.We presenta threestepapproachtosystemrenovation: find components,global restructuring,andrenovatepercom-ponent. The necessarytechniquesfor analysis,transformation,andregenerationof legacy systemsarealsodiscussed.The paperconcludeswith a descriptionofdomainengineeringanddomain-specificlanguagesasviable techniquesfor thestructuringandreuseof the applicationdomainknowledgethat is embeddedinlegacy systems.
1
Acknowledgements. This researchwasfundedby CAP Gemini. DaanRijsenbrij(CorporateScientificOfficer, CAP Gemini) initiated the projectthathasresultedin thispaper.
c�
CAPGemini, 2000
2
Contents
1 Intr oduction 31.1 Thesizeof thelegacy problem . . . . . . . . . . . . . . . . . . . . . 51.2 Whatis softwarerenovation? . . . . . . . . . . . . . . . . . . . . . . 5
2 Renovation Target: Component-basedSoftware 62.1 Components. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.2 ComponentIntegration . . . . . . . . . . . . . . . . . . . . . . . . . 82.3 Domain-SpecificComponent-BasedSoftware . . . . . . . . . . . . . 9
3 Overall approachto SoftwareRenovation 113.1 Technicalapproaches. . . . . . . . . . . . . . . . . . . . . . . . . . 113.2 Methodologicalapproach. . . . . . . . . . . . . . . . . . . . . . . . 12
4 AnalysisTechniquesfor LegacySources 15
5 Transformation Techniquesfor LegacySources 165.1 GeneralTechniques. . . . . . . . . . . . . . . . . . . . . . . . . . . 165.2 Supportfor StepB: GlobalRestructuring . . . . . . . . . . . . . . . 185.3 Supportfor StepC: RenovateperComponent . . . . . . . . . . . . . 18
6 GenerationTechniquesfor LegacySources 196.1 DomainEngineering . . . . . . . . . . . . . . . . . . . . . . . . . . 196.2 Domain-specificLanguages. . . . . . . . . . . . . . . . . . . . . . . 206.3 Risla: aDSL in theFinancialDomain . . . . . . . . . . . . . . . . . 21
7 Conclusions 21
1 Intr oduction
The key challengefor future businessoperationsis change: Businessrequirementschangewhenever thereis a chanceof higherprofitability. At thesametime, informa-tionandcommunicationtechnologiesthatcanbeusedtosupportbusinessprocessesareinnovatedcontinuously. Thesetwo formsof changecannotbeseenin isolation:manynew waysof doingbusinesscanonly berealizedby theuseof new technologies,whileinnovationin softwaretechnologycanleadto theemergenceof completelynew mar-kets.Thetypical exampleis e-commerce: initiatedby thetechnologicaldevelopmentsof theworld-wideweb,it hasnow resultedin waysof doingbusinessunthinkablefiveyearsago.
The flexible organizationwill have to align changesin businessand technology.Therapidly growing integrationof supplychainsrequiresintegrationof systemsbothinsideandbetweenorganizations.Softwaresystemsfrom thepast(“legacy” systems)have never beenbuilt with that objective. The emergenceof third-partysoftwareforEDI ClearingHouses,E-procurement(buying/sellingvia the Internet),auctions,andshopbotsrequiresthatlegacy systemsareusedfor completelynew purposesusingnew,
3
Year New projects Enhancements Repairs Total % New % Maint.
1950 90 3 7 100 90 101960 8500 500 1000 10000 85 151970 65000 15000 20000 100000 65 351980 1200000 600000 200000 2000000 60 401990 3000000 3000000 1000000 7000000 43 572000 4000000 4500000 1500000 10000000 40 602010 5000000 7000000 2000000 14000000 36 642020 7000000 11000000 3000000 21000000 33 67
Table1: Forecastsfor numbersof programmers(worldwide)anddistribution of theiractivities
different,distribution channelslike interactive television, WAP, AutoPCandothers.Thereis a majorgapbetweenInternet-basedtechnologieslike XML andJava thatarebeingusedin new applicationsandthetechnologiesusedin legacy systems.
Thereis a fundamentalbottleneckto alignment: legacy systemstendto be veryhardto adapt. The lack of knowledgeof the old systemsandthe needto train newusersof thesesystems(whenusedin renovatedform) make the potentialadaptationandprolongatedusageof legacy systemsdifficult. Again,e-commerce is a convincingexample: web-enabling,for example,retail bankingservicesrequiresbreakingup abank’s backoffice system.Suchsystemsaregenerallyold, written in Cobol,anddif-ficult to adapt.Consequently, they cannotkeepup with thechangesrequiredfrom thebusiness,potentiallyleadingto lossof marketsharefor theorganizationinvolved.
Softwarerenovationpreparesa legacy systemfor futurechange.It is basedon theview that an organization’s softwaresystemsprovide valuablefunctionality, that hasbeenprovenin practice.As such,it shouldbereusedwheneverpossible.At thesametime,thepackagingof thisbusinessfunctionalityis usuallyfarfrom optimalasthey areoftenbasedon old languages,databasesystems,andtransactionmonitors,monolithicin design,andunmaintainableasaresultof repeatedundocumentedmodifications.Asaconsequence,legacy systemsareveryhardto change(if not immutable)andprohibitthealignmentweaim at.
In this paper, we take a closerlook at softwarerenovation. In Section2 we arguethat the goal of renovation is to arrive at component-basedsoftwaresystems,whichpermit the smoothintegrationof newly built softwareandrenovatedlegacy compo-nents.In Section3 wethenexplainhow legacy systemscanbecomponentizedin threesteps:Find Components,Global Restructuring,andRenovateper Component.Thetechniquesusedto achievethisareanalysis,transformation,andregeneration,coveredin Sections4, 5 and6. In the currentsection,we provide figureson the sizeof thelegacy problem,andsummarizethehistoricrootsof theareaof softwarerenovation.
4
1.1 The sizeof the legacyproblem
It is temptingto assumethat we might be able to dealwith legacy systemsby justthrowing themaway. However, the sheervolumeof legacy softwarerunningworld-wideprohibitssuchanapproach.If wetakealook atsomeindicatorsprovidedby [15],weseethatthetotal volumeof all softwareworld-wideis 7 � 109 functionpoints.Themajorityof softwareis writtenin old, inflexible languages.Forexample,30%is writtenin COBOL, which correspondsto 2 � 2 � 1011 statements.For mainframeapplications,80%of thesoftwareis written in COBOL.
Moreover, whenanindustryapproaches50yearsof age—asis thecasewith com-puterscience—it takesmoreworkersto performmaintenancethanto build new prod-ucts. Basedon currentdata [16, page319], Table 1 shows extrapolationsfor thenumberof programmersworking on new projects,enhancementsandrepairs. In thecurrentdecade,four outof sevenprogrammersareworkingonenhancementandrepairprojects.Theforecastspredictthatby 2020only onethird of all programmerswill beworkingonprojectsinvolving theconstructionof new software.In aworld thatevolvesat Internettime, it is dangerousto make any extrapolations.It is unclearhow devel-opmentslike, for instance,applicationserviceprovidersandoutsourcingwill affecttheratio betweennew developmentandmaintenance.Nonetheless,we feel that thesefiguresareat leastusefulto understandthehistoricevolutionuntil thecurrentdate.
Thesefiguresshow thatmaintenanceandrenovationof softwarethatexiststoday,is anactivity of majoreconomicimportance.Sincethe total amountof softwarewillonly grow, theimportanceof maintenanceandrenovationwill grow accordingly.
1.2 What is software renovation?
Softwarerenovationis pro-activesoftwaremaintenance.It aimsat improving theover-all qualityof softwaresystems,makingthefunctionalityof thesesystemseasierto usein andadaptto new applicationareas.An overview of thisareacanbefoundin [4, 7].
Softwarerenovationhasits rootsin a numberof relatedareas.Sinceit is not at alluncommonthat the only reliableinformationaboutwhat a legacy systemdoesis thesourcecodeitself, reverseengineeringis an essentialaspectof softwarerenovation.Originally, the notion of reverse engineeringcomesfrom hardware technologyanddenotesthe processof obtainingthe specificationof a complex hardwaresystem.Insoftwarereverseengineering,we try to distill asmuchusefulinformationaspossiblefrom thesystem’s legacy sources.Thekey challengeof reverseengineeringis to arriveatnon-trivial, higherlevelsof abstractionthanjust thesourcecode.
Softwarerenovationalsoincludesmodifying the softwarein orderto improve it.Suchtransformationsmayremainat thesamelevel of abstraction(for example,gotoelimination),in which casewe speakof systemrestructuringor refactoring[13]. Thealternative is to performtherenovationvia a reverseengineeringstep,which is calledre-engineering: first aspecificationonahigherlevel of abstractionis constructed,thenatransformationis appliedonthedesignlevel, followedby a forwardengineeringstepbasedontheimproveddesign.
Furthermore,softwarerenovationhasits rootsin compilerandprogramminglan-guagetechnology[1]. Building acompilerfor a languageandtranslatingthatlanguage
5
to assembly, involvessignificantanalysisandtransformationof programs— technol-ogy thatcanbereusedwhenanalyzingandtransformingsoftwarefor renovationpur-poses[3].
Last but not least,the year2000problemandthe Euro conversionhave stronglyaffectedtheareaof softwarerenovation.They haveresultedin anincreasedawarenessof the needfor tools for reverseandre-engineering,adoptionof sourceanalysisandmodificationtools, andthey have increasedthe maturity of the renovation tools andtechniquesavailable.
2 Renovation Target: Component-basedSoftware
The aim of softwarerenovation is to preparea legacy systemfor future change.Anessentialtechniqueto arrive at the requiredlevel of flexibility is to decomposethesysteminto a setof components. In this section,we take a closerlook at componenttechnology, andits role in softwarerenovation.
2.1 Components
Following [20] componentsarebinary units of independentproduction,acquisition,anddeployment,that interact to form a functioningsystem. Thereareseveralreasonswhy softwaresystemsassembledfrom componentsarebettercapableof dealingwithchange,beit in businessor in technology:
� Componentsareindependent,soimplementationchangesto onecomponentcanbemadewithoutaffectingtheothers.
� Componentsaddabstraction, hiding implementationdetailsbehindexplicit in-terfaces,makingit safeto changethis implementationwhenneeded.
� Componentscan be easily replacedby othercomponentsoffering at leastthesamesetof services.
� Componentssupportreuse,sothatnew systemscanbequickly built by assem-bling themfrom existing pieces,leadingto short lead time to market for newsystem(extensions).
� Componentsystemscanbeamixtureof self-madeandboughtcomponents,mak-ing it possibleto benefitfrom new developmentsin commerciallyavailablecom-ponents.
� Componentssupportcustomization, makingit possibleto producea tailor-madecomponent.This canbe achieved by inheriting functionality from an existingclassof components,or by settingattributessteeringthe component’s behav-ior. As such,componentsprovidea middlepathbetweenunadapatablestandardpackagesand(expensive)bespokecustomersolutions.
6
Legacy code
Extracted components
New Businessrequirements
New components
Figure1: Integratingrenovatedandnew components.
Thetechnicalorganizationof collectionsof componentscanvary. Themostcom-monform is thepackagingof componentsin a classlibrary. An applicationprogram-mer can usethe classlibrary by writing a “main” programthat makes calls to thevariousclasses.Theapplicationprogrammerhasto understandthe(possiblylargeandcomplex) structureof the library, but is in completecontrolof theuseof theclasses.Contrastthiswith componentframeworkswhichprovidecannedapplicationsthatonlyhaveto becustomizedby writing someplug-insthatperformnon-defaultbehaviour.
Froma renovationpoint of view, an importantpropertyof componentsis that theinterfacesalsohidetheage of thecomponents,permittingcooperationof legacy com-ponentsandnewly createdsystemparts.Thisis neededin many situations,for examplewhenweb-enablinga backoffice system.Thebackoffice is a legacy systemthathasto berenovatedin orderto allow connectionswith theweb. Theweb-interfaceitself isnewly built.
In Figure1, we sketchthis situation.Froma legacy system(e.g.,thebackoffice)we extractcomponentsthatrepresenttheessentialfunctionalityof the legacy system.At thesametime,new businessrequirements(e.g.,web-enabling)leadto theconstruc-tion of new components.The ultimategoal is to find component-basedarchitecturesanddevelopmentmethodologiesthatenabletheseamlessintegrationof renovatedandnew components.This alsoimpliesthatthereshouldbea strongrelationshipbetweentechniquesfor renovationandtechniquesfor construction.
It is importantto identify whencombiningold andnew componentscanbeappliedwith success:
� Thelegacy componentsthatarereusedshouldbestable: only limited modifica-tionsareexpectedin thefuture.
� The datamodelsusedby the legacy componentsandnew componentscanbe
7
keptconsistent.
� The technicalknowledgeandtools neededfor maintainingthe legacy compo-nentswill remainavailablein theorganization.
Whenoneor moreof theseconditionsis not satisfied,the only alternative is toreimplementthelegacy componentusingnew technology.
2.2 ComponentIntegration
Componentsdo not operatein isolation, but interactto form a functioning system.This requiresagreementbetweencomponentson waysof communication,for whichseveralstandardsexist, suchasCOM+ from Microsoft, Corbafrom theOMG group,andEnterpriseJavaBeans(includingRMI) from Sun.
For thepurposeof this paper, weusea software busapproachto illustratethecon-ceptsof componentintegration,asthis clearlyemphasizesseveralaspectsof compo-nenttechnology, suchasindependence,binarydeployment,distribution,andheteroge-neousimplementations.In largeapplicationsmoreor lessstandardtechnologylike,forinstance,COM+ or Corba,will beusedasmiddlewarelayer. Thesetechnologieshave,however, a steeplearningcurveandareconceptuallyandtechnicallyquiteintricate.
For expositorypurposeswepreferthereforetheuseof amuchsimplersoftwarebuscalledtheToolBus,whichhasbeendevelopedatCWI andtheUniversityof Amsterdam[2]. It is aprogrammablebuswith thefollowing characteristics:
� Components(”tools” in ToolBusparlance)areconnectedto theToolBus.
� Thecooperationbetweencomponentsis describedby a process-orientedscriptin theToolBus.
� UnlikeCOM+, Corbaandothers,theprotocolof cooperationis madeexplicit.
� Tools canbe implementedin differentlanguagesandcanrun on differentma-chines.
� Thereis nodirectcommunicationbetweentools.
TheToolBusarchitectureis sketchedin Figure2. At thebottomweseetools(blueboxes)thathave a bidirectionalconnectionwith theToolBus. Toolscapturethecom-putationlayer, whicharetheprogramsthatcarryoutspecializedtaskssuchassortingafile, displayingawindow, or computingtheestimatedvalueof futuresthroughabank’sbackofficesystems.
On top we seetheToolBusitself (grey rectangle).InsidetheToolBusseveralpro-cessesareexecuting(redcircles).ToolBusprocessescancommunicatewith eachotheraswell aswith tools. The ToolBusprocessescapturethe coordination layer, whichdealswith theway in which programandsystemspartsinteract(by way of procedurecalls,remotemethodinvocations,andthelike). In orderto enabletheexchangeof data,acommondataformatis usedcomparableto XML.
It is our thesisthat a rigorousseparationof coordination andcomputationis thekey to successfulrenovation. Theinflexibility of traditionalcodeis largely dueto the
8
...
ToolBus
Tools
Coordination
Computation
Common dataexchangeformat
Figure2: TheToolBusCoordinationarchitecture.
factthatcoordinationandcomputationarecompletelyentangled.Thismakesit hardtounderstandandrestructuretraditionalcode.Thisalsofollows themantraof work-flowmanagementsystems, with “work” correspondingto the computationlayer, andflowmanagementto thecoordinationlayer.
How cantheconceptualseparationof coordinationandcomputationbemappedonstandardmiddlewarewhich is likely to beusedin largeprojectsbut doesnot providesucha clearseparation?Of course,the separationof coordinationandcomputationcanbeusedasguidanceduring technicaldesign. In addition,we offer the followingsuggestions:
� Introducea separateworkflow engineto implementcoordination.
� Usedesignandimplementationstandardsthat forbid complicatedcontrol pat-ternsinsidecomponents.
� Usea widely acceptedstandardlikeXML for dataexchange.
2.3 Domain-SpecificComponent-BasedSoftware
Following [17], the currentexcitementaboutcomponent-baseddevelopmentresultsfrom theconvergenceof four phenomenaoriginatingfrom quitedifferentbackgrounds:
� Onthescientificside,theprogressin theareaof systematicsoftwarereuse;
� Ontheindustrialside,thewidespreadsuccessof componentsfor GUIs,databases,andsoon,suchasMicrosoft’sVBX, OCX, andActiveX;
� On the political side, the pushby major playersfor standardssuchasCorba,COM, andEnterpriseJavaBeans;
� In thesoftwareworld at large,thegeneralizationof objecttechnology.
Oneof thekey lessonsof thefirst phenomenon,progressin theresearchin system-atic softwarereuse,is that substantialbenefitsof componenttechnologycanonly beachievedby concentratingon a particularapplicationdomain[18, 21]. As an exam-ple, Jacobsonet al. [14] sketchthe stepsthat a typical organizationmaygo through
9
whenadoptingcomponenttechnology:anorganizationstartswith informalcodereuseleadingto somereductionin development,andstepby stepgetscloserto thedomain-specificreuse-drivenorganization, which is capableof rapidcustomproductdevelop-ment.
Technologiessuchas CORBA, COM and JavaBeansprovide integration at the“plumbingandwiring” level. Agreementat thedomain-specific(semantic)level, how-ever, is neededto fully arriveat thebenefitsof componenttechnology, suchasshorterlead time to market, appropriatecustomizationfacilities, and smoothintegrationofboughtcomponents.This may be achievedby furtherstandardization.For example,aspartof theCORBA effort, domainspecificationsexist for electroniccommerce,fi-nance,manifacturing,andsoon. Also, dataexchangestandardsbasedon XML, suchasOpenFinancialExchange(OFX) andXMLIFE for life insurancescanfacilitatein-tegrationof softwarecomponentsat thedomainlevel.
However, arriving at suchdomaindefinitions,be it within oneorganizationor aspartof aninternationalstandardizationeffort, is apainfulactivity. It critically dependson
� thematurityof thedomain;
� thelevel of understandingof thedomain;
� theexistence(or absence)of standardizationefforts in thedomain;
� thecommercialplayersin themarket;and
� thebenefitsthatthevariousstakeholderswill obtain.
We candefinethenotionof a domainasa family or setof systemsincludingcom-monfunctionalityin a specifiedarea [18]. Sucha setof systemsis worth examinigifit is:
� mature,i.e.,a setof legacy systemsexists;
� reasonablystable,i.e.,at leastsomeof thelegacy systemsareworthexamining;
� andeconomicallyviable,i.e.,new systemsareanticipatedin thedomain.
In oursearchfor components,thelegacy artifactsactas:
� anempiricalbasisfor systematicscopingof domaindefinitions;
� a sourceof domainknowledge;
� potentialresourcesto usein reengineering.
Theultimategoalof softwarerenovationis to make this knowledge,embodiedinlegacy systems,explicit.
10
A: Find Components
B: Global Restructuring
C: Renovate per Component
Analysis TransformationRegeneration
Figure3: Relationbetweentechnicalandmethodologicalaspectsof renovation
3 Overall approachto SoftwareRenovation
The aim of software renovation is to understand,transformandregeneratea legacysystemin sucha way thatits alignmentwith new businessobjectivesandnew techno-logicaldevelopmentsis facilitated.We wantto achievethisby separatingcoordinationfrom computationandby supportingtheactualcomputationsat thedomainlevel. Wenow discussthe technicalandthemethodologicalaspectsof renovation. Therelationbetweenthetwo is sketchedin figure3: eachmethodologicalstepusescertaintechno-logicalapproaches.
3.1 Technicalapproaches
We distinguishthreetechnicalaproachesto softwarerenovation:
� Analysisof legacy sources:thegoal is to inspectthesourcesof the legacy sys-temandextract informationfrom themthat revealstheir structure,purposeandarchitecture(Section4). Analysisis non-intrusivesincethe legacy sourcesareonly inspectedandnotmodified.
� Transformationof legacy sources:thegoal is to systematicallyrestructureandimprovethesourcesof thelegacy system(Section5). Transformationis intrusivesincethelegacy sourcesaremodified.
� Generation: thegoalis to identify potentialreusableassetsin thelegacy systemby explicitly introducingandstructuringdomainknowledgein sucha way thatmajorpartsof the legacy systemcanbe regenerated(Section6). Generationisalsointrusivesince(partsof) thelegacy sourcesarereplacedby generatedcode.
Analysisandtransformationof legacy systemsarecloselyrelatedasis shown inFigure 4: analysisgathersinformation that can be used,amongstothers,for trans-formation. Generationbasedon domainengineeringis a higher level approachthatusesinformationfrom theanalysisphaseandcanexploit transformationtechniquestoachieve its goals.
11
Legacy code Computer support&
human insight
Tranformation
Renovated system
Analysis
Hypertext documentation, Object models,Types, Visualization, Components
TransformationRules
Figure4: Analysisandtransformationof legacy systems
3.2 Methodologicalapproach
Giventhevarioustechnicalapproachesto renovation(analysis,transformation,gener-ation),how canwe managethesoftwarerenovationprocess?We describea threestepapproach:
StepA: Find Components Subdivisionof thelegacy sourcesin components:givenalegacy systemasblackbox,discovertheinternalstructureof thisbox.Thisis illustratedin Figure5(a).Thisstepis non-intrusiveandthefollowing issuesplaya rolehere:
� Are all sourcesof thelegacy available?
� For missingsources,usetoolsto recover thesourcefrom binaries.
� Determinethe correspondencebetweensourcecodeversionsand versionsofbinaries.
� Determinethelanguagesusedin thelegacy system.
� Obtainanalysistoolsfor theselanguages,for instance,by instantiatingagenerictool setfor them.
� Runtheanalysistoolson thelegacy systemandinterprettheresults.
� Determinewhethera subdivision in componentsis feasibleandwherethecom-ponentboundariesshouldbe.
12
(a)Subdivisionof legacy sourcesin com-ponents
...
ToolBus
(b) Global restructuringusinga coordinationar-chitecture.
...
ToolBus
...
ToolBus
Renovated component
(c) RenovationperComponent
Figure5: Renovationin ThreeSteps
� If a subdivision in componentsis feasiblecontinuewith stepB. Otherwisetryotherapproaches.Possibilities:graduallyimproving the legacy systemby pro-gramtransformations,or replacingit by anew system.
StepB: Global Restructuring Interconnectionof thecomponentsfound in stepAusinga coordinationarchitecture:replacethe internalstructureof the legacy systemby a communicationstructurebasedon coordination. Legacy components(possiblywrappedto interactwith the coordinationarchitecture),appearascomponents.Thisstepis only modestlyintrusivesincetheonly modificationsmadeto thelegacy are:(a)subdivisionin components;(b) wrappingof thecomponents.Major partsof thelegacyremainuntouched.Theissuesare:
� Write a wrapperfor eachcomponent.
� Write coordinationscriptsto connectthecomponents.
� Testtherestructuredsystem.
13
Object Models Visualization
Legacy Sources ExistingDocumentation
Extractor
RepositoryAbstractCombine
Filter
HypertextDocumentation
Figure6: Architectureof a RedocumentationTool Suite.
Step C: Renovate per Component Renovation of individual components. Nowis the time to considerthe renovation or replacementof individual componentsassketchedin Figure5(c). Theissuesare:
� Determinefor eachcomponentwhich renovationstrategy to use. The optionsare:
– Leaveasis.
– Completelyreplaceit by commercialoff-the-shelvesoftware.
– Performa very detailedanalysisof the componentin orderto extract itsbusinesslogic thatcanbeusedto rebuild or regeneratethecomponent.
– Apply a layeredapproach:stepsA, B andC areappliedat thecomponentlevel.
� Apply theselectedrenovationstrategy.
� Testthecomponent.
14
4 AnalysisTechniquesfor LegacySources
The goal of analysisis to extract informationfrom legacy systemsthat revealstheirstructure,purpose,andarchitecture.Analysistechniquesarecrucial for stepA, FindComponents,andin this sectionwe cover thoseanalysistechniquesthat canhelp tofind candidatecomponentsin a legacy system.
Thesearchfor componentsin legacy systemsis aninteractive process.Toolscol-lectall sortsof informationaboutthesystem,which is usedby therenovationengineerto find candidatecomponents.Thereis no singleway to find all goodcomponents:therefore,the tools shouldprovide many differentviews on one legacy system,andshow potentialcombinationsof theseviews. This requiresa hypertext-basedbrows-ing mechanism,potentiallysupportedby an intelligent agent,as well as an on lineannotationmechanism.An architecturefor sucha programunderstandingtool suiteisdisplayedin Figure6 [9].
The interactive searchfor componentscanhave any oneof a numberof differentstartingpoints.
The first is to interview, if possible,the users, maintainers, anddesigners of thelegacy systemin orderto geta pictureof theoverall functionalityof thesystem,and,morespecifically, to get an impressionwhich functionality shouldbe preserved andwhich is redundant.
Anotherstartingpoint is to usethepersistentdatastores. Of all thedifferentsortsof dataplayingarolein thelegacy system,it is likely thatthedatastoredin thedatabaserepresentsbusiness(domain-specific)entities.Following suchdatadown to theactualcomputationsleadsto thoseprogramsor proceduresthatcouldbecandidate(domain-specific)components.
Anotherstartingpoint consistsof the programcall relationships,which may tellus somethingaboutcohesionandcouplingof modules,or aboutthe layersbuilt intothe legacy system.If oneprocedureinvokesmany others(high fan-out), anddoesn’tget invoked itself, it is likely to be a control (coordination)module,with little built-in functionality. Likewise, if a procedureis calledby many others(high fan-in), it islikely to be somesort of utility routine,dealingwith error handlingor logging. Theprocedureswith bothlow fan-inandlow fan-outaretheonesthatarelikely to containbusinesslogic [8].
Yet anotherstartingpoint canbe thescreensusedin thesystem[19]. Thescreensequencecanbe identified,togetherwith the key strokesleadingto eachsubsequentsreen.Suchscreensequencesarevery closeto usecases, telling whatactionsanenduserperforms.Moreover, following theflow of screeninputfieldsthroughtheprogramidentifiesthoseprogramslicesthatimplementthegivenusecase.Thisform of analysiscanverywell besupportedby automated,interactive,tools.
Techniquesfor combininglegacy elementsinto novel waysin orderto arriveatco-herentcomponentsincludeconceptanalysisandclusteranalysis[10]. Suchtechniquescanbeusedto spotcombinedusageof piecesof dataandfunctionality. For example,they canbe usedto groupdataelementsinto candidateclasses,basedon their usagein programsor procedures— which can thenbe madeinto methodsof the derivedclass. In particularconceptanalysiscanbe usedto displaythe variouscombinationpossibilitiesin aconciseandmeaningfulmanner.
15
Last but not least, the searchcan be for a componentdisplayingsomespecificsort of behavior, for example,a candidatecomponentfor valueingstockoptions. Ahypertext-basedlegacy browsingsystemcanprovidevariousstartingpointsfor suchasearch,suchasindexesonwordsoccurringin comments,columnnames,inferredtypes[11], andsoon. Moreovertypicalcomputationsnecessaryfor thebehavior of thecom-ponentlookedmaybeidentifiedusingplan recognition [12] – andthesecomputationsmaythenbepackagedinto therequiredcomponent.
5 Transformation Techniquesfor LegacySources
Tranformationtechniquesperform intrusive, systematic,modificationsof the legacysystemin orderto enabletheirmaintenanceandincreasetheir flexibility .
5.1 GeneralTechniques
With the insightsgainedby applyinganalysistechniqueswe want to transformthesourcecodeof a legacy systemto:
� globalyrestructurethewholesystem;
� restructurethecodeof individualcomponents;
� applyuniformcommentconventions;
� eliminatedeprecatedlanguagefeatures;
� convert to anew languageversion;
� Translateto anotherlanguage.
The transformationsshouldbe carriedout by a fully automatedrenovation fac-tory [5, 6] asshown in Figure7. Therenovationfactoryhasthreeinputs:
� Thesourcesof thelegacy system.
� (Optionally)therepositorythatresultsfrom theanalysisphase.
� A setof problem-specifictransformationrules
Notethat:
� The transformationrulesdependon the requirementsandarethusprojectspe-cific.
� Thetime neededto write thesetransformationsrangesfrom onehourto severalweeks.
� The transformationscanbe organizedasa pipelineof elementarytransforma-tions;thispromotesreuse.
16
Legacy System
Repository
Rewriting Engine
TransformationRules
Renovated system
Figure7: Architectureof aRenovationFactory.
All inputsarefedto arewriting enginethatappliesthetransformationsto thelegacysources.Theoutputis a renovatedsystem.
Thestate-of-the-artin renovationfactoriescanbecharacterizedasfollows:
� Mostavailabletoolswork only for onelanguage.
� Transformationsareimplemenetedassimplestringeditingoperations;thisdoesnotguaranteethesyntacticcorrectnessof theresult.
� Themostsophisticatedgenericframeworksarebasedon treetransformations.
� Typicalperformance:100-1000KLOC/hour.
Althoughit is appealingto speculateaboutfully automaticrenovationfactories,wewantto separatefactfrom fiction here.
It is afiction to expectuniversaltoolsthatcanautomaticallyrenovateany systeminany language.Thesameholdsfor universaltools thatcanextractbusinesslogic fromarbitrarysystems.
Thecrucialelementsin successfullrenovationtoolsare:
� Languageknowledge.
� Domainknowledge.
� SystemKnowledge.
17
� RequirementsandICT strategy.
It is afactthatmajorpartsof therenovationprocesscanbeautomatedbycombininghumaninsightwith full automationof repetitive tasks(speed,quality, reproducibility).
5.2 Support for StepB: Global Restructuring
Given the subdivision of the legacy systemresultingfrom StepA, we now want toreplacethe direct connectionsbetweenlegacy componentsinto indirect connectionsthatarehandledby thecoordinationarchitecture.Notethatthisstepis only moderatleyintrusive: thesystemis reorganizedinto componentsbut thecodeof eachcomponentis hardlytouched.Figure5(b)sketchesthis idea.
Thefollowing stepsareneededto achievethis:
� Identify thelevel of granularityatwhich thelegacy systemwill bedecomposed.Considerationsare:
– Thetotalnumberof componentsshouldbemanageable.
– Smallercomponentswith many crossrelationshipsarebetterhandledasasingle,larger, component.
� Wraptheidentifiedcomponentsin orderto connectthemwith thecoordinationarchitecture.In many casesthis meansinterceptionof theingoingandoutgoingcallsandreplacingthemby appropriateinteractionwith thecoordinationarchi-tecture.
� Write coordinationscriptsthat simulatethe connectivity in the original legacysystem.
5.3 Support for StepC: Renovate per Component
Given the outcomeof StepB, we canstart the renovationof individual components.This is acompletelyintrusiveprocessthatmaycompletelychangetheoriginalcodeofthecomponent.Themostinterestingpropertiesof thisapproachare:
� Themutualdependenciesbetweentherenovationprojectsof thevariouscompo-nentshavebeeneliminated.
� The renovationstrategy may differ per component:somecomponentsmay bereplacedby bying an existing commercialpackagewhile others,that containbusiness-specificknowledge,will undergo a very detailedanalysisandrestruc-turing.
In thosecasesthatthedecisionis to performadetailedrenovationbasedontheex-istingcode—typicallywhenmuchusefullbusinesslogic iscontainedin it—transformationtechniquesmaybeappliedto thecodeof thecomponent.Typical issuesare:
� Tranformationsaimingatcodeimprovement(e.g.,applyinguniformlayoutcon-ventions,gotoelimination,coderestructuring,anddeadcodeelimination).
18
� Transformationsaimingatthereplacementof certainpropertiesof thecode(e.g.,changeof theuser-interfaceor thedatabaseengine).
� Full translationof the codeto anotherlanguageor platform (e.g., conversionbetweenCOBOLdialectsor translationfrom obsolete4GL to standard3GL).
6 GenerationTechniquesfor LegacySources
The mostsophisticatedtechnicalapproachto renovation is to decomposethe legacysysteminto componentsin sucha way that thesecomponentsbecomereusableacrossdifferentapplications.Customizedversionsof thesecomponentscanthenbeusedindifferentconfigurations.
In orderto achieve thesegoalsa deeperunderstandingof the applicationdomainis needed.Domainengineeringattemptsto distill domainknowledgeout of legacysystems.
First, the legacy systemhasto be reorganizedto provide the domainknowledgeat theproperlevel of abstraction.Second,a notationtailoredtowardsthedomain—aDomain-SpecificLanguage(DSL)—hasto beprovidedto enabletheeasycompositionof componentsinto a workablesystem.Finally, the DSL will be usedasinput for agenerator(theDSL compiler).
Themajoradvantagesof thisapproacharea veryshorttime to marketachievedbyextensivereuseandveryeffectivecomponentcomposition.
Examplesof DSLsareEXPRESS—theinformationmodelinglanguagespecifiedinSTEP(ISO 10303-11)—thatis usedfor productdatarepresentationandexchangeandRisla6.3thatis usedfor thedescriptionof interest-basedfinancialproduct.Many otherDSLsexist in areaslikeplantcontrol,web-sitegeneration,configurationmanagement,etc.
6.1 Domain Engineering
Domainengineeringis concernedwith the identificationanddemarcationof applica-tion domains.A domainis hereunderstoodasa family of systemsin a well-definedapplicationarealike financeor processcontrol.
As a rule of fist, domainengineeringpaysoff whentherearealreadythreelegacysystemsavailablein thegivendomainandwhenit is expectedthatat leastthreemoreapplicationsin thedomainwill bebuild in thefuture.Domainengineeringis a form oforganizationallearning:theexperienceandknowledgethat is implicit in a numberoflegacy systemsis distilled into a conciseDSL andsupportinggenerator.
Thefollowing methodologicalissuesplayarole in DomainEngineering:
� Identificationof thestakeholders.
� Determinationof theboundariesof thedomain.
� Identificationof thedomainexperts.
19
Legacy code High-level product/system description
DSLcompiler
Extracted components
Renovated, component-based, system
Figure8: A domain-orientedapproach.
� Reengineeringof thelegacy systems.Resultsareinsightsin domainconceptsaswell asreusablecomponentsto beusedby theDSL compiler.
� Determinationof a lexiconof domain-specificterms.
� Determinationof the commonalityandvariability betweenthe variouslegacysystems.
� Designof a DSL.
� Designandimplementationof agenerator(compiler)for theDSL.
6.2 Domain-specificLanguages
Thelink betweenrenovationandconstructioncanbeimprovedby introducinga DSL.Theideais illustratedin Figure8 andworksasfollows. Insteadof implementingnewbusinessrequirementsdirectly(aswasdonein Figure1), weintroducehigh-levelprod-uct/systemdescriptionsatamuchhigherlevel. Thedescriptionsarewritten in anewlydesignedDSL that is tailoredtowardsthedomainin question.In addition,renovatedcomponentsthat have beenextractedfrom the legacy systemact asa library for theDSL. Fromthesedescriptionsandthecomponentlibrary, a DSL compilergeneratesanew application.
20
6.3 Risla: a DSL in the Financial Domain
Wehavealreadyappliedtheapproachsketchedherein thefinancialdomain.TheRislalanguage(partof CapGemini’sFPSproductsuite)is in useat, for instance,MeesPier-sonandING to describefinancialproducts.It wasbornout of two observations: thetime-to-marketfor innovativeproductsbecomesshorterandabank’sbackofficeis hardto adapt.
Basedon domainnotions(e.g., loan, swap, future, andFRA) a simplespecifica-tion languagehasbeendesignedthatcanbetranslatedto COBOL code.A library ofstandardcomponentsis linkedwith thegeneratedcode.
Key to thesuccessof Risla(andto futureprojects)are:
� Extensive domainknowledgethatwasembodiedin a high-qualityprocedureli-brary. This library capturestheknowledgeof a seriesof preceedingprojectsinthesamedomain.
� The insight that theplain useof the library itself resultedin lengthy, repetitive,unmaintainablecode. By introducinga DSL anda generatorthatproducesthecalls to the library (andotheradditionalglue code)commonknowledgeaboutthedomainandtheunderlyinginfrastructureis concentratedin thegenerator(asopposedto beingrepeatedin eachprogram).
7 Conclusions
Themajorchallengefor futurebusinessoperationsis to align changingbusinessgoalsandchangingtechnologies,while preservingthe assetsthat arehiddenin the legacysystemssupportingtoday’s businessoperations.In this paperwe have formulatedthemain questionssystemrenovationhasto solve, andwe have given a comprehensiveoverview of techniquesandapproaches.We have discussedtheanalysisandtransfor-mationof legacy systemsandhavealsodescribedhow domainengineeringcanhelptoidentify reusabledomainknowledge. By stressingthe needfor cooperationbetweenrenovatedsoftwareandnew softwarewe naturallyarrivedat theneedfor component-basedapproachesandcoordinationarchitecturesthat definethe cooperationbetweenold andnew components.We have presenteda threestepapproachto systemrenova-tion: find components,global restructuring,andrenovatepercomponent.Theneces-sarytechniquesfor analysis,transformation,andregenerationof legacy systemswerealsodiscussed.We concludedwith a descriptionof domainengineeringanddomain-specificlanguagesasviabletechniquesfor thestructuringandreuseof theapplicationdomainknowledgethatis embeddedin legacy systems.
Ourmainconclusionsare:
� Maintenanceandrenovationareeconomicallymotivatedactivities.
� Legacy systemsarea valuableassetfor businessoperations.
� Softwarerenovationaimsat extractingtheseassetsin theform reusablecompo-nents.Domain-specificlanguagescaptureessentialdomainknowledgeandcanbeusedto (re)generatesoftwaresystems.
21
� Component-basedarchitecturesenablethe cooperationbetweenrenovatedandnew components.
Basedon the analysisin this paper, we concludethat the key successfactorsforsoftwarerenovationarethefollowing:
� Renovationmethodsandtechniquesshoulddeliver reuseablecomponents.
� Oneshouldusesoftwarearchitecturesanddevelopmentmethodsthatarecomponent-basedandenabletheintegrationof renovatedandnew components.
� Domainknowledgecanbecapturedin reusablecomponentsthatcanbeexploitedin combinationwith DSLs.
References
[1] A.V. Aho, R. Sethi, and J.D. Ullman. Compilers. Principles, TechniquesandTools. Addison-Wesley, 1986.
[2] J.A. BergstraandP. Klint. Thediscretetime ToolBus—asoftwarecoordinationarchitecture.Scienceof ComputerProgramming, 31:205–229,1998.
[3] M.G.J.vandenBrand,P. Klint, andC. Verhoef. Re-engineeringneedsgenericprogramminglanguagetechnology. ACM SIGPLANNotices, 32(2):54–61,1997.
[4] M.G.J.van denBrand,P. Klint, andC. Verhoef. Reverseengineeringandsys-temrenovation– anannotatedbibliography. ACM Software EngineeringNotes,22(1):57–68,1997.
[5] M.G.J.vandenBrand,M.P.A. Sellink,andC.Verhoef.Generationof componentsfor software renovation factoriesfrom context-free grammars. In I.D. Baxter,A. Quilici, andC. Verhoef,editors,ProceedingsFourth Working ConferenceonReverseEngineering, pages144–153.IEEEComputerSociety, 1997.
[6] J. BrunekreefandB. Diertens. Towardsa user-controlledsoftware renovationfactory. In P. NesiandC. Verhoef,editors,Proc.Third EuropeanConferenceonSoftwareMaintenanceandReengineering, pages83–91.IEEEComputerSociety,1999.
[7] A. vanDeursen,P. Klint, andC. Verhoef. Researchissuesin the renovationoflegacy systems.In J.-P. Finance,editor, FundamentalApproachestoSoftwareEn-gineering(FASE’99), volume1577of Lecture Notesin ComputerScience, pages1–21.Springer-Verlag,1999.
[8] A. vanDeursenandT. Kuipers.Rapidsystemunderstanding:Two COBOLcasestudies.In SixthInternationalWorkshoponProgramComprehension;IWPC’98,pages90–98.IEEEComputerSociety, 1998.
22
[9] A. vanDeursenandT. Kuipers.Building documentationgenerators.In Interna-tional ConferenceonSoftwareMaintenance, ICSM’99, pages40–49.IEEECom-puterSociety, 1999.
[10] A. van DeursenandT. Kuipers. Identifying objectsusingclusterandconceptanalysis. In 21st InternationalConferenceon Software Engineering, ICSE-99,pages246–255.ACM, 1999.
[11] A. van DeursenandL. Moonen. UnderstandingCOBOL systemsusingtypes.In Proceedings7th Int. Workshopon ProgramComprehension,IWPC’99, pages74–83.IEEEComputerSociety, 1999.
[12] A. van Deursen,A. Quilici, andS. Woods. Programplan recognitionfor year2000tools. Scienceof ComputerProgramming, 1999.
[13] M. Fowler. Refactoring: Improving the Designof Existing Code. Addison-Wesley, 1999.
[14] I. Jacobson,M. Griss,andP. Jonsson.SoftwareReuse;Architecture, ProcessandOrganizationfor BusinessSuccess. AddisonWesley, 1997.
[15] C. Jones. AppliedSoftware Measurement: AssuringProductivityand Quality.McGraw-Hill, 1991.
[16] C. Jones.AppliedSoftwareMeasurement. McGraw-Hill, secondedition,1996.
[17] B. Meyer andC. Mingins. Component-basedevelopment:Frombuzz to spark.IEEEComputer, 23(7):35–37,1999.
[18] M. Simos.Organizationdomainmodelling(ODM) guidebookversion2.0. Tech-nicalReportSTARS-VC-A025/001/00,SynquiryTechnologies,Inc, 1996.URL:http://www.synquiry.com/ . 450pp.
[19] E. Stroulia, M. El-Ramly, L. Kong, P. Sorenson,and B. Matichuck. Reverseengineeringlegacy interfaces:An interaction-drivenapproach.In 6th WorkingConferenceonReverseEngineering, WCRE’99, pages292–301.IEEEComputerSociety, 1999.
[20] C. Szyperski. ComponentSoftware; Beyond Object-OrientedProgramming.Addison-Wesley, 1998.
[21] J. Withey. Investmentanalysisof softwareassetsfor productlines. TechnicalReportCMU/SEI-96-TR-010,SoftwareEngineeringInstitute,1996.
23
