A Scenario Selection Methodology Supporting Performance ......Performance Analysis of Theater Ballistic Missile Defense Engagement Coordination Concepts James F. Engler Sr., Brian

260 JOHNSHOPKINSAPLTECHNICALDIGEST,VOLUME23,NUMBERS2and3(2002)

J. F. ENGLER Sr., B. L. HOLUB, and S. MOSKOWITZ

D

AScenarioSelectionMethodologySupportingPerformanceAnalysisofTheaterBallisticMissileDefenseEngagementCoordinationConcepts

James F. Engler Sr., Brian L. Holub, and Simon Moskowitz

istributedweaponscoordinationdescribesanapproachforefficientlyrespond-ingtoincomingraidsthroughmaximumuseofavailableforceresources.Whilethiscouldbeappliedtoanumberofdifferentmissionareas,theprimaryfocustodayisonTheaterBallisticMissileDefense(TBMD),inwhichlargeraidsandadiversesetofdefendingunitsareexpected.Becauseofitshighlycomplexnature,real-worldexperimentsforthissce-narioareunlikely,indicatingthateffortstounderstandwhichdistributedweaponscoor-dinationconceptsaremosteffectivewillbeanalyticalinnature.TheseanalysesdependheavilyontheselectionofscenariosthattaxtheabilitiesofTBMDresourcestohandleanincomingraid.APLhasdevelopedamethodology,describedherein,basedonasetoffac-tors,includingraidsize,timing,locationsoflaunchandimpactpoints,andlocationsandcapabilitiesofdefendingassetsagainsteachthreat,toestablishstressingTBMDscenariosfordistributedweaponscoordinationanalysis.

INTRODUCTIONIn a world of proliferating, more accurate, longer-

range threats like theater/tactical ballistic missiles(TBMs),1heldbynationsthatarewillingtousethemand opposed by limited defensive capabilities, thedefenders must intelligently use their assets to defeatthese threats.2 One approach proposed is the use ofengagementcoordination (EC), ameansofdetermin-ingwhichofseveraldefendingunitswillbechosen(a“preferred shooter”) to engage each threat in a TBMraid. EC encompasses the methods, procedures, andprocesses used to preferentially select, schedule, andexecute engagements when multiple weapon systems

arecapableofengagingthreats.Selectionamongexist-ing preferred shooters is based on the application ofmutually established rules and criteria to the current,commonlyunderstoodtacticalsituation.

Shortofexperiencingafullraid,performanceevalua-tionsofdifferentECconceptswilloccurviaanalysisofasetofscenariosdeemedasappropriaterepresentationsofthechallengesthatTheaterBallisticMissileDefense(TBMD) forces will encounter. (TBMD and NationalMissileDefensearenowpartoftheoverarchingmissionofBallisticMissileDefense[BMD].)Ascenario3consistsofasetting(initialconditions),actors(TBMthreatsand

JOHNSHOPKINSAPLTECHNICALDIGEST,VOLUME23,NUMBERS2and3(2002) 261

SCENARIO SELECTION METHODOLOGY

TBMD units) that carry out events (for TBM threats,attack on opposition forces or sites; for TBMD units,threat detection and assessment, weapon selection andengagement),andgoals(forTBMthreats,destructionofoppositionlocations;forTBMDunits,thenegationofaraidtoapredefinedlevel).InexistingTBMDscenarios,threatsarenonreactiveandscriptedaspartoftheinitialconditions.

The challenge is to use scenarios that can demon-stratetheconditionsunderwhichdifferentECconceptsperformatdifferentlevelsofsuccess.Someofthesesce-nariosmayexist,althoughsomemaystillrequiregen-eration.Sinceanalyseshave limited resources,notallscenarioscanbetested.

Wecontendthatcriteriathatrepresentkeyfeaturesofusefulscenarioscanbeestablishedandusedaspartofamethodologytoselectviablecandidates.Thesecrite-riaareexpressedintermsoffeaturesofeachscenario’sunique environment description (threats and defend-ers).Bycomparing thesecriteria, theprocesshelps todeterminea setofappropriate scenarios thatwill sup-portthedesiredanalyses.

Two ongoing analysis efforts at APL—sponsoredseparatelybytheOfficeofNavalResearchandtheNavyTheaterWide(NTW)program—havebeentestingdif-ferentTBMDECconceptsthroughmodelingandsim-ulation. These two efforts have coalesced the specificcharacteristicsandmethodology for scenario selectionintotheprocessdiscussedinthisarticle.

BACKGROUNDThe challenge for the defending force is to use

all available defensive systems in a combined armsapproach and to select appropriate intercept opportu-nitieswiththeavailableresources.ThroughEC,apre-ferredshooterisselectedtoengageeachthreatsothataraid—onethatwouldoverwhelmanyindividualunit—canbehandledby the coordinated force.Theperfor-manceofECismeasuredprimarilyby

• Leakers:numberofthreatsthatreachtheirtargets• Free riders:numberofthreatsthatreachtheirtargets

withoutbeingengaged• Unintentional overengagements: number of intercep-

tors the force launches per threat in excess of thefiringdoctrine

Thechoiceofpreferredshooter,alongwithweaponchoice and intercept time (which together constitutea “preferred shot”), can be made using one of severalmethods,4distinguishedbyfourmaincharacteristics.

1. When the decision is made. Decisions can either bestatic (predeterminedandfixedduring theplanningphase; based on rules that are not sensitive to tem-poral situational changes) or dynamic (based on

the reactions and capabilities of other units as theraidunfolds).Static conceptsdowell atminimizingunintentionaloverengagements;however, ifa singleunit becomes saturated, no mechanism is availablefortransferringengagementresponsibilitytoanotherunit.

2. Basis for the decision.Decisionscaneitherbeunbid-ded (no external data on other units’ capabilitiesandintentionsareusedtomakedecisions)orbidded(data may be obtained on other units’ intentionsand capabilities prior to making a decision). Byemploying universal measures of performance, bid-dingallowsunitstodecideonthepreferredshooterwithoutdetailedknowledgeofthecapabilitiesoftheother units’ weapons, but comes at the expense ofincreaseddataexchangerequirementsandincreasedtimetomakethedecision.

3. Where the decision is made. Decisions can either bedecentralized (decisions are made locally by theTBMD unit on whether to engage) or centralized(decisions are made in one location by a “masterunit,” and orders are sent out to all TBMD units).Lossordisruptionofcommunications to individualunitsemployingdecentralizedconceptsislesslikelytoperturbcoordinationamongallotherunits.Thishelpstoavoidthevulnerabilityofcentralizedmeth-ods to single-point failure at the risk of potentialinconsistenciesinthedecisionsmade.

4. Decision-making frequency. This relates to the peri-odicthreatreassessmentandrecalculationofengage-abilityandschedules torespondtoneworupdatedtracks.Iterativedecisionsarecapableofrespondingtochanging/developing situations that even dynamic,noniterativeconceptsmaynotbeabletodo.

Taken in their simplest form, raids present threedifferent situations for preferred shooter solutions:(1) mutually exclusive, (2) multiple choice, or (3)singleoptimalsolution.Mutuallyexclusivesituations(Fig.1a) involve threats thathaveonlyonepossiblethreat–weapon pairing solution; no EC is required.Multiple choice (Fig. 1b) involves threats that canbeengagedbymorethanoneunitandresultsinmul-tiple solutionswithonlymarginaldifferences inper-formance.Asingleoptimalsolution(Fig.1c)involvesacombinationofsharedanduniqueengagements,sothatthechoiceofpreferredshooterwillresultindif-ferentlevelsofperformancefordifferentECconcepts,depending on threat timing and the relative geome-triesoftheengagements.Situationslikethoseshownin Fig. 1c demonstrate solutions that are nontrivialandrepresentchallengingorstressingconditionsforabattleforce.(“Challenging”or“stressing”referstositu-ationsinwhichresultantperformancelevelsarenearoroutsidetheacceptablemarginscalledforinsystemspecifications.)



Forexample,inthecaseofFig.1c,theECconceptof“shoot-and-shout”might leadto less-than-optimumresults. Assume that ships 1 and 2 have identicalresourcecapabilities insensors,weapons,andcommu-nications. Ship 1 is in a position (as defined by theengagementenvelopeshown)toengagethreatsheadedtowardimpactatdefendedassetMonatrajectoryfromlaunchpointJ(hereaftercalled“threatJ”),whileship2 is inaposition toengage threat JorK.Assume forsimplicity that each ship can only engage one threatata time. Inthatcase, ship2will launchfirstagainstthreat J because it has an earlier intercept time, andthenwillinformship1thatithasdoneso.IfthreatKwere launched a bit later than threat J, ship 2 wouldhavealaterlaunchtimeagainstthreatKcomparedtoJ.Becauseship1cannotengagethreatKandship2isbusywiththreatJ,threatKbecomesafreerider.Thesamesituationcanoccurforotherreasons,suchaswhendis-similarplatformcapabilitiesareemployed.

However, consider an iterative distributed engage-mentdecision-makingECconceptinwhicheachunitinforms the others of its engageability against eachthreatandlocallyarrivesatthesameconclusionsastowhich unit should shoot. In this concept, the initialassignmentcouldbethesameasinshoot-and-shout.AslongasthreatKisdetectedbeforelaunchagainstthreatJ by ship 2, distributed engagement decision makingwouldstillbeabletomakethereassignmentdecision.Thus,therewouldbenofreerider.

Compensating for performance shortfalls resultingfrom the use of a particular EC concept will have apotential impactonothersystemareas.Fortheprevi-ousexampleofshoot-and-shout,successagainstaraid

would require changes in system requirements anddesign,suchasdiscriminationofoneofthesethreatsbythenonlaunching shipor forwardpassof the in-flightinterceptor missile’s engagement to the other ship,among other options. Through the use of scenarios,theanalystcanhelpidentifytheextentoftheshortfallsof proposed EC concepts and their impact on overallsystemperformance;thus,acriticalneedexistsforthedevelopmentofappropriatescenarios.

SCENARIO DESIGN SPACE Scenarios are constructed to reflect theprimaryele-

mentsordriving factors that affectperformance in theanalysis problem space, as shown in Fig. 2. “Environ-ment”comprisesthesetofphysicalsituationsinwhichthe threats and defending forces must operate. This isnot justamatterofgeography,butcan includeaspectsofweather,timeofday,andgeneralatmosphericcondi-tions. “Threat capabilities” contain all possible tacticsandweaponseithercurrentlyinuse,expectedtobeusedinthetimeframeofthethen-fieldedsystems,ortheoreti-callypossible.Traditionally, themost influentialdriverhas been “defender capabilities” or engineering designrequirements. (These can include the minimum set ofcapabilities theuserwill accept, referred to as “thresh-old requirements,”andthe set that theend-userwouldprefer,referredtoas“objectiverequirements.”)

Whiletypicallyscenarioshavebeendevisedonasin-gle-platform basis, more recently5 emphasis has beenaddedonforce-levelsupportcapabilitiesthatallowavail-ableassetstooperatetogetherasa“familyofsystems.”This functionalitybeyond theunit level includes such

Figure 1. Ground-truth views of threats and engagement envelopes show three possible situations for resolution of threat–weapon pair-ings: (a) mutually exclusive, (b) multiple choice, and (c) single optimal solution.

J K

M N

JEngagement

envelope

J K

M N

JEngagement

envelope

N

J K

M N

JEngagement

envelope

M N

Ship 1Ship 2

Ship 1

Ship 2

Ship 1

Ship 2

(a) (c)(b)

X

X

X

X

X

X



abilitiesasbattleforceinteroperability(standardrepre-sentationandunderstandingofmessages),commonairpicture,andEC,butraisesissuesconcerningtheimpactoftiming,latency,andconnectivityonforce-wideper-formance.

AnappropriateECscenarioistheintersectionofthethree drivers in Fig. 2, which should be viewed as aninstanceofasinglescenario.Scenariosselectedshouldhaveenoughelementsof the threedrivers toprovideamorecomplexproblemspaceandmultiplesolutions,somebetterthanothers,toallowperformancesensitiv-ity analyses.Different scenarioswouldhavedifferent-sized intersections, indicating some thatmaybemoreappropriatethanothers.

SCENARIO CHARACTERISTICSAsdiscussedpreviouslyinrelationtothesituations

inFig.1,themostappropriatetypeofscenariooccurswhen at least one optimal solution set (i.e., one thatminimizesleakers,freeriders,andunintentionaloveren-gagements) of unique weapon/unit/threat assignmentsexistswithnumerouspossiblesuboptimalsolutionsets.Incomplete coverage overlap (i.e., all targets are notengageablebyeveryunit)andoversaturationofatleastone unit (i.e., higher localized raid density than theunitcanhandle)areexamplesofconditionsthatshouldhighlightthedifferencesamongvariousapproachestoEC. However, threat capabilities, raid geometry andtiming,defendedassetcharacteristics,defending forcecomposition,anddefendingunitcapabilitiesallimpactengagementdecisions.Thus,acombinationof factorsandcriteriacharacterizechallengingscenariosforbattleforceECconcepts.

ThecriteriathatwereconsideredintheselectionofscenariosforinitialECanalysesfallintothethreecat-egoriespresentedinFig.2andarediscussedbelow.

EnvironmentA complete scenario will involve a representative

environment.Elementsoftheenvironmentaretheter-rain(includingdefendedassets),weather,atmosphericconditions,andtimeofdayandyear.Itmayalsoincludeall air traffic (friendly, neutral, and hostile), commu-nicationstraffic,andcommandstructures.Forthepur-posesofECscenarioselection,environmentalelementsthatshouldbeconsideredaredescribednext.

Geography (Theater Location)Wherepossible,onemustusethetheatersidentified

in existing scenario documentation relevant to theTBMDprogram(s) involved.Locatingstressingopera-tionalsituationsinmultipletheatersprovidesagreatervariety of geometry in which to compare EC alterna-tives.Unlikely theaters (e.g.,ChileattackingAntarc-tica) or fictitious theaters (Persian Gulf attached toYellowSea)shouldbeavoided.

Defended Assets (Threat Impact Points)Multiple defended assets can complicate coordina-

tion because it is more likely that different units willhave different capabilities and responsibilities versusthethreatsheadedtotheseimpactpoints.Theseassetsshouldbeinvalidlocations(intheaterandinfriendlyoccupiedlandandseaareas).Thenumber,size,anddis-persionofareasthatanenemymightthreatenpresentchallengesinplanningablueforcelaydown,includingthe assignment of and interactions with one or moreblueforceunitsinthedefenseoftheseassets.

Threat CapabilitiesThe threat capabilities in EC scenarios consist of

thethreatplatformsused—whethercurrentorexpectednear-termweaponssystemsorsystemsthataretechni-callyfeasible—thetacticsusedintheirdeployment,andaspectsof thedefender’s reactions to the threats.Theoverallorderofbattle,withemphasisontheTBMorderofbattle,isalsopartofthisconsideration.Largeraids,havingseveraldifferentassetstargetedovertheentirerangeofthetheater,areaprobableapproachandmaybemorecredible.

Striking multiple targets would be a tactic used toovercome defense capabilities and permit at least onesite to be destroyed (this assumes multiple objectivesforanenemy);othertacticsmighttargetmorethanoneasset in the expectation of overcoming multiple sites.However, different tactics should also be considered,such as attacking a single defended asset with a largeraid,whilebasingtheplacementofdefendingunitsoncoveringtheentiretheater’sdefendedassetlist.Aswiththeenvironment,thereisasubsetofthreatcriteriathathelpsfocusselectionofanappropriateECscenario.

Figure 2. Scenario design space.

Environment

AppropriateEC scenario

Threatcapabilities Defender

capabilities



Types of ThreatsDifferentdefenderunitsmaybedesignedtoengage

dissimilarsetsofthreats(i.e.,differentseparatingand/ornonseparatingthreatswithdifferentranges).Theinclu-sionofmixedthreattypescanmakethesituationmorerealisticandpotentiallymorechallenging.Thisisespe-ciallytrueforseparatingthreatsandthosewithweap-onsofmassdestruction.However,it isnotrealistictoincludeallrequired/knownthreatsfortheworld’sorderofbattlewithinasinglecountry’sinventory.

Araidofamixtureofthreattypesprovidesasuitablerepresentation of all the threats that a force can beexpected to encounter at a given point in time. Abattleforcewillcontaindifferentunittypesdevelopedtodefendagainstspecifiedthreatsets,whichwilldiffer,evenamongunitswhosemissionsmaybesimilar.Twoexam-plesareTheaterHigh-AltitudeAreaDefense(THAAD)and NTW. THAAD is a ground-based BMD platformcounteringtheterminalphaseofballisticmissilesintheexo-andhighendo-atmosphericregions;NTWcoun-ters the exo-atmospheric midcourse phase of ballisticmissilesandrecentlywasrenamedSea-BasedMidcourseDefense.

In the formulation of a raid scenario, allocation ofsomenumberofeachthreattypeshouldbeconsidered,either equallydistributedamong typesor as apropor-tionalweightingofonetypeoveranother.Atthesametime,considerationmustbegiventoreality—thatthemixtureisrepresentativeofwhataparticularenemyhasorcanbeprojectedtohaveinthequantitiestoberep-resented in the scenario (ifnot, the credibility of thescenariomightbesuspect).

PositionThisaspectconsistsofvalidlocationsforthreatsin

theengagement,inparticularvalidlocationsforlaunchsites (generally enemy-occupied land areas, althoughsea-based positions may be posited). Multiple launchareasarelikely.Whileaparticulartypeofthreatmightbe restricted to a single launch site (because of fixedlaunchfacilities),manywillbemobile.Abalancemustbestruckhere.Althougheachthreatmightbelaunchedfromadifferentsite, it isalsotruethat,dependingonthesizeofthelauncharea,multiplelaunchesmayoccurthere. The possibility of launches outside these mostlikely areas should not be completely excluded fromconsideration because intelligence updates may ulti-matelyexpandthedefinitionofprobablelaunchsites.

Number of ThreatsThe preferred minimum value for raid size is the

threshold raid sizeof theprogram(s)of interest ifoneof thegoals is to test requirements. Ifagoal is to testagainsttheprojectedcapabilitiesofanadversaryandifthisistheinitialraidofaconflict,thentheadversary’s

maximum capability should be used. If this is not aninitialraid,theadversary’scapabilitywilllikelynotbemaximizedbecauseofattritionorlackofsetuptime.

Additionally,theremayberequirementinconsisten-ciesacrossdocumentsandplatforms.Requiredraidsizeisusuallywelldefined.Forexample,requiredraidsizeshavebeencalledoutintheNTWSystemRequirementsDocumentandNavyTBMDOperationalRequirementsDocument (ORD), but the Capstone RequirementsDocument for TBMD has its own values; other pro-grams’raidsizesmayalsodiffer.

Raid density is not as straightforward, however.Required unit raid densities have been called out intheNTWSystemRequirementsDocumentandNavyTBMD ORD but may differ from an overall requiredforceraiddensityandmayalsodifferbyplatform.Forasingleforceraiddensityvaluetoapplytoascenariowith dissimilar systems, the minimum values of forceraiddensity(derivedfromtheapplicabledocumentsoftheparticipating systems)mustbeused; this situationmayresultinsomeunitsnotmeetingaminimumunitraiddensityrequirement,butshouldbeconsidered.

Raid timing should also be taken into account.ThreatslaunchedwithadistributionoversecondswillaffectTBMDperformancedifferentlyfromthreatswithadistributionoverminutes.

Threat EngageablityThreat engageability is more an analysis criterion

thananyof theothersdiscussed so faranddealswithwhether all threats are potentially engageable. To beengageable,thethreatmustcomeintorangeofadefend-ing unit’s weapon system. This does not make anyassumptionsonwhetheraunitshouldengage,onlyonwhetheraunitiscapableofengagingaparticularthreat.It alsodoesnotmean that all threatswill actuallybeengaged,onlythatthepossibilitytoengageexists.

Allthreatsincludedinascenarioshouldbeengage-ablebysomeportionofthedefendingforce.Ifatleastonethreat isnotengageable, thenthescenario repre-sents an imperfect force laydown, a lack of sufficientforcestoaccomplishtheobjectives,orinsufficientunitcapabilities.

Defender CapabilitiesThe focus for EC scenarios is on the TBMD units

and their capabilities, taken separatelyandasa force.Elementsinthisdomainincludetheorderofbattleofdefender forces, the threshold and objective require-ments levied on individual systems platforms and onthebattleforce,thelaydownofalltheforces,andthecapabilities the force has to support fighting. As withtheotherdrivers forECscenarios,APLhas identifieda subset of criteria, described next, that supports theselectionprocess.



Firing UnitsForcecompositioninthetimeframeanalyzedshould

be considered in developing more realistic scenarios.For EC analyses, there must be at least a minimumnumberof units to perform coordination. Force-wide,thisminimumisat leasttwoTBMDunits,withthreepreferred.Thevaluesshouldbehigherifcoordinationamongthesametypeofunits isalsotobeconsidered(e.g.,twofromprogramXandtwofromprogramY,whenprograms might envision coordinating with like unitsbeforecoordinatingwithunlikeunits).Whiletwounitsprovidecoordination,decisionsarelimitedtotheselec-tionofoneortheotherunit(ornone);additionalunitsprovidemorethanonealternativetoamainfiringunitandincreasethenumberofpotentialsolutionsfortheentireraid.

ThenumberandtypesofTBMDunits included inthescenariowillalsoaffecttheoutcomeofEC.Analy-sesfocusedontheprogram-specificlevelarestraightfor-wardbecausetheyinvolveonlyonetypeofTBMDunit.However,amission-areaanalysiswillinvolveplatformsdevelopedundermorethanoneprogram.ForTBMD,thisiscomplicatedbythefactthatsomeunitshavedif-ferentrolesinalayereddefense.Theycanbemidcourseorterminal.Also,inamultimissionsituation,particu-larlythecaseforNavyvessels,therewillbeunitswithmorethanonemissionoperatingsimultaneously,com-petingforsystemresources.

AfurtherconsiderationiswhethertheECconceptwillmakedecisionsthattakeintoaccountallmissionareas(e.g.,TheaterAirandMissileDefense[TAMD],composedofrelatedmissionssuchasTBMD,Anti-AirWarfare[AAW],andOverlandCruiseMissileDefense[OCMD])orwillmakedecisionsforeachwarfareareaseparately and independently of other mission areasand/orservices.

PositionThisaspectconsistsofvalidlocationsfordefending

units based on terrain and bathymetric data (friendlylandsitesforground-basedunitsandsea-basedareasforsea-based units in deep enough water to support theships and far enough off shore from enemy positionstobedefendedby friendly forces).Also,TBMDunitsneedtobepositionedwheretheycandefendalltheirassigned assets against all of the assigned (suspected)threatlaunchareas.Manyexistingscenariosplaceunitswheretheycannotgo(e.g.,shipsinshallowwater)orcannotexecutetheirassignedmission.

Depth of FireA good defense-in-depth design will place firing

unitstomaximizeforce-widereengagementopportuni-ties, as well as to provide redundant coverage for thehigher-priority assets. The coverage and depth-of-fire

assessmentisusuallybasedonananalysisoftheengage-abilityorprobabilityofkillandtimeofflightofweap-onsagainstgiventhreats.However,therearenotalwaysenoughunits in the theater toprovide robustdefenseindepth,andagoodblueforcelaydownisnotalwaysagoalofthescenario.Thus,tomakechallengingandrealisticscenarios,instancesofuniquefiringunits(i.e.,nootherunithasopportunitiesagainstagiventhreat)oftenshouldbeincluded.

Mission RepresentationWhile the focus of this article is on the develop-

mentofscenariostosupportTBMDECanalyses,Navyplatforms tend tobemultimission innature.As such,resourceavailabilityforthosespecificunitsmaybefur-therlimited.Thus,dependingontheanalysisinvolved,morethanonemission(i.e.,AAWandTBMD)maybeconsidered.

Ifmorethanonemissionistoberepresented,theques-tionofwhetherthescenariosupportsalltherequiredmis-sionswouldalsoneedtobeconsidered.Nearlysimultane-ousthreatsinmultipleTAMDmissionareas(e.g.,AAWandTBMD)representamassedraidinwhichanenemycoordinatesweaponsfromaspectrumofapproaches(e.g.,TBM,overlandcruisemissiles,andairbreathers)toover-cometheabilitiesofdefenders.Attheprogramandmis-sion area levels, this has not been considered an issuebecause these were generally single-mission oriented;however, the TAMD Capstone Requirements Docu-ment and the Navy TBMD ORD contain key perfor-mance parameters for interoperability, and the basis oftheinteroperabilityparametersiscompliancewithasetof interoperability exchange requirements.The scenar-ioswillneedtosupporttheserequirements.Soalthoughmissionrepresentationisnotthesubjectofthisarticle,itislistedforthesakeofcompleteness.

METHODOLOGY APLanalysiseffortshaveledtothedevelopedasce-

narioassessmentprocess(Fig.3)consistingofidenti-fication,evaluation,andselection.“Identifycandidatesources” is theprocessoffindingpotential sourcesofexisting scenarios relevant to the analysis; this mayincludemissionsofinterest,forcelaydowns,ordetailsona theaterof interest (terrain, air and surface traf-fic,andweather).“Assesscandidates” istheprocessofdeterminingvaluesforasetofmetricsforeachcandi-datescenario.APLfoundthatacombinationofvisualinspectionofthenumbersofdifferentobjects(launchandimpactpoints,threattypes,anddefendingunits)andatoolthatcangeneratelaunchandinterceptwin-dows(e.g.,EADSIM,theExtendedAirDefenseSimu-lation) provides a reasonable method for performingtheneededcalculations.“Selectscenarios”providesasmallsubsetofcandidatesforuseintheanalysis.Ifnosuitablecandidatesarefoundaftergoingthroughthis



process,variantsonthecandidatesshouldbetried;ifthatfails,newscenariosshouldbedeveloped.Ineithercase,thissameprocessshouldbeapplied.

Manyscenariosmaybeconsideredascandidates,butsomemaybemoreappropriatethanothers.Todeterminewhichwouldworkbest,benchmarkcriteriavaluesneedto be established against which scenarios can be com-pared.Benchmarkcriteriashouldbetraceabletoexistingplans,documentation,intelligence,andoperatorexperi-ence.Insomesituations,noneofthescenariosselectedas candidates may meet the minimum conditions oftheseothercriteria.Inthosecases,thecriteriashouldberelaxed,eitherbylooseningthestricturesofagivencri-terionorignoringcertaincriteria.

Anotherconsideration is thedevelopmentofcom-posite stressing scenarios.Because candidate scenariosarederivedfromdifferentprograms,compositescenar-ios(asinmultimissionshipoperationsorJointTBMDanalyses)arepossible.Inthesecomposites,appropriateelementsfromdifferentsourcesfillinthefullpictureofthetheater(weather,allforces,terrain,andactivity)tosupportsimultaneousmultiprogram,multiservice,mul-timissionoperations.

An example of elements that may be includedin a composite scenario is given in Fig. 4. Activ-

ity time periods for events related tomultiplemissions(TBMD,OCMD,andAAW)areindicatedalongwiththreatsfor different programs (endo- and exo-atmospheric TBMs) and a background(level of communications traffic, airtraffic,weather,andterrain).

Atthecoreoftheprocessisacheck-list thatgives theanalystamethod fordetermining the appropriateness of agivenscenario.Notalltheitemsonthe

Selectedscenarios

Selectscenarios

Assesscandidates

Identifycandidatesources

Start

Selection criteria

Figure 3. Methodology process.

listhavetobesatisfied.However,ifaparticularitemisnot involved then theanalyst shouldunderstandwhythescenarioisstillvalid.

At times during “what if ” analysis, values will beused that are larger or smaller than the documentedrequirementsvalues.Forexample,ifascenarioresultsintoomanyleakers,thenonemaywanttoexaminewithlesservalueswhethertheincreasednumbersofleakersislinearorhassomekneeinitscurve.Also,oneortwotargetsmay,attimes,beunengageable.ThisknowledgecouldbeusefulwhenevaluatingscenariovariantssuchastheimpactontheECconceptwhenashooteroracommandandcontrolnodeisdestroyed.

Theselectionofscenarioscanbeconsideredatwo-stepprocessinvolvingthechecklistinTable1.Inthefirst pass, the analyst evaluates a candidate scenariobased on the minimum that it must do. This meansthat the scenario, as presented, meets a minimumset of acceptable criteria; as soon as a scenario doesnot meet one criterion, it is not a candidate for thesecond pass. In the second pass, the analyst evaluateseachsurvivingscenarioandassignsquantitativevaluesusing visual inspection and simulation tools. Thesescenarios can then be compared to determine theirrelativeworth.

“BACKGROUND”

Time (Not Drawn to Scale)

TBMD

OTHERMISSIONS

Exo. flight Des

Endo. Flight

Weather (e.g., stressing/average, varying/static)

Terrain

Air/Surface Traffic (Friendly, Hostile, Neutral)

Communications Traffic

Command Structure (e.g., defined constant, defined changing)

Sensor, Weapon and Communications Resource Loading Requirements

“Background”

Time (not drawn to scale)

TBMD

Othermissions

Exo Des.

Endo. flight Des.


Terrain

Air/surface traffic (friendly, hostile, neutral)

Communications traffic

Command structure (e.g., defined constant, defined changing)

Sensor, weapon, and communications resource loading requirements

Asc.

Asc.

Figure 4. Example of a composite scenario.



Table 1.Two-passmethodologychecklistsintermsofscenariocharacteristics.

ChecklistCharact-eristic Pass1 Pass2Theater UseofTBMspossible/likelylocation

Positions BlueNavyforcesnottooclosetotheadversary’s shore-baseddefenses

Waternottooshallow(noshipsonthebeach)

Bluelandunitsplacedwithinthebluesector (unlessitisanaircraftorspecialoperationsforces)

Redforcesinappropriatelocationsfortheaggressor (nolandlaunchpointsinthemiddleoftheblue sector)

Threat Multipledefendedassets Raid density in terms of threats for each defendedimpact asset(asdemonstratedinahistogrambinnedbythreatspoints perdefendedasset,labled“Pass2DARaidDensity”in Figs.5and6)

Typesof Amixtureoftypes,withpreferencetohavingatleast Howwellthethreattypesarerepresentedthreats oneofeachthreattyperepresented Multiplethreattypespresentsimultaneously,whichcan bedeterminedbycomparingachartofinterceptand launchwindowsforeachunitagainsteachthreat(tobe foundinFigs.5and6)withtheirassociatethreattypesNumbersof ContaintheappropriateORD/systemrequirementsthreats documentspecifications(thresholdforminimum valueandobjectiveformaximumvalue);flexibility desirable

Forceraidsizecommensuratewiththenumberofunits involved(i.e.,theproductofnumberofunitsandre- quiredunitraiddensity;e.g.,iftherequiredunitraid sizewere2threatsperunitandtherewere4units,then aforceraidsizeof8wouldbeminimallyacceptable,even iftherequiredforceraidsizewere6threatsor12threats)

Allunitshaveatleastaminimumrequiredraidsize Oneormoreunitswithamaximumrequiredunitraid densityattheunitlevel,asdemonstratedinatableof density shipsandassociatedcountofthreatspershiptobe foundinFigs.5and6

Unitshaveatleasttheminimumrequiredraid Raidtimingpermitsmultiplethreatstobeinengage- timingrelativetorequirements mentevelopessimultaneously

Engage- Allthreats(thataplatformisdesignedtoengage) Numberofthreatsthathaveonlyonepotentialability areengageableatsomepointintheirflightpaths defenderthatcanengage.Thisisdemonstratedina asdemonstratedinatableofthreatsandassociated histogramofthenumberofthreatinstancesbinnedby countofshipsperthreat,labeled“Pass1Engage- thenumberofshipsthatcanengageathreat,labeled abilityTest”inFigs.5and6 “Pass2EngageabilityTest”inFigs.5and6

Firing Multipleshooters—program-specific(e.g.,Navy Mission-specific(e.g.,TBMD)withmultipleplatformunits AreaandNTW) types

Multipleshooters—multimissionwithmultiplatforms

Depthof Unitandforcedepthoffireforeachthreat(asdemon-fire stratedbyatablerepresentinginterceptopportunitiesfor eachunitagainsteachthreatandfortheforceasawhole againsteachthreat,labeled“Pass2DepthofFireTest” inFigs.5and6).Tocalculateunit/forcedepthoffire, assumeaprobabilityofkillofzeroandperfectkillassess- ment;calculationresultsin,atmost,howmanytimes theplatformandforcecantakeashot

Missionrepre- Howmanywarfareareasaresupportedbythescenario, NearlysimultaneousthreatsinmultipleTAMDwarfaresentation andaretheythecorrectwarfareareasfortheanalysis? areas



EXAMPLE APPLICATIONS OF THE METHODOLOGY

Themethodologycanbeillustratedwithapairofsce-nariostobereviewedaspotentialcandidates.Becauseofissueswithclassification,theexamplesdescribedherearegenericandhypotheticalbutstillservetoillustratethe methodology. Figures 5 and 6 contain elementsusedtoevaluatetherelativemeritsofthetwoscenarios(for brevity, a subset of the features described in thisarticleispresented).TheupperleftportionsofFigs.5and6showagraphicoftheforcelaydownandscriptedthreat trajectories and can be used to calculate the

Figure 5. Feature summaries, first candidate scenario.

Figure 6. Feature summaries, second candidate scenario.

pass1tables.Theinterceptandlaunchwindowchartsin thecenterofFigs.5and6 identify the intervalsoftimeinwhicheachshipunitcanlaunchagainsteachthreatwithatleastaminimumacceptableprobabilityofkillandinwhicheachlaunchedmissilecanbeexpectedto intercept the threat. (Thepairsof threatswith thesame launchand impactpointsare illustratedherebyonlyoneofthepairforsimplicity.)Thischartservesasabasisforcalculatingthevaluesinthepass2tablesontherightofFigs.5and6.

Figure5showsarepresentationofaraidof10threats,launchedinpairs(labeledintheinterceptandlaunchwindowchartandinthefigure’stablesasA1,A2,B1,



B2,etc.)from5launchsites(A,B,C,D,andE)toward2impactpoints(XandY).Threeunitsofthesametype(e.g.,upper-tiership-based)aredefendingwithsimilarengagementvolumesthroughwhichthesethreatspass.Differentthreattypesoriginatefromeachofthelaunchareas.

Assumeforthisgenericcasethatthedefendingsystemis required tohave capability against 5 threat types (athreshold raid size of 10 with an objective of 20) andthresholdunit raiddensitiesof2 threatsper shipwithan objective of 4. Figure 6 shows a second scenario,similar to thefirst,having the samenumberof launchpoints, impact points, and defending units. However,one launch complex, labeled E, launches one TBM atassetXanditssecondTBMatY,whileA,B,C,andDtargettheirpairsatonlyoneofthetwoimpactpoints.

Inpass1ofthemethodology,bothscenarioscanbeidentified as candidates. The relevant criteria valuescan be determined from the scenario laydown andcolumnoftablesontheleftofFigs.5and6.Unitraidsizesareidenticalandarewithintherangeofforceraidsizeasdefinedinthechecklist.Allunitsmeetand,inthiscase,exceedtheobjectivevaluesof thenumberofengagementstheycansupport.Thesameunitsexistinboth scenarios. In fact, for every item in thefirstpass,thesetwoscenariosarealmostequalcandidates,with the slight differences visible in pass 1 tables ofFigs.5and6.

Itisinthesecondpassthatthedifferencesbetweenthetwoscenariosappearmorepronounced.Becauseof the shiftofoneTBMaimpoint,oneof the threefiringunitshasengagementopportunitiesagainstoneadditionalthreatthanitdidbeforewithoutdetrimentto the number of threats that the other two couldengage.Inmostcases,thelaunchandinterceptwin-dowsoverlap; if thefiringpolicy is assumed tobe asinglesalvowithashoot-look-shootcapability,mostof the firing units can launch a second interceptormissile before the launch window closes (in somecases,athirdshotmightbepossible).However,thereareinstanceswhereaunitmayonlylaunchonce.Theresult of these differences is a slightly greater depthoffireandmoreengagementopportunitiesinFig.6,at the expense of slightly fewer threats having onlyone ship that can engage them. This implies thatFig. 5, because there are more instances of single-solutionthreat–weaponpairings,maybemorelikelyto present different performances between differentECconcepts.

Bothoftheseexamplesarewellqualifiedforuseinanalysis activities. However, if one were to prioritizethemforevaluation,Fig.5wouldbethefirstchoice.

CONCLUSIONSThebasicmethodologydescribedhereinwasapplied

in the selection of an existing scenario for ongoing

NTWECanalyses.Oftentherewillbeaclearindica-tion,evenafterthefirstpass,thatonescenariomakesa better candidate than another. However, in caseswhere scenarios are very similar, this methodologyhelps to identify which conditions are more likely toyieldcasesthatdemonstratedifferencesinECconceptperformance.

Thepurposeof themethodology is to identify thatintersectionofthreat,platform,andenvironmentthatwillbeappliedtoallECconcepts.Thisisonlythefirststepintheanalysiseffort;theactualapplicationofthescenariostothedifferentECconceptsandacomparisonoftheirresultingperformanceareseparatetasks.Suchcomparisonsmay leadtothe identificationofconcep-tualshortfallsthatshouldbetakenintoaccountduringtheactualsystemdesignphase.

The methodology indicates a number of differentdirectionsinwhichfurtherresearchcouldbeperformed.Three such avenues are discussed in the followingparagraphs.

Thefirst area is theextensionof thismethodologytosupporttheautomatedgenerationofnewscenarios.Quantificationofthechecklistwouldsupportthispro-cessbyprovidingafigureofmeritforcomparingsepa-ratescenarioinstancesintermsofrelativefitness,whichis similar to a strategy identified for scenariomanage-ment.6Forinstance,afigureofmeritforthecomposi-tionofa scenario raid(thenumberof threatsofeachthreattypeintheraid)couldbebasedonhowequallythethreattypesarerepresented,orasaweightingofonetype over another. One could posit a tool that couldinvestigatewhichmixturewouldprovidethebestchal-lenge to a given force or which mixture of defendingassets would be most successful against a given threatmixture.

Asecondareaforfurtherresearchisthegenerationand evaluation of scenarios using a Monte Carloapproach.Here,theTBMorderofbattlewouldbefixedandallocatedtospecificlaunchareas.Mobilelauncherswouldrandomlywanderfromtheirbasesandrandomlytargetaperceiveddefendedassetarea(to fulfilla raidquotaorasweightedbyassetpriority).TheTBMimpactpoints would differ from the intended asset location(potentiallyresultinginmissingthetargeteddefendedareas,whichcould lead to thedecisionnot toengagethethreatbytheTBMDplatforms).Inaddition,theirflight paths and orientations would differ from speci-fied nominal values. TBMD units would be assignedoperating areas but would assume random positionswithin those areas for each experimental run. Thisapproachwouldconstrainthreatcapabilitieswhiletai-loringtacticalflexibilitytopresentvaryingrealizationsofwhatmightoccur.

The focus of this article has been on the selectionof scenarios for one mission area only, but, as noted,other missions are also part of the equation. Navy



surfacecombatantsaretypicallymultimissioninnature,andacoordinatedenemyattackinvolvingavarietyofsimultaneousairandmissileweaponsmayseverelytaxforceresources.

Figure 7 depicts this extension of the compositescenario shownearlier in thearticle.Theextensionofthismethodologytomultimissionoperationsisalogical,albeitchallenging,nextstep.Themethodologyattemptsto quantify situations, but to date has not directly

addressedtheissueofmultimissions.Therelativeimpor-tanceofonemissionareaoveranother is subjective innature,soquantifyingrelativeimportanceandobtainingcommunity concurrence in the weighting of these dif-ferentmissionswillbecontentious.However,thisisanareainwhichfurtherinvestigationiswarrantedbecausethesemissionareasareverydifferent,yethaveoverlap-pingresourcerequirements(useofsensors,launchersandpermittedsimultaneousengagements).

Figure 7. Example of composite, multimission scenario (ALCM, GLCM, and SLCM are, respectively, air-, ground-, and ship-launched cruise missiles).

REFERENCES 1Oberg,J.,“MissilesforAll:TheNewGlobalThreat,”IEEE Spectrum36(3),20–28(Mar1999). 2Kadish,R.T.,testimonybeforetheDefenseSubcommitteeoftheSenateAppropriationsCommittee,106th

Congress(12Apr2000),availableathttp://www.acq.osd.mil/bmdo/bmdolink/html/kadish12apr00.html. 3Chance,B.D.,andMelhart,B.E.,“ATaxonomyforScenarioUseinRequirementsElicitationandAnalysis

ofSoftwareSystems,”inProc. 1999 IEEE Conf. and Workshop on Engineering of Computer-Based Systems,IEEECSPress,LosAlamitos,CA,pp.232–238(1999).

4Shafer,K.,“DevelopmentandAnalysisofWeaponsCoordinationConceptsConsideringtheImpactofanImperfectAirPicture,”inProc. National Fire Control Sym. (2001).

5Cebrowski,A.K.,andGarstka,J.J.,“Network-CentricWarfare:ItsOriginandFuture,”U.S. Naval Inst. Proc. 24(1),28–35(Jan1998).

6Alspaugh,T.A.,Anton,A.I.,Barnes,T.,andMott,B.W.,“AnIntegratedScenarioManagementStudy,”inProc. IEEE Intl. Symp. on Requirements Engineering,pp. 142–149(1999).

ACKNOWLEDGMENTS:Thisarticlewouldnothavebeenpossiblewithoutthesupportofanumberofindividualsandorganizations.TheauthorswouldespeciallyliketothankKenShaferoftheAPLCoordinatedEngagementSimulationteam,whoreviewedthearticleandwhosecontinuinginterestandencouragementhavehelpedtorefineit.Thisarticlewassup-portedbytheactivityoftheOfficeofNavalResearchandNTWprogramoffices.

“Background”

Time (Not Drawn to Scale)

TBMD

OCMD

AAW ABT Ingress

Boat

Launch/Flight of GLCM

Launch/Flight of SLCM

Launch/Flight of ALCM Terminal

Terminal

Terminal

Launch/Flight of OCM Term.

Asc. Exo. Flight Des

Asc. Endo. flight Des


Terrain

Air/Surface Traffic (Friendly, Hostile, Neutral)

Communications Traffic

Command Structure (e.g., defined constant, defined changing)

Time (not drawn to scale)

TBMD

OCMD

AAW ABT ingress

Boat

Launch/flight of GLCM

Launch/flight of SLCM

Launch/flight of ALCM Terminal

Terminal

Terminal

Launch/flight of OCM Term.

Asc. Exo. flight Des.

Asc. Des.


Terrain

Air/surface traffic (friendly, hostile, neutral)

Communications traffic

Command structure (e.g., defined constant, defined changing)



THE AUTHORS

JAMESF.ENGLERSr.isamemberofAPL’sSeniorProfessionalStaffintheSys-temsEngineeringGroupofADSD.HereceivedaB.E.E.in1981fromtheCatholicUniversityofAmericaandanM.S.incomputersciencefromTheJohnsHopkinsUniversityin1985.BeforejoiningAPL,heworkedforWestinghouseElectricCor-poration(nowNorthropGrumman)inBaltimorefor16years.WhileatWesting-house,Mr.EnglerwasinvolvedinthedesignandsimulationofembeddedsignalanddataprocessingavionicssystemsforDoDprogramsandlaterinthedesign,model-ing,andinstallationofsystemsforlawenforcementagencies.SincejoiningAPLin1997,Mr.EnglerhasfocusedonBMC4IeffortsforNTWandonnavalinteroper-ability.HeisamemberoftheInstituteofElectricalandElectronicsEngineers.Hise-mailaddressisjames.engler@jhuapl.edu.

SIMONMOSKOWITZearnedaB.S.inaerospaceengineeringfromtheUniversityofVirginiain1993andanM.S.insystemsengineeringfromtheUniversityofAri-zonain1994.HeisaSeniorProfessionalStaffmemberoftheAirDefenseSystemsEngineering Group of ADSD. Mr. Moskowitz joined APL in 1994. Theater AirandMissileDefense radar searchandengagement coordinationalgorithmdevel-opmentandmodelingcomprisehis currentprimary focus.His expertise includesTBMDthreatcharacterization,conceptofoperations/scenariodevelopment,effec-tivenessanalysis,visualization,requirementsdevelopment,andspecificationreview.Hise-mailaddressissimon.moskowitz@jhuapl.edu.

BRIANL.HOLUBisamemberoftheSeniorProfessionalStaffintheAirDefenseSystemsEngineeringGroupofADSD.HereceivedaB.S. ingeneralengineeringin1982 and anM.S. indigital systems in1985 fromLoyolaCollege.He is cur-rentlyworkingonaSystemsEngineeringCertificatefromtheJHUWhitingSchoolofEngineering.BeforejoiningAPLin1999,Mr.HolubworkedforWestinghouseElectricCorporation(nowNorthropGrumman),wherehefocusedonsystemsengi-neeringinsupportofC2functionsforairdefense,airtrafficcontrol,andlawenforce-mentproducts.SincejoiningAPL,hehassupportedNavyBMC4IeffortsfocusingonTBMDandNTWsystems.HehasbeenamemberofIEEEfor13yearsandamemberoftheInternationalCouncilofSystemsEngineering(INCOSE)[email protected].

A Scenario Selection Methodology Supporting Performance Analysis of Theater Ballistic Missile Defense Engagement Coordination ConceptsJames F. Engler Sr., Brian L. Holub, and Simon MoskowitzINTRODUCTIONBACKGROUNDSCENARIO DESIGN SPACESCENARIO CHARACTERISTICSEnvironmentGeography (Theater Location)Defended Assets (Threat Impact Points)

Threat CapabilitiesTypes of ThreatsPositionNumber of ThreatsThreat Engageablity

Defender CapabilitiesFiring UnitsPositionDepth of FireMission Representation

METHODOLOGYEXAMPLE APPLICATIONS OF THE METHODOLOGYCONCLUSIONSREFERENCESTHE AUTHORSFIGURES and TABLESFigure 1. Ground-truth views of threats and engagement envelopes show three possible situations for resolution of threat–weapon pairings.Figure 2. Scenario design space.Figure 3. Methodology process.Figure 4. Example of a composite scenario.Figure 5. Feature summaries, first candidate scenario.Figure 6. Feature summaries, second candidate scenario.Figure 7. Example of composite, multimission scenario.Table 1. Two-pass methodology checklists in terms of scenario characteristics.

Documents

A Scenario Selection Methodology Supporting Performance ......Performance Analysis of Theater Ballistic Missile Defense Engagement Coordination Concepts James F. Engler Sr., Brian