Embed Size (px)
Citation preview
1
Front-End Operations Research
Neville J CurtisLand Operations Division
Defence Science and Technology [email protected]
2
Outline of the presentation
The wide scope of OR What do I mean by OR What do I mean by the front end Where the degrees of freedom are Why we have to be rigorous Some ways of working at the front end Some examples of what we’ve learnt
- Words- Pictures- No formulae- And very few numbers!
3
Jobs that an Operations Researcher may do
Provide an exact number to solve something Provide a best fit of inputs to do a job Do a quantitative comparison of some similar bits of kit to do
a well defined role Compare competing options of procedure, organisation or
acquisition Look at an existing system to find options for change
caused by new desired outcomes, environments or technologies
Influence the debate on a future change
Provide analytical scrutiny for any of the items above
4
Puzzles, problems and messes
Puzzle Problem Mess
Issue Agreed Agreeable (terms of reference, weightings etc)
Arguable (what’s the issue? etc)
Advice Agreed Arguable Really arguable
Type of advice
Numerical solution with checkable maths
Reasoned answer with an audit trail
Bright idea (policy, concept etc)
Example
How many tanks? Which tank to
buy?
Think about what capability a tank offersHarder OR
Softer OR
5
The Puzzle, Problem and Mess Spectrum- what do we do if we have a Mess?
Puzzle Problem Mess
Issue Agreed Agreeable (terms of reference, weightings etc)
Arguable (what’s the issue? etc)
Advice Agreed Arguable Really arguable
Type of advice
Numerical solution with checkable maths
Reasoned answer with an audit trail
Bright idea (policy, concept etc)
Example
How many tanks?
Which tank to buy?
Think about what capability a tank offersHarder OR
Softer OR
Turn it into a problem
Work in themess space
6
Why Front-End OR?
Because we need:- To understand, as far as we can, the issues, so that:- We can usefully explore the system, so that:- We can propose bright ideas, so that:- Decision makers have credible, auditable and transparent material, so that:- Longer term programs are set in the right direction with less risk
downstream
And we don’t want:
with disastrous long term consequences…..
Ill definedcurrent situation
Rigid planfor change
“Then a miracleoccurs”
7
What do we need to get things going?
When no one has given us an optimisation puzzle or three options to compare, and:
- We don’t know much about the system under study- We don’t know much about measures or ways of looking at it- We don’t know how things are linked (trade-offs, antagonisms,
synergisms)- We don’t know what matters and what doesn’t- We don’t know what the clients and owners are worried about- We don’t know what can be changed- We don’t know what data there is (or if it matters)- We don’t know how important the “length of the string” is- We don’t know how the system will respond to changes in context- And we only know that there is a desire to improve a system, and of
course: We don’t know what “improve” means
- These questions, of course, tell us what we need to do!
8
Light-hearted diversions
We tend to see two types of response to the list on the previous vugraph:
1. The pessimist – throws hands up in horror, desperately looks for previous work as a reference, realises that there is no logical way of doing it, and asks to do something else
2. The optimist – greets task with glee, sees lack of bounds and constraints as an opportunity, relishes being able to be creative, becomes very enthusiastic
Outrageous (inflammatory) generalisation:- Soft OR is better suited to Myers-Briggs NT preferences (physical
scientists) – possibilities and analysis- Hard OR is better suited to Myers-Briggs ST preferences
(mathematicians) – facts and analysis
9
This presentation
Will not tell you how to do a specific Front-End study Instead I’ll give some ideas of how it could be done through
examples:- How do we progress soldier modernisation?- How do we address counter terrorism?- How do we guide the future Army?
These have differing degrees of previous shaping, ranging from making sure that you’ve included all that needs to considered (soldier modernisation) to a blank sheet (counterterrorism)
The common feature is that they are all areas where we had to do a preliminary (front end) OR investigation
I will present in terms of models and some manipulations We are now getting to the stage where we have “standard”
methods in our tool box
10
Front End Operations Research
I will explore and structure issues that are poorly defined or understood through using my scientific skills, backed by numeracy, literacy and communication so that subsequent studies are much better focussed and in the long run, decisions on ways forward are made with less risk
ie I will do P, by means of Q to achieve R
11
Some ideas to get started at the Front-End
Model the system, include:- Context- Actions (verbs) and influences- System components (nouns) and interactions- Axes of interest- Notions of measures (may be hierarchical) - While remembering that models are useful not necessarily right and that
models could be diagrams, tables, pictures, spreadsheets, agent based simulations……
Manipulate the models so that:- We have a “fit for purpose” feel for what’s going on when things are done
under different contexts or vignettes- We know where things can change (ie go wrong or be made better)- We can come up with “bright ideas” as options for change- We can say something useful about these options- We can advise decision makers about possible ways forward (ie not the
way)
12
Example of Front-End studies for Soldier Modernisation
Context – Army’s role is changing: technology is becoming smaller, cheaper, less likely to break and users are more technology savvy. We want to look at modernising the dismounted soldier. This is a “Problem” (but it turned out that it needed some work to refine it better..) .
Outcome (Concept) – I will advise decisions makers on what sort of things need to be looked at (equipment, doctrine, integration, useability, testing and evaluation), by means of doing a thorough front end study including understanding what the dismounted soldiers do now, how they could do things in the future and how technological insertion and acquisition may be justified to decision makers so that dismounted combatants will be better equipped for success in modern warfare
To put it another way – we want to have sufficient understanding of the system so that we can discover good ways forward with our partners (coach role) and provide credible evidence to the fund owners (judge role) that soldier modernisation should go forward
- NB fine details of exactly what and how many to buy come further down the track and are not part of a front-end study
13
Our first question
What does the infantry do and how will it be improved by the use of this new technology?
This is where the work was needed…
14
What did soldier modernisation teach us (1)?- understand the whole picture
Soldier interactions: equipment, physical environment, civilians, red forces, other soldiers…
Another soldier system
This visual model reminds us to consider the entire system, not just the soldier and his kit. Eg what is the effect of
the physical environment on the equipment?NB it has changed in the last ten years: now fight in the urban
– this brings new physical environments and civilians into the mix
SoldierCivilians
Red forces
Equipment
Physicalenvironment
15
What did soldier modernisation teach us (2)?- We need to classify actions, but a single list isn’t enough
Activities (what)Patrol
Assault
Defend
Observation post
Check point
Tactical relocation
Escort
…..
(NB more than 7)
Core Skills (how)Engagement
Information Collection
Sustainment
Movement
Decision Making
Communication
Protection
(NB the magic 7)
Big question: what do soldiers do when on an operation?Approach: create a model with two axes
NNB these all have meaning to the decision makers and the end users (front end OR is not a black box)
16
Specific versus generic: a degree of freedom
Activities are:- Situation specific (attack that hill, with that force, that equipment, that
enemy, those environmental conditions), that- Can be wargamed and this gives you quantitative results - Can give “statistical” results, but it’s only one point- Can be broken down into steps- Can be investigated in depth to find the critical part- Can be perturbed to learn a little more- And if you do enough in-depth analyses you can get some insights about
the overall landscape Core skills are:
- Generic elements used to a greater or lesser extent in the activities, that- Are less amenable to quantification- Provide a different sort of insight- Can be modelled in terms of contribution to the overall goal (the “ility”
issue)- Provide common threads on issues like equipment/technology insertion
17
What did soldier modernisation teach us (3)?- use both axes to tease out all the issues
A Bunch of Guys Sitting around a Table (BOGSAT)… A seminar wargame of thinking about the issues that are likely to occur
with the first round of soldier modernisation equipment For a stage in an activity go through all the prompts (skills) and ask
what can go wrong (pessimistic analysis):- Technology not good enough- Environment against you- Blue force constraints- Red force issues (logical and lateral thinking!)
Come up with technology products (or other approaches) that could improve things
Group by core skill We now have an audit trail to the needs/requirements of the second
round of soldier modernisation
18
Same thing on a diagram
Preparationmethods
Executionmethods
Assessmentmethods
Interrogation
Collationand collection
Reporting
Stage 1 Theory of blueactions
Stage 2 Prompts forexamination of detail
of theory
Stage 3 Interrogationof specified pairs oftheory component
and prompt
Stage 4 Analysis
Stage 5 Presentation ofresults
Prompt 1 Prompt 2 Prompt 3 Prompt 4
Th
eo
ryc
om
po
ne
nts
Planningmethods
Activities
Skills
19
Example results
Patrol plus Information Collection:- Issue - can’t see through walls- Ideas – talk to the building’s inhabitants, penetrating radar
Assault plus Engagement:- Issue – civilians present- Ideas – write rules of engagement, less than lethal weapons
Check point plus Protection:- Issue – soldiers vulnerable- Ideas – new procedures, remote checking
Tactical relocation plus Sustainment:- Issue – equipment weighs too much- Ideas – motorised forces, modular equipment
Note the sequence:- Is there a better procedure?- Can you improve things by a better organisational structure?- If all else fails, is there a technology fix?
KEY MESSAGES– the “bright ideas” now have an audit trail and have some weight as “answers” that decision makers will fund. If you group them up by core skill then you can start to say something justifiable about priority. In the longer term, the concept of operations will lead to a technical specification (eg what sort of walls do you want to look through?) The protocol also means that you have less chance of missing things, or being charged with being selective…
20
What did soldier modernisation teach us (4)? We need to think about measures better
Consider a hierarchy- Lowest level (engineering Measure of Performance)
eg ballistic dispersion (measurable)- Now do something with the kit (call this a Measure of Effectiveness)
eg kill probability (comprises measures of performance plus..)- Now think about combining some things to do a purposeful activity (call
this a Measure of Outcome) Eg was the assault successful? eg “This assault is a success if we
take the hill within a set time, use less than 40% of resources, lose less than 25% of our force and we can repel a counter attack of….” – combines measures of effectiveness plus…
- But the general doesn’t care about the ballistic dispersion (and it may not matter anyway), he cares about a Measure of Capability
Eg How good is the force equipped with the new kit? So who’s asking the question and what does the decision maker need
to know? - Intrinsically one can see how we’re going from a hard OR approach to a
soft OR one as we go down the page.
21
Example of Front-End studies for Counterterrorism
Context – the World has changed and high level planners need guidance about what to do in a new area
Context rephrased – decisions makers need to do something, but don’t know what (“a Mess”)
Outcome – some bright ideas on some initiatives that could be taken and some insights about the relative merits of options
Approach – try and understand the system, so that we know what matters, what is vulnerable, what can be fixed etc so that we have a framework for action
22
What did working on counterterrorism teach us (1)?Influence diagrams are models that help give a common understanding of the system, and are a source of options when we manipulate them
actionachievement
Australianpopulationterrorised
actioninitiation
opportunitiesfor terrorism
terrorismthreat
+
+
terroristcapability
HomelandDefence
capability
-
+
++
Governmentresolve
+/-
opposedGovernment
position+
terroristintent
+
+-
+
Intervention 1
Intervention 3
Intervention 2
23
What did working on counterterrorism teach us (2)? Look at both the specific and the generic
The specific gives us a sequence of events that we can examine to show vulnerabilities
The generic gives us insights of what things are important Influence diagrams are very useful
Event 1 Event 2 Event 3 Event 4
Capability
Lethality
Mobility
Survivability
Lower ility 1
Lower ility 2
24
A Specific Case – “the Terrorists’ Year”
Identify all the possible steps that red takes in planning, preparing, executing and exploiting a terrorism event
What can blue do for each of these?
Narrative goalof red
transformation
Impliedvulnerability
Attemptedblue
perturbation
What are theblue
problems?
Redtransformation
Collect up all the issues:
- gives a framework for a capability audit- gives an indication of degree of deficiency
- identifies changes that need to be made(of which some are S&T based)
- gives an indication of importance (somethings may appear many times)
NB high deficiency and high importance= high priorityNB hundreds of
lines of this!
Applicable to any “Purposeful Human Activity” System
25
What did working on counterterrorism teach us (3)?When we draw influence diagrams, nodal analysis is useful
variable
variable variable
Bottleneck
High pay-out High flux
Look for these as priority items:
This gives us some rules to follow when we manipulate the models(this is a step below systems dynamics)
26
Examples of Front end studies for the future Army
Context: there will be a future Army which almost certainly will not look like the current Army (“a Mess”)
Outcome: what advice can we give to the Army on future force structure development so that sound options are presented to the funding committees and good decisions are made?
Scientific challenges- How do we make sure that the contexts are properly considered?- How do you make sure that concepts are sound and well
explored?- How do you model the Army? – how do we define what an Army
does?- How do you include S&T in the models?- How do you look for “better things” rather than “same things better”
(ie replace what we’ve got)?
27
What did the future army work teach us (1)?This model shows that we should be advising on the development process not handing over a solution
ADFnow
IntermediateForce Option A
IntermediateForce Option B
IntermediateForce Option C
IntermediateForce Option D
Future environment 1
Future environment 2Future environment 3
ConceptualFuture Force A
ConceptualFuture Force E
ConceptualFuture Force D
ConceptualFuture Force C
ConceptualFuture Force B
Time
DecisionPoint
There is nosingle future!
So ID what you’d need for each future
But it helps ifyou can describe what we do now
And keep an eyeon key indicators
28
Practical implications
You never reach the future (the Army after Next) therefore it can’t be seen as a project to achieve
The future pulls you (“backcasts”) as the light on the horizon Some intermediate forces already exist:
- Legacy items- Things are that planned to be bought
New intermediate forces will be acquired as you understand the trajectory of change
You have to sell the notion of “the road to get there” (wherever there is) as the OR product, not a rigid project (a professional challenge)- “messes are managed not solved”
29
What did the future army work teach us (2)?Present in the real world, work in the model world
numerousChallenges
Current Force Structure(equipment, personnel
and organisations)
New Force StructuresModel #1 structure
Model #2the migration
Challenge1A -
expresshow Army
does things
Abstracted effects
Exploratoryensembles
Feasibility,applicability and
viability of concepts
Abstracted effects“optimised”
Challenge 2A - find away of exploring
things
test
reviseChallenge 2B -distill results
Challenge1B - find a
way ofachievingwhat you
want done
REAL WORLD
MODEL WORLD
Present results for the real world
Explore verbs and only go to nouns to see if your bright idea can be made to work. Work in
the soft OR area and then dive deep into specific
vignettes for learnings (could be hard OR) - complementarity
But do your workin the model world
30
What did the future Army work teach us (3)?When we model the core skill interactions we demonstrate how complex the Army is
Any change will perturb the system…
31
What did the future Army work teach us (4)?We can model the core skills by influence diagrams- but they can be hard to read!
higher level goal number of actionsconducted
efficiency of process
feed backfunction
activities requiring action
technology basedvariables
number of blueentities in system
(+/-)
(+)
(+)
(+)
(+/-)
number of redentities in system
(+/-)
(+)
(+)
inherent redcapability
(+)
(+/-)
environmental factors(+/-)
(+/-)
(+/-)
key technologyfeatures(+/-)
OTHER SKILLS
(+/-)
Eg Blue positioningUsage rate
Targetting capabilityWeapon capability
Eg RangeAccuracy
Terminal effect
This is one way of expressing a core skill
32
What did the future Army work teach us (5)?We can do things in a semi quantitative manner to assist our narrative arguments
Eg rate our current Technology Based Variables (TBV)- 3 = high capability, 2= medium, 1 = low- Give a weighting based on the nodal analysis and other reasons- Have a pay off matrix that links changes to other TBVs- Can derive an indicative “system value” in terms of blue capability or blue
minus red Then insert a change (eg technology)
- What happens to the indicative “system value”?- And why?
Example: hybrid engines:- Better Movement & Sustainment (and thus Information Collection) ie use
the weight saving in the form of extended range (option 1)- Or better Engagement and Protection ie replace fuel weight with
something else (option 2)- Option 1 looks better. “The raw scores for blue system value are 3.6 to 2.8
while blue minus red differentials are 4.5 to 2.9 – this is supported by the following narrative comment…..”
33
What did the future army work teach us (6)?morphological analysis can be used to develop options
Model Armies can be constructed from balances of the core skills: consider a set of 7 core skills – Engagement, Information collection, Communication, Mobility, Sustainment, Protection and Decision making (axes of interest)
Give each a rating of 1 (poor) to 3 (good) and come up with some feasible combinations of these
NB can’t all be good (expense), can’t all be poor (unpalatable), some aren’t possible (Field Anomaly Relaxation) so we’ll have a limited set. Which of these represent reasonable options?
ie “at the touch of a button” we can generate concepts (sum subscripts = 14)- Eg: E2I2C2M2S2P2D2 the all round force option, (baseline) - E3I2C2M3S1P1D2 the rapid moving strike force option, - E2I2C1M1S3P3D2 the attrition force option, or- E2I3C2M1S1P2D3 the information rich force option etc
Agent based models can be tied to the core skills:- Code in the characteristics learned from the core skills- In reverse, look at military history and express these in terms of core skills
Play them off (for combat…)
34
Results (MANA study)
Broad skill set
Engagement/Decision Making
Heavy
InformationCollection/
Movement Heavy
beats
beats
beats
Comments: in itself this is not a bad result:• We can reason why it occurs and learn something
• It tells us that there will be competing issues to consider
35
Data collection
First position – self (subjective)Second position – other (speculative)
Third position - fly on the wall (objective)
36
How do I know that my Front EndStudy is good?
If it was a “hard” study we’d get peer review to check the maths
“Soft” therefore we need to consider “fit for purpose” – or was “due analytical diligence” followed? - a code of best practice for soft OR
6 rights to consider:- Is the context right?- Do we understand the system right?- Can we explore changes to the system right?- Is this the right way to find options?- Are the measures right?- Is this the right way of doing the study?
37
Some comparisons of the Front-End to studies on more developed issues
The models can be deceptively simple (this is an art) We don’t try to capture everything (too hard or not possible) Everything can be challenged (show due diligence) Numbers are indicative not straitjackets (treat with caution as there are
probably subjective weightings, only some of the issue will be looked at it and we don’t understand it all anyway)
Nothing can be logically deduced (creativity) No front end study is right (but can be useful) Give the same job to two different people and you’ll get two different
approaches (but similar insights) Even though there’s no formal definition of a mathematical objective
function there will be something in the decision maker’s mind (help them)
Soft OR can be hard to do (the paradox)
38
The Front End as part of the DSTO/UNSW@ADFA MSc program
Compulsory courses (common to all DSTO MSc):- System Engineering- Research Methods
Compulsory foundation courses:- Operations Research- Problem Structuring- Quantitative OR
Foundation courses (2 of):- Analysis of military systems- Capability Options Analysis- Simulation- Strategic decision making- Supply chain science
Electives (1 of):- Any of the foundation courses- An agreed elective
Minor thesis
39
Acknowledgement – this presentation included material by some of my coworkers:
Jolanta Ciuk Peter Dortmans Wayne Hobbs Denis Shine Niem Tri
Much of this work is public domain
40
Adelaide
Questions?
