To Q or Not To Q: Is Simulation the Question? CMG 2011 Session 325 – Paper Number 1115 – Tuesday, Dec 6, 2011 Dr. Tim R. Norton SimSolTraining SM Training

To Q or Not To Q: Is Simulation the Question?

CMG 2011Session 325 – Paper Number 1115 – Tuesday, Dec 6, 2011

Dr. Tim R. Norton

SimSolTrainingSM Training division of Simalytic Solutions, LLC

719-635-5825

email: [email protected]

http://www.simalytic.com

© 2011 Simalytic Solutions, LLC To Q or Not To Q: Is Simulation the Question? 2

Agenda

What is Modeling? Brief look at Modeling Concepts Modeling Fundamentals

Practical Application of Modeling Real-world considerations


What is Modeling? A Model:

A schematic description of a system, theory, or phenomenon that accounts for its known or inferred properties and may be used for further study of its characteristics.

The American Heritage Dictionary, 3rd Edition, 1992

Modeling: To make or use a model.


What is Modeling? Modeling

A formula A picture An object A technique A useful representation


Classes of Modeling Physical

An object somewhat like the system Aerodynamics

Mathematical A formula representing system attributes

Ballistics

Process A representation of system interactions

Artificial environments Virtual physical models


Modeling Fundamentals What are the parts to Modeling?

Events or processes Queuing Distributions Modeling Techniques

Analytic Formulae Simulation – DES and Continuous Markov Chains Petri Nets Others


Modeling Fundamentals What are Events or Processes?

Events Something discrete – counted – beginning & end

Transactions, customers, cars, etc.

Processes Something that happens over time – measured

Bouncing ball, cloud formation, business workflow, etc.

Independent Model uses one or the other

Inter-Related Events trigger processes – processes trigger events


Modeling Fundamentals Queuing

Behavior where events or processes are waiting External – to get into the system Internal – at service points within the system

Major consideration in many models Slows down the system

Increases response time Reduces through-put


Modeling Fundamentals Service Centers and Queues

T = Total time at the Service Center W = Time waiting for service S = Time getting service

QueueArrivals Departures

ServiceCenter

W S

T

Queue Queue

Arrivals

Departures

ServiceCenter

ServiceCenter

Simple Service Center with Queue System of Simple Service Centers


Modeling Fundamentals Open Class

Workload Intensity – Arrival rate Unbounded customers

Closed Class Workload Intensity – Customer population Bounded customers

QueueArrivals Departures

ServiceCenter

Queue

ServiceCenter

Queue


Modeling Fundamentals Distribution Functions are patterns

Need to match model drivers Complex or simple Is close good enough?

Why Distributions Matter How to know the unknown future?

Uncertainty Variability Randomness


Modeling Fundamentals Modeling Techniques

Analytic Formulae Simulation

DES Continuous

Markov Chains Petri Nets Others

S

ST

1


Modeling Fundamentals Markov Chains

Foundation of most modeling Finite state-space Probabilities for state transitions

0 1 2 3 k

0 1 2 3 k-1 k

0 1 2 3 k k+1


Modeling Fundamentals Petri Nets

Places, transitions, arcs and tokens Tokens move from place to pace Transitions

control movement Rules to move


Modeling Fundamentals Analytic Formula

Queuing Theory Queuing Networks

Example Calculate how long it takes a customer to get their

fast food T is the average total time per customer S is the time at the counter (lambda) is the number of customers that come into

the restaurant per minute


Modeling Fundamentals Morning vs. Lunch?

Is another register needed? Does single server scenario exceed expectation?

Time to serve a customer, S, is the same 3 minutes for both cases

Customer arrivals, (lambda), changes Morning: 0.1 per minute (1 every 10 minutes) Lunch: 0.3 per minute (~1 every 3 minutes)

Calculate T for each case: Morning: T = 4.286 minutes Lunch: T = 30 minutes

S

ST

1


Modeling Fundamentals Why so long?

Customers show up slower that it takes to get their order.

The formulae assumes a non-uniform distribution Distributions are the key

Most activities involving people are Poisson arrivals The system is just as busy in either case

Simulation chart


Modeling Fundamentals What is “Poisson Arrivals”?

The inter-arrival times are non-uniform random numbers from an exponential distribution.

Arrival # Interarrival time1 0.62 1.63 2.84 0.05 1.16 2.37 8.68 3.39 6.6

10 0.611 4.112 4.913 2.314 1.0Generated using EasyFitXL

15 3.216 11.417 3.118 1.019 0.720 0.321 3.922 3.023 0.124 8.025 2.126 1.327 4.628 7.729 0.430 3.1


Modeling Fundamentals Other Distributions

Useful for specific situations Normal – testing Erlang – telephony Pareto (heavy-tail) – network traffic Many, many others

Normal Pareto Triangular

Exponential

Erlang


Modeling Fundamentals Queuing Theory Formulae

Built-in assumptions about the arrival and service distributions

Kendall notation: A/B/C/K/N/D – often shortened to A/B/C

A = Arrival distribution B = Service distribution C = Number of servers K = Number of places in the system (capacity); defaults to N = Source population size; defaults to D = Queuing discipline; defaults to FIFO


Modeling Fundamentals Common Distributions for A & B

M – Markovian (Poisson/Exponential) D – Degenerate (deterministic or fixed) Ek – Erlang (k = the shape parameter) G – General Some others


Modeling Fundamentals M/M/1 Model

Poisson arrivals Exponential service time One server Infinite capacity in the system Infinite number of potential customers FIFO priority


Modeling Fundamentals Simulation

DES – Discrete Event Simulation Models that evolve over time where system state

variables change instantaneously at separate (countable) points in time.

Continuous Models that evolve over time where the values of

system state variables can be calculated for any arbitrary point in time.


Modeling Fundamentals

DES Example Calculate how long it takes customers to get

their fast food at lunch S is the time at the counter

3 minutes Customer arrivals

Exponential inter-arrival time distribution (shown in table from earlier example)

Arrival # Interarrival time1 0.62 1.63 2.84 0.05 1.16 2.37 8.68 3.39 6.6

10 0.611 4.112 4.913 2.314 1.0

15 3.216 11.417 3.118 1.019 0.720 0.321 3.922 3.023 0.124 8.025 2.126 1.327 4.628 7.729 0.430 3.1


Modeling Fundamentals DES Example

Step through each arrival

Increment clockfor each event

Additional system state counters

Customers Waiting Waiting time etc.

Analytic chart


Modeling Fundamentals Continuous Example

Uses formulaeto calculate positionfor each plot point

Gravity: acceleration down

Elasticity: acceleration up

Energy is absorbed etc.

Use SlideShow mode to see active graph.


Modeling Fundamentals

Business Modeling Continuous Simulation Models Business elements

Process Flowsand Levels

Business Modeling Tools

Powersim, Vensim, STELLA and ithink

Call_Backlog

Transactions_per_Call

Computer_Time

Number_of_Operators

Transaction_Setup_Time

Calls_Completed

Transaction_Response_Time

New_Calls

Calls_per_Operator

Other_Time


Practical Application

Real-world considerations Modeling Often Not Well Received

Large effort for unwanted results? Modeling takes too long? Doesn’t provide anything useful? Looking for the ‘right’ modeling tool?

Modeling Problem Business problem Not technical problem


Practical Application

Common sense axioms Modeling technique Target system Business problem

Practical techniques

“The right answer and it’s provable.” vs.

“A reasonable answer and it’s useful.”


“The Problem”

How to identify the Desired Result and differentiate it from the Requested Result? Identify the Recipient Understand their Goals

“I know that your believe that you understand what you think I said, BUT I am not sure your realize that what your heard is not what I meant!” Anonymous (on a postcard), © 1982 H. a L. Inc.


Know Your Goals

Benefit of Understanding Goals “If you don't care where you’re going, then any

road will do.” Example of Goal Impacting Solution

Taking a Trip


The Trip

Planning a Trip to New York City What are the goals? Who are the players? What are the constraints?


The Trip - Why

What is the Reason for the Trip? Business Pleasure Family emergency Have to leave town


The Trip - Who

Who Else is Taking the Trip? Person alone Boss Co-worker Couple Family Group


The Trip - How

How many modes of transport are there to get from Washington DC to New York City Car, train, bus, plane, boat, bike, walk, others... Which one to pick depends on the objectives of

the trip: Get there fast, see the sites, relax and enjoy, get there

cheap, safety first, visit someone on the way, hidden agenda.

Each “Why” maps to one or more “How”


The Trip - Summary

What makes a Good Plan? Hard to judge without fully understanding the goals Approach:

Who & Why Then How

Is the Desired Result different from the Requested Result? Hidden agenda?


Modeling Objectives

Objective Find the successful middle ground

Cost Performance

Performance Assessed against business objectives and goals

Projected business volume and functionality Model to predict the likely performance Meet business goals at future volumes


Problem Identification

Identify Key Players and Objectives Focus on the right problem

Approach: Establish a Process

Informal for own use Formal for project results Standardized for certification

Standardize the Deliverables Quantifies time and effort required


Where to Focus

Where to Focus the Modeling Effort? Find the Answer

vs. Justify the Solution


Required Solution

What is the Required Solution? Scope

The extent it applies to one or more domains Granularity

The level of detail to be included Scale

The relative time scale to use Measurement

The technique to determine success


Solution Scope

Technical Identify the best technology to solve the problem

CPU X runs workload 20% faster than CPU Y

Political Support the solution implemented for “business”

reasons Will CPU Y will provide the workload acceptable

resources for the rest of the year?


Solution Granularity

High-level Generalized projections of growth

Projected growth requires system hardware increase of 30% next year

Detailed Specific requirements

Sys A needs an additional CPU and Sys B needs 32MB


Solution Scale

Resource Usage Average consumption over time

Average CPU usage remains below 80%

Bottleneck Identification Transient conditions that impact response time

The “Waiting for CPU” queue goes over 50 every day between 2 PM and 3 PM


Solution Measurement

Service Level Objectives Objective measurement against business needs

Workload X response time is staying below the goal of 95% in 0.8 seconds

“User Happiness” Subjective measurement

“I just got a call from an unhappy VP in Finance!”


Solution Focus

Balance Requirements vs. Solutions Requested Result vs. Desired Result Perceptions vs. Reality

Approach: Identify applicable techniques Apply the correct tools How to present the results


Identify Requirements

Audience: Requester or messenger or both Multiple agendas

Output: Report White paper Presentation


Spiral Process

Plan How to meet the objectives with least effort

Develop Generate some results

Evaluate How well does output meet the objectives

Present Get feedback to revise plan


Spiral Model Development

The Spiral Modeling Methodology was derived from the Spiral Software Development Methodology first discussed by Barry W. Boehm in “A Spiral Model of Software Development and Enhancement”, IEEE Computer, May 1988.

Plan

Develop Evaluate

Present

TrendVerify

SWAG

Model ValidateCalibrate

ObjectivesVerify

Validate

Model

Confirm

Informal Formal

Results

Objectives

Feedback:Re-evaluate

and ReviseWhat

How

Tool Assessment

Revised Requirements

Initial Requirements


Spiral Approach Benefits Objective

Focus on Requester Reduce

Time Effort

Responsiveness Perceived Improvement Adjust to changes

Requirements or objectives


Increasing Complexity

Start Simple and Work Up Guess

Almost no data but a reasonable conjecture Experience

Little data and a familiar situation Trends

Historical data and statistical analysis Models

Lots of data and a predictive technique


SWAG

“Scientific Wildly Analyzed Guess” What Effects the End Result:

What’s of interest? (CPU, Disk, etc.) Assume it’s zero Does the response time change significantly? Only worry about things that make a difference


SWAG Examples

Where is the Major Delay? Only B worth doing a

CPU upgrade Faster Disk helps A

and maybe C High Wait in C may

be application problem Server resources

won’t help D

0 1 2

D

C

B

A

CPU I/O Wait

I/O Network


Trend Analysis

Patterns in Historical Data Assumes future is an extension of past The more historical data the better

Works Well: Mature and stable environments Accommodates seasonal and regular anomalies

Fails: Sudden growth or erratic environments


LFFT

“Linear Function Fit Tool” Devised by Ray Wicks (IBM)

Simple Trend Analysis Tool Plot the data Pick up an LFFT (a pencil works well) Lay it on the plot Find the last section of data it fits Extend that section


LFFT Examples

Four Workloads Uniform growth Trend fits all

months well Good “feel” about

predicting next quarter

0

20

40

60

80

100

120

140

160

Ja

n

Fe

b

Ma

r

Ap

r

Ma

y

Ju

n

Ju

y

Au

g

Sep

ABCD

Trend


LFFT Examples

Same Four Workloads Workload D jumps

by 40 in April Really two different

the same rate Anomaly not growth Next quarter prediction

better with trend of last group

020406080

100120140160180200220

Ja

n

Fe

b

Ma

r

Ap

r

Ma

y

Ju

n

Ju

l

Au

g

Sep

ABCD

Real Trend

Overall Trend


Simple Models

What is the Minimum Model? Before You Use a Tool

Collect data at several workload rates Find relationships:

Response time per transaction? Resource usage per transaction? Workload to overall system resource usage? Any others that apply to application?

Project it out and talk to the Requester Understand what type of tool is needed


Conclusion

Many Modeling Techniques Analytic Simulation

DES or Continuous Specialized

Petri Nets, Markov Chains, Business Process, etc.

Business Problem Understand the requirements Focus on the requestor’s goals

To Q or Not To Q: Is not the Question,

but ratherWhat’s the best way to a useful

answer?

Presentation will be available at: http://www.simalytic.com/Papers.html

Documents

To Q or Not To Q: Is Simulation the Question? CMG 2011 Session 325 – Paper Number 1115 – Tuesday, Dec 6, 2011 Dr. Tim R. Norton SimSolTraining SM Training