Simulation optimization: A review of algorithms and ... · Simulation optimization: A review of ... excellent tutorial on simulation optimization, and ... and also talk about estimation

arX

iv:1

706.

0859

1v1

[cs

.DS]

26

Jun

2017

Simulation optimization: A review ofalgorithms and applications

Satyajith Amaran1,2, Nikolaos V. Sahinidis1, Bikram Sharda3, and Scott J.Bury4

1Department of Chemical Engineering, Carnegie Mellon University,

Pittsburgh, PA 15213, USA2Now at Core R&D, The Dow Chemical Company, Freeport, TX 77541,

USA3Core R&D, The Dow Chemical Company, Freeport, TX 77541, USA4Core R&D, The Dow Chemical Company, Midland, MI 48667, USA

Abstract

Simulation Optimization (SO) refers to the optimization of an objective function subject to constraints,both of which can be evaluated through a stochastic simulation. To address specific features of a particularsimulation—discrete or continuous decisions, expensive or cheap simulations, single or multiple outputs,homogeneous or heterogeneous noise—various algorithms have been proposed in the literature. As one canimagine, there exist several competing algorithms for each of these classes of problems. This documentemphasizes the difficulties in simulation optimization as compared to mathematical programming, makesreference to state-of-the-art algorithms in the field, examines and contrasts the different approaches used,reviews some of the diverse applications that have been tackled by these methods, and speculates on futuredirections in the field.Keywords: Simulation optimization; Optimization via simulation; Derivative-free optimization

1 Introduction

Advances in modeling and availability of cheapcomputational power have enabled the science,engineering, and business research communities tomake use of simulations to model phenomena andsystems. It is only natural that there be a greatinterest in manipulating degrees of freedom in thesimulations to optimize them.

The term Simulation Optimization (SO) is anumbrella term for techniques used to optimizestochastic simulations. Simulation optimizationinvolves the search for those specific settings of theinput parameters to a stochastic simulation suchthat a target objective, which is a function of thesimulation output, is, without loss of generality,

minimized.As opposed to mathematical programming, SO

does not assume that an algebraic description ofthe simulation is available—the simulation maybe available as a black box that only allows theevaluation of the objective and constraints for aparticular input. In fact, many SO algorithmicapproaches solely depend on such input-output datafrom the simulation in their search for optimal inputsettings.

In addition, many large-scale and/or detailedsimulations may be expensive to run, in terms oftime, money, or resources. As a result, thereis also a need to perform few simulations in thissearch for optimal parameters. Outputs from thesestochastic simulations are not deterministic, and

©2017. This manuscript version is made available under the CC-BY-NC-ND 4.0 license

http://creativecommons.org/licenses/by-nc-nd/4.0/. The formal publication of this article

is available at https://link.springer.com/article/10.1007/s10479-015-2019-x.

http://arxiv.org/abs/1706.08591v1

http://creativecommons.org/licenses/by-nc-nd/4.0/

https://link.springer.com/article/10.1007/s10479-015-2019-x

Simulation optimization: A review of algorithms and applications 2

usually follow some output distribution, which mayor may not vary across the parametric space. Thisuncertainty or variability in output also adds tothe challenge of optimization, as it becomes harderto discern the quality of the parametric input inthe presence of this output noise. In addition,when an algebraic description of the simulationis not accessible, derivative information is usuallyunavailable, and the estimation of derivatives fromthe use of finite differences may not be suitabledue to noisy outputs and the expensive nature ofsimulations.

The nature of the stochastic simulations understudy will determine the specific technique chosento optimize them. The simulations, which areoften discrete-event simulations, may be partiallyaccessible to us in algebraic form, or may be purelyavailable as an input-output model (as a blackbox); they may have single or multiple outputs;they may have deterministic or stochastic output(s);they may involve discrete or continuous parameters;and they may or may not involve explicit, or evenimplicit/hidden constraints.

A very general simulation optimization problemcan be represented by (P1).

min Eω[f(x, y, ω)]

s.t. Eω[g(x, y, ω)] ≤ 0

h(x, y) ≤ 0

xl ≤ x ≤ xu

x ∈ Rn, y ∈ D

m.

(P1)

The function f can be evaluated through simulationfor a particular instance of the continuous inputs x,discrete inputs y, and a realization of the randomvariables in the simulation, the vector ω (which mayor may not be a function of the inputs, x and y).Similarly, the constraints defined by the vector-valuedfunction g are also evaluated with each simulationrun. In this formulation, expected values for thesestochastic functions are used. There may be otherconstraints (represented by h) that do not involverandom variables, as well as bound constraints onthe decision variables.

The relaxation of any of these conditions wouldconstitute a problem that would fall under thepurview of SO. Most algorithms focus on problemsthat either have solely discrete choices, or solelycontinuous decisions to make. Each constraint maybe thought of as representing additional outputsfrom the simulation that need to be taken intoconsideration. In addition, there may be boundconstraints imposed on decision variables, that mayeither be available or obtained from domain-specific

knowledge. Relatively few existing algorithmsattempt to address both discrete and continuouschoices simultaneously, although some broad classesof approaches naturally lend themselves to beapplicable in either, and therefore both, settings.Further, the discrete variables may either be binary,integer-ordered, or categorical and lie in some discretespace D.

As can be seen, the formulation P1 is extremelygeneral, and therefore a wide variety of applicationsfall under the scope of simulation optimization.Various applications of simulation optimization indiverse research fields are tabulated in Section 2.

Another common assumption is that f is areal-valued function and g is a real vector-valuedfunction, both of whose expected values may ormay not be smooth or continuous functions. Themost common objective in SO is to optimize theexpected value of some performance metric, but otherobjective functions may be appropriate dependingon the application. For instance, an objectivethat minimizes risk could be a possible alternative,in which case one would incorporate some sort ofvariance measure as well into the objective.

This paper is an updated version of Amaran et al.[4] and is meant to be a survey of available techniquesas well as recent advances in simulation optimization.The remainder of the introduction section provides aliterature survey of prior reviews, and elaborates onthe relationship of simulation optimization to math-ematical programming, derivative-free optimization,and machine learning. Section 2 provides a glimpseinto the wide variety of applications of simulationoptimization that have appeared in the literature.Section 3 focuses on various algorithms for discreteand continuous simulation optimization, providesbasic pseudocode for major categories of algorithms,and provides comprehensive references for each typeof algorithm. Section 4 provides a listing of availablesoftware for simulation optimization and Section5 discusses means to compare their performance.Section 6 summarizes the progress of the field, andoutlines some current and future topics for research.

1.1 Prior reviews of simulation optimization

Several review papers (cf. [135, 99, 174, 15, 59, 29,7, 16, 191, 61, 60, 194, 62, 9, 88, 5, 152]), books andresearch monographs (cf. [186, 171, 115, 35]), andtheses (cf. [12, 53, 45, 30, 58, 104]) have traced thedevelopment of simulation optimization.

Meketon [135] provides a classification of algo-rithmic approaches for optimization over simulationsbased on how much information or structure about

Amaran, Sahinidis, Sharda and Bury 3

the underlying model is known. The paper surveysthe progress of the field between 1975 and 1987,and focuses on continuous simulation optimization.Andradottir [7] provides a tutorial on gradient-basedprocedures for continuous problems. Carson andMaria [29] and Azadivar [16] also give brief outlinesof and pointers to prevailing simulation optimizationalgorithms.

Fu et al. [61] contains several positionstatements of eminent researchers and practitionersin the field of simulation, where the integrationof simulation with optimization is discussed. Theissues addressed include generality vs. specificityof an algorithm, the wider scope of problemsthat simulation optimization methodologies have thepotential to address, and the need for integratingprovably convergent algorithms proposed by theresearch community with metaheuristics often usedby commercial simulation software packages.

Of the more recent surveys, Fu [59] provides anexcellent tutorial on simulation optimization, andfocuses on continuous optimization problems morethan discrete optimization problems. The paperfocuses specifically on discrete-event simulations.Fu [60] provides a comprehensive survey of thefield and its scope—the paper outlines the differentways in which optimization and simulation interact,gives examples of real-world applications, introducessimulation software and the optimization routinesthat each of them use, provides a very basic tutorialon simulation output analysis and convergence theoryfor simulation optimization, elaborates on algorithmsfor both continuous and discrete problems, andprovides pointers to many useful sources. Fu etal. [62] provide a concise, updated version of all ofthis, and also talk about estimation of distributionalgorithms.

Tekin and Sabuncuoglu [194] provide a table thatanalyzes past review papers and the techniques theyfocus on. Apart from providing detailed updates onadvances in approaches and algorithms, the paperalso lists references that attempt to compare differentSO techniques. Hong and Nelson [88] classifysimulation optimization problems into those with (1)a finite number of solutions; (2) continuous decisionvariables; and (3) discrete variables that are integer-ordered. The paper describes procedures for eachof these classes. Perhaps the most recent survey,[5], classifies simulation optimization algorithms andprovides a survey of methods as well as applicationsappearing in the literature between 1995 and 2010.

The present paper provides an overview oftechniques, and briefly outlines well-establishedmethods with pointers to more detailed surveys,

while expounding on more recent methods ina concise manner. Though several reviewsexist, we catalog the most recent developments—the emergence of derivative-free optimization andits relationship with simulation optimization, theappearance of simulation test-beds for comparingalgorithms, the recent application of simulationoptimization in diverse fields, the development ofand interest in related techniques and theory by themachine learning community and the optimizationcommunity, as well as the sheer unprecedented natureof recent interest in optimizing over simulations. Areflection of a surge in recent interest is evidencedby the fact that more than half of the works wereference were published in the last decade. Theintent is to not only trace the progress of the field,but to provide an update on state-of-the-art methodsand implementations, point the familiar as well asthe uninitiated reader to relevant sources in theliterature, and to speculate on future directions inthe field.

1.2 A note on terminology and scope

As simulation optimization involves the use ofalgorithms that arose from widely differing fields(Section 3), has relationships to many diversedisciplines (Section 1.3), and has been applied tomany different practical applications from biologyto engineering to logistics (Section 2), it is notsurprising that it is known by various namesin different fields. It has also been referredto as simulation-based optimization, stochasticoptimization, parametric optimization, black-boxoptimization, and Optimization via Simulation(OvS), where the continuous and discrete versionsare accordingly known as Continuous Optimizationvia Simulation (COvS) and Discrete Optimizationvia Simulation (DOvS). Each algorithmic techniquemay also go by different names, and we attempt toreconcile these in Section 3.

Inputs to the simulation may be variouslyreferred to as parameter settings, inputsettings, variables, controls, solutions, designs,experiments (or experimental designs), factors,or configurations. Outputs from the simulationare called measurements, responses, performancemetrics, objective values, simulation replications,realizations, or results. The performance of asimulation may also be referred to as an experiment,an objective function evaluation, or simply afunction evaluation. We will use the term ‘iteration’to refer to a fixed number of function evaluations(usually one) performed by a simulation optimization


algorithm.

A note of caution while using SO methods is toincorporate as much domain specific knowledge aspossible in the use of an SO algorithm. This maybe in terms of (1) screening relevant input variables,(2) scaling and range reduction of decision variables,(3) providing good initial guesses for the algorithm;and (4) gleaning information from known problemstructure, such as derivative estimates.

Table 1 classifies the techniques that are usuallymost suitable in practice for different scenarios inthe universe of optimization problems. Certainbroad classes of algorithms, such as random searchmethods, may be applicable to all of these typesof problems, but they are often most suitable whendealing with pathological problems (e.g., problemswith discontinuities, nonsmoothness) and are oftenused because they are relatively easy to implement.

The possibilities of combining simulation andoptimization procedures are vast: simulation withoptimization-based iterations; optimization withsimulation-based iterations; sequential simulationand optimization; and alternate simulation andoptimization are four such paradigms. A recentpaper by Figueira and Almada-Lobo [56] delvesinto the taxonomy of such problems, and providesa guide to choosing an appropriate approach fora given problem. As detailed by Meketon [135],different techniques may be applicable or moresuitable depending on how much is known aboutthe underlying simulation, such as its structure orassociated probability distributions. We focus onapproaches that are applicable in situations whereall the optimization scheme has to work with areevaluations of f(x, y, ω) and g(x, y, ω), or simply,observations with noise.

1.3 Relationship to other fields

Mathematical Programming. As men-tioned earlier, most mathematical programmingmethods rely on the presence of an algebraic model.The availability of an algebraic model has manyobvious implications to a mathematical programmingexpert, including the ability to evaluate a functionquickly, the availability of derivative information, andthe possibility of formulating a dual problem. None ofthese may be possible to do/obtain in an SO setting.

In the case with continuous decisions, derivativeinformation is often hard to estimate accuratelythrough finite differences, either due to the stochasticnoise associated with objective function evaluations,

or due to the large expense associated with obtainingfunction evaluations, or both. The inherentstochasticity in output also renders automaticdifferentiation (AD) [161, 76] tools not directlyapplicable. Moreover, automatic differentiationmay not be used when one has no access tosource code, does not possess an AD interface toproprietary simulation software, and, of course, whenone is dealing with a physical experiment. Thelack of availability of derivative information hasfurther implications—it complicates the search fordescent directions, proofs of convergence, and thecharacterization of optimal points.

Simulation optimization, like stochastic program-ming, also attempts to optimize under uncertainty.However, stochastic programming differs in thatit makes heavy use of the model structure itself[27]. Optimization under uncertainty techniques thatmake heavy use of mathematical programming arereviewed in [175].

Derivative-Free Optimization. BothSimulation Optimization and Derivative-FreeOptimization (DFO) are referred to in the literatureas black-box optimization methods. Outputvariability is the key factor that distinguishes SOfrom DFO, where the output from the simulation isdeterministic. However, there are many approachesto DFO that have analogs in SO as well (e.g., responsesurfaces, direct search methods, metaheuristics), cf.Section 3.

Another distinction is that most algorithms inDFO are specifically designed keeping in mind thatfunction evaluations or simulations are expensive.This is not necessarily the case with SO algorithms.

With regard to rates of convergence, SOalgorithms are generally inefficient and convergencerates are typically very slow. In general, one wouldexpect SO to have a slower convergence rate thanDFO algorithms simply because of the additionalcomplication of uncertainty in function evaluations.As explained in [43], some DFO algorithms, undercertain assumptions, expect rates that are closerto linear than quadratic, and therefore earlytermination may be suitable. As described in somedetail by [59], the best possible convergence ratesfor SO algorithms are generally O(1/

√k), where

k is the number of samples. This is true from thecentral limit theorem that tells us the rate at whichthe best possible estimator converges to the trueexpected function value at a point. This impliesthat though one would ideally incorporate rigoroustermination criteria in algorithm implementations,


Table 1: Terminology of optimization problems

Algebraic model available Unknown/complexproblemstructure

Deterministic Traditional math program-ming (linear, integer, andnonlinear programming)

Derivative-freeoptimization

Uncertainty present Stochastic programming, ro-bust optimization

Simulationoptimization

most practical applications have a fixed simulationor function evaluation budget that is reached first.

Machine Learning. Several subcommunitiesin the machine learning community address problemsclosely related to simulation optimization. Tradi-tional machine learning settings assume the avail-ability of a fixed dataset. Active learning methods[40, 178] extend machine learning algorithms to thecase where the algorithms are allowed to query anoracle for additional data to infer better statisticalmodels. Active learning is closely related in thatthis choice of sampling occurs at every iteration ina simulation optimization setting as well. The focusof active learning is usually to learn better predictivemodels rather than to perform optimization.

Reinforcement learning [190] is broadly concernedwith what set of actions to take in an environment tomaximize some notion of cumulative reward. Rein-forcement learning methods have strong connectionsto information theory, optimal control, and statistics.The similarity with simulation optimization is thatthe common problem of exploration of the searchspace vs. exploitation of known structure of thecost function arises. However, in the reinforcementlearning setting, each action usually also incurs acost, and the task is to maximize the accumulatedrewards from all actions—as opposed to finding agood point in the parameter space eventually.

Policy gradient methods [155] are a subfield ofreinforcement learning, where the set of all possiblesequences of actions form the policy space, and agradient in this policy space is estimated and agradient ascent-type method is then used to moveto a local optimum. Bandit optimization [68]is another subfield of reinforcement learning thatinvolves methods for the solution to the multi-armedbandit problem. The canonical example involves a

certain number of slot machines, and a certain totalbudget to play them. Here, each choice of samplecorresponds to which slot machine to play. Eachplay on a slot machine results in random winnings.This setting is analogous to discrete simulationoptimization (DOvS) over finite sets, although witha different objective [158]. Again, in DOvS overfinite sets, we are only concerned with finding thebest alternative eventually, whereas the cumulativewinnings is the concern in the multi-armed banditproblem.

Relationship to other fields. Most, ifnot all, simulation optimization procedures haveelements that are derived from or highly related toseveral other fields. Direct search procedures andresponse surface methodologies (RSM) have strongrelationships with the field of experimental design.RSM, sample path optimization procedures, andgradient-based methods heavily incorporate ideasfrom mathematical programming. RSM also involvesthe use of nonparametric and Bayesian regressiontechniques, whereas estimation of distributionalgorithms involves probabilistic inference, andtherefore these techniques are related to statisticsand machine learning. Simulation optimizationhas been described as being part of a larger fieldcalled computational stochastic optimization. Moreinformation is available at [157].

2 Applications

SO techniques are most commonly applied to either(1) discrete-event simulations, or (2) systems ofstochastic nonlinear and/or differential equations.

As mentioned in [59], discrete event simulationscan be used to model many real-world systems suchas queues, operations, and networks. Here, the


simulation of a system usually involves switching orjumping from one state to another at discrete pointsin time as events occur. The occurrence of events ismodeled using probability distributions to model therandomness involved.

Stochastic differential equations may be used tomodel phenomena ranging from financial risk [137]to the control of nonlinear systems [183] to theelectrophoretic separation of DNA molecules [39].

With both discrete-event simulations and stochas-tic differential equation systems, there may beseveral parameters that one controls that affectsome performance measure of the system underconsideration, which are essentially degrees offreedom that may be optimized through SOtechniques. Several applications of SO from diverseareas have been addressed in the literature and welist some of them in Table 2.

3 Algorithms

Algorithms for SO are diverse, and their applicabilitymay be highly dependent on the particular applica-tion. For instance, algorithms may (1) attempt tofind local or global solutions; (2) address discreteor continuous variables; (3) incorporate randomelements or not; (4) be tailored for cases wherefunction evaluations are expensive; (5) emphasizeexploration or exploitation to different extents; (6)assume that the uncertainty in simulation outputis homoscedastic or that it comes from a certainprobability distribution; or (7) rely on underlyingcontinuity or differentiability of the expectation (orsome function of a chosen moment) of the simulationoutput. The sheer diversity of these algorithmsalso makes it somewhat difficult to assert which oneis better than another in general, and also makesit hard to compare between algorithms or theirimplementations.

As mentioned in Section 1.3, many algorithms thatare available for continuous simulation optimizationhave analogs in derivative-based optimization andin derivative-free optimization, where functionevaluations are deterministic. In any case, the keylies in the statistics of how noise is handled, andhow it is integrated into the optimization scheme.We will provide pointers to references that areapplicable to simulation optimization in particular.A comprehensive review of methods for derivative-free optimization is available in [166].

Each major subsection below is accompanied bypseudocode to give researchers and practitionersunfamiliar with the field an idea of the general

approach taken by each of these algorithms. Manyof the sections include pointers to convergenceproofs for individual algorithms. Optimalityin simulation optimization is harder to establishthan in mathematical programming or derivative-free optimization due to the presence of outputvariability. Notions of optimality for simulationoptimization are explored in [59]; for the discretecase, [206], for instance, establishes conditions forlocal convergence, where a point being ‘better’than its 2m + 1 neighboring solutions is said tobe locally optimal. There has also been somework in establishing Karush-Kuhn-Tucker (KKT)optimality conditions for multiresponse simulationoptimization [24]. Globally convergent algorithmswill locate the global optimal solution eventually, butassuring this would require all feasible solutions to beevaluated through infinite observations; in practice,a convergence property that translates to a practicalstopping criterion may make more sense [88].

Based on their scope, the broad classes ofalgorithms are classified in Table 3. Algorithmsare classified based on whether they are applicableto problems with discrete/continuous variables, andwhether they focus on global or local optimization.However, there may be specific algorithms that havebeen tweaked to make them applicable to a differentclass as well, which may not be captured by this table.

3.1 Discrete optimization via simulation

Discrete optimization via simulation is concernedwith finding optimal settings for variables that canonly take discrete values. This may be in the formof integer-ordered variables or categorical variables[153]. Integer-ordered variables are allowed to takeon integer or discrete values within a finite interval,where the order of these values translates to somephysical interpretation. For example, this could bethe number of trucks available for vehicle routing,or the set of standard pipe diameters that areavailable for the construction of a manufacturingplant. Categorical variables refer to more generalkinds of discrete decisions, ranging from conventionalon-off (0-1 or binary) variables to more abstractdecisions such as the sequence of actions to takegiven a finite set of actions. It should be notedthat though integer-ordered variables, for instance,may be logically represented using binary variables,it may be beneficial to retain them as integer-orderedto exploit correlations in objective function valuesbetween adjacent integer values.

A rich literature in DOvS has developed overthe last 50 years, and the specific methods


Table 2: Partial list of published works that apply simulation optimization

Domain of application Application and citations

Operations Buffer location [132], nurse scheduling [193], inventorymanagement [120, 176], health care [11], queuingnetworks [63, 25, 138]

Manufacturing PCB production [48], engine manufacturing [192],production planning [107, 114], manufacturing-celldesign [98], kanban sizing [79]

Medicine and biology Protein engineering [168], cardiovascular surgery [202],breast cancer epidemiology [55], bioprocess control [197,165], ECG analysis [67], medical image analysis [136]

Engineering Welded beam design [209], solid waste management[210], pollution source identification [14], chemical supplychains [103], antenna design [159], aerodynamic design[203, 205, 204, 122], distillation column optimization[162], well placement [18], servo system control [160],power systems [54], radar analysis [108]

Computer science, networks,electronics

Server assignment [125], wireless sensor networks [49],circuit design [130], network reliability [123]

Transportation and logistics Traffic control and simulation [211, 17, 151],metro/transit travel times [83, 148], air traffic control[119, 96]

Table 3: Classification of simulation optimization algorithms

Algorithm class Discrete Continuous Local Global

Ranking and selection × ×Metaheuristics × × ×Response surface methodology × × ×Gradient-based methods × ×Direct search × × ×Model-based methods × × × ×Lipschitzian optimization × ×


developed are tailored to the specific problem setting.Broadly, methods are tailored for finite or for verylarge/potentially infinite parameter spaces.

3.1.1 Finite parameter spaces

In the finite case, where the number of alternatives issmall and fixed, the primary goal is to decide how toallocate the simulation runs among the alternatives.In this setting, there is no emphasis on ‘search’,as the candidate solution pool is small and known;each iteration is used to infer the best, in somestatistical sense, simulation run(s) to be performedsubsequently.

The optimization that is desired may differdepending on the situation, and could involve:

1. The selection of the best candidate solutionfrom a finite set of alternatives;

2. The comparison of simulation performancemeasures of each alternative to a knownstandard or control; or

3. The pairwise comparison between all solutioncandidates.

Item (1) is referred to as the ranking and selection

problem. Items (2) and (3) are addressed underliterature on multiple comparison procedures, withthe former referred to as multiple comparisons with

a control.

Ranking and Selection. In traditionalranking and selection, the task is to minimize thenumber of simulation replications while ensuring acertain probability of correct selection of alternatives.Most procedures try to guarantee that the designultimately selected is better than all competingalternatives by δ with a probability at least 1 −α. δ is called the indifference zone, and is thevalue deemed to be sufficient to distinguish betweenexpected performance among solution candidates.

Conventional procedures make use of the Bon-ferroni inequality which relates probabilities of theoccurrence of multiple events with probabilitiesof each event. Other approaches involve theincorporation of covariance induced by, for example,the use of common random numbers to expedite thealgorithmic performance over the more conservativeBonferroni approach. Kim and Nelson [111, 112]and Chick [38] provide a detailed review andprovide algorithms and procedures for this setting.Extensions of fully sequential ranking and selectionprocedures to the constrained case have beenexplored as well, e.g., [10].

An alternative formulation of the ranking andselection of the problem would be to try to do thebest within a specified computational budget, calledthe optimal computing budget allocation formulation[33]. Chen et al. [36] present more recent work, whilethe stochastically constrained case is considered in[129].

Recent work [95] in the area of DOvS over finitesets provides a quick overview of the field of rankingand selection, and considers general probability dis-tributions and the presence of stochastic constraintssimultaneously.

A basic ranking and selection procedure [112] isoutlined in Algorithm 1, where it is assumed thatindependent data comes from normal distributionswith unknown, different variances.

Algorithm 1 Basic ranking and selectionprocedure for SO

Require: Confidence level 1−α, indifferencezone parameter δ

1: Take n0 samples from each of the 1, . . . , Kpotential designs

2: Compute sample means, tk,n0and sample

variances, Sk, for each of the designs3: Determine how many new samples, Nk :=

max{n0,

⌈ψ2S2

k

δ2

⌉}, to take from each

system, where the Rinott constant ψ isobtained from [21]

4: Select the system with the best newsample mean, tk,Nk+n0

.

Multiple comparison procedures. Here,a number of simulation replications are performed onall the potential designs, and conclusions are made byconstructing confidence intervals on the performancemetric. The main ideas and techniques for multiplecomparisons in the context of pairwise comparisons,or against a known standard are presented in [86],[59], and [90]. Recent work in multiple comparisonswith a control include [110] and [146], which providefully sequential and two-stage frequentist proceduresrespectively; and [201], which addresses the problemusing a Bayesian approach.

Comprehensive treatment of ranking and selectionand multiple comparison procedures may be foundin Goldsman and Nelson [74] and Bechhofer et al.


[21]. A detailed survey that traces the developmentof techniques in simulation optimization over finitesets is available in [194].

3.1.2 Large/Infinite parameter spaces

To address DOvS problems with a large number ofpotential alternatives, algorithms that have a searchcomponent are required. Many of the algorithmsthat are applicable to the continuous optimizationvia simulation case are, with suitable modifications,applicable to the case with large/infinite parameterspaces. These include (1) ordinal optimization(2) random search methods and (3) direct searchmethods.

Ordinal optimization methods [84] are suitablewhen the number of alternatives is too large tofind the globally optimal design in the discrete-eventsimulation context. Instead, the task is to find asatisfactory solution with some guarantees on quality(called alignment probability) [127]. Here, the focusis on sampling a chosen subset of the solutions andevaluating them to determine the best among them.The key lies in choosing this subset such that itcontains a subset of satisfactory solutions. Thequality or satisfaction level of this selected subset canbe quantified [34]. A comparison of subset selectionrules is presented in [100] and the multi-objective caseis treated in [195].

Random search methods include techniques suchas simulated annealing (e.g., [3]), genetic algorithms,stochastic ruler methods (e.g., [208]), stochasticcomparison (e.g., [75]), nested partitions (e.g.,[182]), ant colony optimization (e.g., [52, 51]),and tabu search (e.g., [71]). Some of these—simulated annealing, genetic algorithms, and tabusearch—are described in Section 3.6). Antcolony optimization is described under model-basedmethods (cf. Section 3.7.2). Proofs of globalconvergence, i.e., convergence to a global solution,or local convergence are available for most of thesealgorithms [88] (note that these definitions differ frommathematical programming where global convergenceproperties ensure convergence to a local optimum

regardless of the starting point).Nested partition methods [181] attempt to

adaptively sample from the feasible region. Thefeasible region is then partitioned, and samplingis concentrated in regions adjudged to be themost promising by the algorithm from a pre-determined collection of nested sets. Hong andNelson propose the COMPASS algorithm [87] whichuses a unique neighborhood structure, defined asthe most promising region that is fully adaptive

rather than pre-determined; a most promising ‘index’is defined that classifies each candidate solutionbased on a nearest neighbor metric. More recently,the Adaptive Hyberbox Algorithm [207] claimsto have superior performance on high-dimensionalproblems (problems with more than ten or fifteenvariables); and the R-SPLINE algorithm [198],which alternates between a continuous search ona continuous piecewise-linear interpolation and adiscrete neighborhood search, compares favorably aswell.

A review of random search methods is presented in[8, 149]. Recent progress, outlines of basic algorithms,and pointers to specific references for some of thesemethods are presented in [26], [88], and [145].

Direct search methods such as pattern search andNelder-Mead simplex methods are elaborated on inSection 3.5.

3.2 Response surface methodology

Response surface methodology (RSM) is typicallyuseful in the context of continuous optimizationproblems and focuses on learning input-output rela-tionships to approximate the underlying simulationby a surface (also known as a metamodel or surrogatemodel) for which we define a functional form. Thisfunctional form can then be made use of by leveragingpowerful derivative-based optimization techniques.The literature in RSM is vast and equivalentapproaches have variously been referred to as multi-disciplinary design optimization, metamodel-basedoptimization, and sequential parameter optimization.RSM was originally developed in the context ofexperimental design for physical processes [28], buthas since been applied to computer experiments.Metamodel-based optimization is a currently populartechnique for addressing simulation optimizationproblems [20, 115].

Different response surface algorithms differ in thechoice between regression and interpolation; thenature of the functional form used for approximation(polynomials, splines, Kriging, radial basis functions,neural networks); the choice of how many and wherenew samples must be taken; and how they update theresponse surface.

RSM approaches can either (1) build surrogatemodels that are effective in local regions, andsequentially use these models to guide the search, or;(2) build surrogate models for the entire parameterspace from space-filling designs, and then use themto choose samples in areas of interest, i.e., wherethe likelihood of finding better solutions is good


Algorithm 2 Basic RSM procedure

Require: Initial region of approximation X ,choice of regression surface r

1: while under simulation budget and notconverged do

2: Perform a design of experiments inrelevant region, using k data points

3: ti ← simulate(xi), i = {1, . . . , k}{Evaluate noisy function f(xi, ω)}

4: λ∗ ← argminλ∑

(ti − r(xi, λ))2

{Fit regression surface r through pointsusing squared loss function}

5: x∗ ← {argminx r(x, λ∗) : x ∈ X}

{Optimize surface}6: Update set of available data points

and region of approximation7: end while

according to a specified metric. A generic frameworkfor RSM is presented in Algorithm 2.

Classical sequential RSM. Originally,RSM consisted of a Phase I, where first-ordermodels were built using samples from a design ofexperiments. A steepest descent rule was used tomove in a certain direction, and this would continueiteratively until the estimated gradient would beclose to zero. Then, a Phase II procedure thatbuilt a more detailed quadratic model would beused for verifying the optimality of the experimentaldesign. A thorough introduction to response surfacemethodology is available in [142]. Recent work inthe field includes automating RSM [143, 147] andthe capability to handle stochastic constraints [13].

Bayesian global optimization. Thesemethods seek to build a global response surface,commonly using techniques such as Kriging/Gaussianprocess regression [173, 163]. Subsequent sampleschosen based on some sort of improvement metricmay balance exploitation and exploration. Theseminal paper by Jones et al. [102] which introducedthe EGO algorithm for simulations with deterministicoutput, uses Kriging to interpolate between functionvalues, and chooses future samples based on anexpected improvement metric [140]. Examples ofanalogs to this for simulation optimization areprovided in [93, 118]. The use of Kriging for

simulation metamodeling is explored in [22, 117,116]. Other criteria that have been used tochoose samples are most probable improvement [139],knowledge gradient for continuous parameters [177],and maximum information gain [189].

Trust region methods. Trust region meth-ods [42] can be used to implement sequential RSM.Trust regions provide a means of controlling theregion of approximation, providing update criteriafor surrogate models, and are useful in analyzingconvergence properties. Once a metamodel orresponse surface, g, is built around a trust regioncenter xi, trust region algorithms involve the solutionof the trust-region subproblem (mins g(xi + s) : s ∈B(xi,∆)), where B is a ball defined by the center-radius pair (xi,∆). There are well-defined criteria toupdate the trust region center and radius [42] thatwill define the subsequent region of approximation.

The use of trust regions in simulation optimizationis relatively recent, and has been investigated tosome extent [46, 32]. Trust-region algorithms havebeen used, for example, to optimize simulations ofurban traffic networks [151].

3.3 Gradient-based methods

Stochastic approximation methods or gradient-basedapproaches are those that attempt to descendusing estimated gradient information. Stochasticapproximation techniques are one of the oldestmethods for simulation optimization. Robbins andMonro [167] and Kiefer and Wolfowitz [109] werethe first to develop stochastic approximation schemesin the early 1950s. These procedures initially weremeant to be used under very restrictive conditions,but much progress has been made since then.

These methods can be thought of being analogousto steepest descent methods in derivative-basedoptimization. One may obtain direct gradients ormay estimate gradients using some finite differencescheme. Direct gradients may be calculated bya number of methods: (1) Perturbation Analysis(specifically, Infinitesimal Perturbation Analysis)(PA or IPA), (2) Likelihood Ratio/Score Function(LR/SF), and (3) Frequency Domain Analysis(FDA). Detailed books on these methods areavailable in the literature [85, 69, 172, 156, 65] andmore high-level descriptions are available in papers[194, 60]. Most of these direct methods, however, areeither applicable to specific kinds of problems, needsome information about underlying distributions, or


are difficult to apply. Fu [60] outlines which methodsare applicable in which situations, and Tekin andSabuncuoglu [194] discuss a number of applicationsthat have used these methods.

Stochastic approximation schemes attempt toestimate a gradient by means of finite differences.Typically, a forward difference estimate wouldinvolve sampling at least n + 1 distinct points,but superior performance has been observed bysimultaneous perturbation estimates that requiresamples at just two points [185], a methodreferred to as Simultaneous Perturbation StochasticApproximation (SPSA). The advantage gained inSPSA is that the samples required are nowindependent of the problem size, and, interestingly,this has been shown to have the same asymptoticconvergence rate as the naive method that requiresn + 1 points [184]. A typical gradient-based schemeis outlined in Algorithm 3.

Algorithm 3 Basic gradient-based proce-dure

Require: Specify initial point, x0. Defineinitial parameters such as step size (α),distances between points for performingfinite difference, etc.

1: i← 02: while under simulation budget and not

converged do3: Perform required simulations, tjii ←

simulate(xi), with ji replications to

estimate gradient, J , using either IPA,LR/SF, FDA or finite differences

4: xi+1 ← xi − αJ5: i← i+ 16: end while

Recent extensions of the SPSA method includeintroducing a global search component to thealgorithm by injecting Monte Carlo noise duringthe update step [134], and using it to solvecombined discrete/continuous optimization problems[199]. Recent work also addresses improving Jacobianas well as Hessian estimates in the context of theSPSA algorithm [187]. Much of the progress instochastic approximation has been cataloged in theproceedings of the Winter Simulation Conferenceover the years (http://informs-sim.org/). Arecent review of stochastic approximation methods isavailable in [188], and an excellent tutorial and review

of results in stochastic approximation is presented in[154].

3.4 Sample path optimization

Sample path optimization involves working with anestimate of the underlying unknown function, asopposed to the function itself. The estimate isusually a consistent estimator such as the samplemean of independent function evaluations at a point,or replications. For instance, one may work withFn = 1

n

∑n

i=1f(x, y, ωi), instead of the underlying

function E[f(x, y, ω)] itself. It should be noted thatthe functional form of Fn is still unknown, it is justthat Fn can be observed or evaluated at a point in thesearch space visited by an algorithm iteration. Thealternative name of sample average approximationreflects this use of an estimator.

As the algorithm now has to work with an esti-mator, a deterministic realization of the underlyingstochastic function, sophisticated techniques fromtraditional mathematical programming can now beleveraged. Sample path methods can be viewed as theuse of deterministic optimization techniques withina well-defined stochastic setting. Yet another namefor them is stochastic counterpart. Some of thefirst papers using sample path methods are [82] and[179]. Several papers [172, 37, 77, 180, 46] discussconvergence results and algorithms in this context.

3.5 Direct search methods

Direct search can be defined as the sequentialexamination of trial solutions generated by acertain strategy [89]. As opposed to stochasticapproximation, direct search methods rely on directcomparison of function values without attemptingto approximate derivatives. Direct search methodstypically rely on some sort of ranking of quality ofpoints, rather than on function values.

Most direct search algorithms developed forsimulation optimization are extensions of ideas forderivative-free optimization. A comprehensive reviewof classical and modern methods is provided in[121]. A formal theory of direct search methods forstochastic optimization is developed in [196]. Directsearch methods can be tailored for both discreteand continuous optimization settings. Pattern searchand Nelder-Mead simplex procedures are the mostpopular direct search methods. There is someclassical as well as relatively recent work done oninvestigating both pattern search methods [196, 6,131] and Nelder-Mead simplex algorithms [144, 19,


94, 31] and their convergence in the context ofsimulation optimization.

These methods remain attractive as they arerelatively easy to describe and implement, and arenot affected if a gradient does not exist everywhere,as they do not rely on gradient information. Sinceconventional procedures can be affected by noise,effective sampling schemes to control the noise arerequired. A basic Nelder-Mead procedure is outlinedin Algorithm 4.

Algorithm 4 Basic Nelder-Mead simplexprocedure for SO

Require: A set of n − 1 points in theparameter space to form the initialsimplex


2: Generate a new candidate solution,xi, through simplex centroid reflections,contractions or other means

3: tjii ← simulate(xi), i = {i − n +

1, . . . , i}, ji = {1, . . . , Ni} {Evaluatenoisy function f(x, ω) Ni times, where Ni

is determined by some sampling scheme}4: Calculate

∑jitjii

Ni, or some similar

metric to determine which point (i.e.,with the highest metric value) should beeliminated

5: end while

3.6 Random search methods

3.6.1 Genetic algorithms

Genetic algorithms use concepts of mutation andselection [164, 200]. In general, a genetic algorithmworks by creating a population of strings and eachof these strings are called chromosomes. Each ofthese chromosome strings is basically a vector of pointin the search space. New chromosomes are createdby using selection, mutation and crossover functions.The selection process is guided by evaluating thefitness (or objective function) of each chromosomeand selecting the chromosomes according to theirfitness values (using methods such as mapping ontoRoulette Wheel). Additional chromosomes are thengenerated using crossover and mutation functions.

The cross over and mutation functions ensure thata diversity of solutions is maintained. Geneticalgorithms are popular as they are easy to implementand are used in several commercial simulationoptimization software packages (Table 4). TheGECCO (Genetic and Evolutionary ComputationConference) catalogs progress in genetic algorithmsand implementations.

3.6.2 Simulated annealing

Simulated Annealing uses a probabilistic methodthat is derived from the annealing process in whichthe material is slowly cooled so that, while itsstructure freezes, it reaches a minimum energy state[113, 23]. Starting with a current point i in astate j, a neighborhood point i′ of the point i isgenerated. The algorithm moves from point i to i′

using a probabilistic criteria that is dependent onthe ‘temperature’ in state j. This temperature isanalogous to that in physical annealing, and serveshere as a control parameter. If the solution at i′

is better than the existing solution, then this newpoint is accepted. If the new solution is worse thanexisting solution, then the probability of acceptingthe point is defined as exp(−(f(i′) − f(i))/T (j)),where f(.) is the value of objective function at agiven point, and T (j) is temperature at the statej. After a certain number of neighborhood pointsare evaluated, the temperature is decreased and newstate is j + 1 is created. Due to the exponentialform, the probability of acceptance of a neighborhoodpoint is higher at high temperature, and is lower astemperature is reduced. In this way, the algorithmsearches for a large number of neighborhood pointsin the beginning, but a lower number of points astemperature is reduced.

Implementation of simulated annealing proceduresrequire choosing parameters such as the initial andfinal temperatures, the rate of cooling, and number offunction evaluations at each temperature. A varietyof cooling ‘schedules’ have been suggested in [41]and [78]. Though simulated annealing was originallymeant for optimizing deterministic functions, theframework has been extended to the case of stochasticsimulations [2]. The ease of implementing a simulatedannealing procedure is high and it remains a populartechnique used by several commercial simulationoptimization packages.

3.6.3 Tabu search

Tabu search [70] uses special memory structures(short-term and long-term) during the search


process that allow the method to go beyond localoptimality to explore promising regions of thesearch space. The basic form of tabu searchconsists of a modified neighborhood search procedurethat employs adaptive memory to keep track ofrelevant solution history, together with strategies forexploiting this memory [66]. More advanced forms oftabu search and its applications are described in [72].

3.6.4 Scatter search

Scatter search and its generalized form, pathrelinking, were originally introduced by Gloverand Laguna [73]. Scatter search differs fromother evolutionary approaches (such as GeneticAlgorithms (GA)) by using strategic designs andsearch path construction from a population ofsolutions as compared to randomization (by crossoverand mutation in GA). Similar to Tabu search, ScatterSearch also utilize adaptive memory in storing bestsolutions [73, 133]. Algorithm 5 provides the scattersearch algorithm.

3.7 Model-based methods

Model-based simulation optimization methods at-tempt to build a probability distribution over thespace of solutions and use it to guide the searchprocess.

3.7.1 Estimation of distribution algorithms

Estimation of distribution algorithms (EDAs) [126]are model-based methods that belong to theevolutionary computation field. However, generationof new candidate solutions is done by sampling fromthe inferred probability distribution over the space ofsolutions, rather than, say, a genetic operator suchas crossover or mutation. A comprehensive review ofestimation of distribution algorithms is presented in[64]. EDAs usually consider interactions between theproblem variables and exploit them through differentprobability models.

Cross-entropy methods and Model ReferenceAdaptive Search (MRAS) are discussed next and canbe seen as specific instances of EDAs.

Cross-Entropy Methods. Cross-entropymethods first sample randomly from a chosenprobability distribution over the space of decisionvariables. For each sample, which is a vector defininga point in decision space, a corresponding functionevaluation is obtained. Based on the function values

Algorithm 5 Basic scatter search procedurefor SO

Require: An initial set of trial points x ∈P , chosen to be diversified according to apre-specified metric

1: tj ← simulate(xj), where j = 1, . . . , |P |2: k ← 03: Use a comparison procedure (such as

ranking and selection) to gather the bestb solutions (based on objective valueor diversity) from the current set ofsolutions P , called the reference set, Rk

4: R−1 = ∅

5: while under simulation budget and Rk 6=Rk−1 do

6: k ← k + 17: Choose Si ⊂ R, where i = 1, . . . , r{Use a subset generation procedure toselect r subsets of set R, to be used as abasis for generating new solution points}

8: for i = 1 to r do9: Combine the points in Si, to form

new solution points, xj , where j ∈ J =|P |+1, . . . , |P |+J , using weighted linearcombinations, for example

10: tj ← simulate(xj), j ∈ J{sample the objective function at newtrial solutions}

11: Update sets Rk, P12: end for13: end while


observed, a pre-defined percentile of the best samplesare picked. A new distribution is built aroundthis ‘elite set’ of points via maximum likelihoodestimation or some other fitting method, and theprocess is repeated. One possible method thatimplements cross-entropy is formally described inAlgorithm 6.

Algorithm 6 Pseudocode for a simple cross-entropy implementation

Require: θ, an initial set of parameters fora pre-chosen distribution p(x; θ) over theset of decision variables; k, a numberof simulations to be performed; e, thenumber of elite samples representing thetop δ percentile of the k samples


2: for i = 1→ k do3: sample xi from p(x; θ)4: ti ← simulate(xi)5: end for6: E ← ∅7: for i = 1→ e do8: Ej ← argmaxi/∈E ti9: end for10: p(x; θ)← fit(xE)11: end while

The method is guaranteed (probabilistically) toconverge to a local optimum, but it also incorporatesan exploration component as random samples areobtained at each step. However, the intuitionbehind the selection of subsequent samples can beshown to be analogous to minimizing the Kullback-Leibler divergence (KL-divergence) between theoptimal importance sampling distribution and thedistribution used in the current iterate [171].

There exist variants of the cross-entropy methodto address both continuous [124] and discreteoptimization [170] problems. A possible modificationis to use mixtures of distributions from current andprevious iterations, with the current distributionweighted higher. This can be done by linearlyinterpolating the mean covariance in the case ofGaussian distributions. This also helps in avoidingsingular covariance matrices. Cross-entropy canalso deal with noisy function evaluations, withirrelevant decision variables, and constraints [124]. If

decision variables are correlated, the covariance of thedistribution will reflect this.

The immediately apparent merits of cross-entropymethods are that they are easy to implement,require few algorithmic parameters, are based onfundamental principles such as KL-divergence andmaximum likelihood, and give consistently accurateresults [124]. A potential drawback is that cross-entropy may require a significant number of newsamples at every iteration. It is not clear as tohow this would affect performance if samples wereexpensive to obtain. The cross-entropy method hasanalogs in simulated annealing, genetic algorithms,and ant colony optimization, but differs from each ofthese in important ways [44].

More detailed information on the use of cross-entropy methods for optimization can be foundin [44], a tutorial on cross-entropy and in[171], a monograph. The cross-entropy webpage,http://iew3.technion.ac.il/CE/ provides up-to-dateinformation on progress in the field.

Model reference adaptive search(MRAS). The MRAS method [91, 92] is closelyrelated to the cross-entropy method. It also worksby minimizing the Kullback-Leibler divergence toupdate the parameters of the inferred probabilitydistribution. However, the parameter updatestep involves the use of a sequence of implicitprobability distributions. In other words, while thecross-entropy method uses the optimal importancesampling distribution for parameter updates, MRASminimizes the KL-divergence with respect to thedistribution in the current iteration, called thereference model.

Covariance Matrix Adaptation–Evolu-tion Strategy (CMA-ES). In the CMA-ES algorithm [81], new samples are obtained from amultivariate normal distribution, and inter-variabledependencies are encoded in the covariance matrix.The CMA-ES method provides a way to update thecovariance matrix. Updating the covariance matrixis analogous to learning an approximate inverseHessian, as is used in Quasi-Newton methods inmathematical programming. The update of themean and covariance is done by maximizing thelikelihood of previously successful candidate solutionsand search steps, respectively. This is in contrast toother EDAs and the cross-entropy method, where thecovariance is updated by maximizing the likelihoodof the successful points. Other sophistications suchas step-size control, and weighting of candidate


solutions are part of modern implementations [80].

3.7.2 Ant colony optimization

Ant colony optimization methods [52, 51] are heuris-tic methods that have been used for combinatorialoptimization problems. Conceptually, they mimic thebehavior of ants to find shortest paths between theircolony and food sources. Ants deposit pheromonesas they walk; and are more likely to choose pathswith higher concentration of pheromones. Thisphenomenon is incorporated in a pheromone updaterule, which increases the pheromone content incomponents of high-quality solutions, and causesevaporation of pheromones in less favorable regions.Probability distributions are used to make thetransition between each iteration. These methodsdiffer from EDAs in that they use an iterativeconstruction of solutions.

This and other algorithms that incorporateself-organization in biological systems are said to usethe concept of ‘swarm intelligence’.

3.8 Lipschitzian optimization

Lipschitzian optimization is a class of space-partitioning algorithms for performing global op-timization, where the Lipschitz constant is pre-specified. This enables the construction of globalsearch algorithms with convergence guarantees. Thecaveat of having prior knowledge of the Lipschitzconstant is overcome by the DIRECT (DIvidingRECTangles) algorithm [101] for deterministiccontinuous optimization problems. An adaptation ofthis for noisy problems is provided in [47].

4 Software

4.1 Simulation optimization in commercialsimulation software

Many discrete-event simulation packages incorporatesome methodology for performing optimization.A comprehensive listing of simulation software,the corresponding vendors, and the optimizationpackages and techniques they use can be found inTable 4. More details on the specific optimizationroutines can be found in [128]. OR/MS-Today, theonline magazine of INFORMS, conducts a biennialsurvey of simulation software packages, the latestof which is available at [150]. The survey lists43 simulation software packages, and 31 of these

have some sort of optimization routine; fewer stillhave black-box optimizers that interact with thesimulation.

4.2 Academic implementations of simulationoptimization

Table 5 contains a small subset of academicimplementations of SO algorithms, and classifiesthem by type. Some of these are availablefor download from the web, some have codewith suggested parameters in corresponding papersthemselves, and others are available upon requestfrom the authors.

5 Comparison of algorithms

As far as comparisons between algorithms areconcerned, the literature does not yet provide acomprehensive survey of the performance of differentimplementations and approaches on large test beds.In this regard, simulation optimization lags behindother optimization fields such as linear, integer,and nonlinear programming, global optimization andeven derivative-free optimization, where the firstcomprehensive comparison appeared in 2013 [166]. Astudy of prior comparisons in simulation optimizationis provided by [194], but these comparisons arefairly dated, are inconclusive about which algorithmsperform better in different situations, and compareonly a small subset of available algorithms. Onedifficulty lies in the inherent difficulty of comparingsolutions between algorithms over true black-boxsimulations, as one does not usually know thetrue optimal point and can only compare betweennoisy estimates observed by the solvers. Lessimpeding difficulties, but difficulties nonetheless,include the need to interface algorithms to a commonwrapper, the objective comparison with solvers thatincorporate random elements as their results may notbe reproducible, and lack of standard test simulationsfor purposes of benchmarking.

The benchmarking of algorithms in mathematicalprogramming is usually done by performance profiles[50], where the graphs show the fraction of problemssolved after a certain time. For derivative-freealgorithms, data profiles are commonly used [141],where the fraction of problems solved after acertain number of iterations (function evaluations)or ‘simplex gradients’ is shown. The definition ofwhen a problem is ‘solved’ may vary—when the trueglobal optimum is known, the solutions found withina certain tolerance of this optimal value may be called


Table 4: Simulation optimization packages in commercial simulation software

Optimizationpackage

Vendor Simulationsoftwaresupported

Optimizationmethodology

AutoStat Applied Materials,Inc.

AutoMod Evolutionarystrategy

EvolutionaryOptimizer

Imagine That, Inc. ExtendSim Evolutionarystrategy

OptQuest OptTek Systems,Inc.

FlexSim, @RISK,Simul8, Simio,SIMPROCESS,AnyLogic,Arena, CrystalBall, EnterpriseDynamics,ModelRisk

Scatter search,tabu search, neuralnetworks, integerprogramming

SimRunner ProModel Corp. ProModel,MedModel,ServiceModel

Genetic algorithmsand evolutionarystrategies

RISKOptimizer Palisade Corp. @RISK Genetic algorithm

WITNESSOptimizer

Lanner Group, Inc. WITNESS Simulated annealing,tabu search, hillclimbing

GoldSimOptimizer

GoldSim TechnologyGroup

GoldSim Box’s complexmethod

PlantSimulationOptimizer

Siemens AG Siemens PLM soft-ware

Genetic algorithm

ChaStrobeGA N/A Stroboscope Genetic algorithm

GlobalOptimizationtoolbox

The MathWorks SimEvents (Matlab) Genetic algorithms,simulated annealing,pattern search


Table 5: Academic simulation optimization implementations

Algorithm Type Citation Year

Continuous

SPSA Stochastic approximation [185] 1992SPSA 2ndOrder

Stochastic approximation [185] 1999

SKO Global response surface [93] 2006CE method Cross-entropy [124] 2006APS Nested partitioning [105] 2007SNOBFIT Multi-start local response surface [97] 2008CMA-ES Evolutionary strategy [80] 2011KGCP Global response surface [177] 2011STRONG Local response surface, trust region [32] 2011GR Golden region search [106] 2011SNM Direct search (Nelder-Mead) [31] 2012DiceOptim Global response surface [169] 2012

Discrete

KG Global response surface [57] 2009COMPASS Neighborhood search (integer-ordered problems) [206] 2010R-SPLINE Neighborhood search (integer-ordered problems) [198] 2012

Discreteandcontinuous

MRAS Estimation of distribution [91, 92] 2005NOMADm Mesh adaptive direct search [1] 2007

solutions, but when this optimum is not known, thesolvers that find the best solution (within a tolerance)for a problem, with respect to the other solversbeing compared, may be said to have solved theproblem. The latter metric may also be used whenfunction evaluations are expensive, and no solver isable to reach within this tolerance given the limitedsimulation budget.

In both of these cases, the output of thesimulations are deterministic, and so it is clear asto which algorithms have performed better thanothers on a particular problem. In simulationoptimization, however, usually one does not knowthe true solution for the black box system, nor doesone see deterministic output. All that one possessesare mean values and sample variances obtained fromsample paths at different points. There does not

exist a standard method to compare simulationoptimization algorithms on large test beds. Manypapers perform several macroreplications and reportthe macroreplicate average of the best sample means(along with the associated sample variance) at theend of the simulation budget. The issue with this isthat the performance of the algorithms with differentsimulation budgets is not seen, as in the case ofperformance or data profiles. Other papers reportthe average number of evaluations taken to find asample mean that is within the global tolerancefor each problem. Here, results are listed for eachproblem and one does not get an idea of overallperformance. In addition, the difference in samplevariance estimates is not highlighted. As simulationoptimization develops, there is also a need formethods of comparison of algorithms on test beds


with statistically significant number of problems.With regard to standardized simulation testbeds,

to our knowledge, the only testbed that providespractical simulations for testing simulation opti-mization algorithms is available at www.simopt.org[153]. At the point of writing this paper,just 20 continuous optimization problems wereavailable from this repository. Most testingand comparisons happen with classical test prob-lems in nonlinear optimization (many of whichhave been compiled in [166] and available athttp://archimedes.cheme.cmu.edu/?q=dfocomp), towhich stochastic noise has been added. There is aneed for more such repositories, not only for testing ofalgorithms over statistically significant sizes of prob-lem sets, but for comparison between different classesof algorithms. The need for comparison is evident,given the sheer number of available approaches tosolving simulation optimization problems, and thelack of clarity and lack of consensus on which typesof algorithms are suitable in which contexts.

As observed by several papers [61, 194, 88], therecontinues to exist a significant gap between researchand practice in terms of algorithmic approaches.Optimizers bundled with simulation software, asobserved in Section 4, tend to make use of algorithmswhich seem to work well but do not come withprovable statistical properties or guarantees of localor global convergence. Academic papers, on the otherhand, emphasize methods that are more sophisticatedand prove convergence properties. One reason thatmay contribute to this is that very few simulationoptimization algorithms arising from the researchcommunity are easily accessible. We wholeheartedlyencourage researchers to post their executable files, ifnot their source code. This could not only encouragepractitioners to use these techniques in practice,but allow for comparisons between methods andthe development of standardized interfaces betweensimulations and simulation optimization software.

6 Conclusions

The field of simulation optimization has progressedsignificantly in the last decade, with severalnew algorithms, implementations, and applications.Contributions to the field arise from researchers andpractitioners in the industrial engineering/operationsresearch, mathematical programming, statistics andmachine learning, as well as the computer sciencecommunities. The use of simulation to modelcomplex, dynamic, and stochastic systems has onlyincreased with computing power and availability of a

wide variety of simulation languages. This increaseduse is reflected in the identification and applicationof simulation and simulation optimization methodsto diverse fields in science, engineering, and business.There also exist strong analogies between, and ideasthat may be borrowed from recent progress in relatedfields. All of these factors, along with the everincreasing number of publications and rich literaturein this area, clearly indicate the interest in the field ofsimulation optimization, and we have tried to capturethis in this paper.

With increased growth and interest in thefield, there come also opportunities. Potentialdirections for the field of simulation optimizationare almost immediately apparent. Apart from theability to handle simulation outputs from any well-defined probability distribution, the effective use ofvariance reduction techniques when possible, andthe improvement in theory and algorithms, there isa requirement to address (1) large-scale problemswith combined discrete/continuous variables; (2)the ability to effectively handle stochastic anddeterministic constraints of various kinds; (2) theeffective utilization of parallel computing at thelinear algebra level, sample replication level, iterationlevel, as well as at the algorithmic level; (3) theeffective handling of multiple simulation outputs;(4) the incorporation of performance measuresother than expected values, such as risk; (5)the continued consolidation of various techniquesand their potential synergy in hybrid algorithms;(6) the use of automatic differentiation techniquesin the estimation of simulation derivatives whenpossible; (7) the continued emphasis on providingguarantees of convergence to optima for local andglobal optimization routines in general settings;(8) the availability and ease of comparison of theperformance of available approaches on differentapplications; and (9) the continued reflection ofsophisticated methodology arising from the literaturein commercial simulation packages.

References

[1] Abramson, M.A.: NOMADm version 4.5 User’sGuide. Air Force Institute of Technology,Wright-Patterson AFB, OH (2007)

[2] Alkhamis, T.M., Ahmed, M.A., Tuan, V.K.:Simulated annealing for discrete optimizationwith estimation. European Journal ofOperational Research 116, 530–544 (1999)


[3] Alrefaei, M.H., Andradottir, S.: A simulatedannealing algorithm with constant temperaturefor discrete stochastic optimization. Manage-ment Science 45, 748–764 (1999)

[4] Amaran, S., Sahinidis, N.V., Sharda, B., Bury,S.J.: Simulation optimization: A review ofalgorithms and applications. 4OR–A QuarterlyJournal of Operations Research 12, 301–333(2014)

[5] Ammeri, A., Hachicha, W., Chabchoub, H.,Masmoudi, F.: A comprehensive literature re-view of mono-objective simulation optimizationmethods. Advances in Production Engineering& Management 6(4), 291–302 (2011)

[6] Anderson, E.J., Ferris, M.C.: A direct searchalgorithm for optimization with noisy functionevaluations. SIAM Journal on Optimization11, 837–857 (2001)

[7] Andradottir, S.: Chapter 9: Simulationoptimization. In: J. Banks (ed.) Handbookof Simulation: Principles, Methodology, Ad-vances, Applications, and Practice. John Wiley& Sons, New York (1998)

[8] Andradottir, S.: An overview of simulationoptimization via random search. In: S.G.Henderson, B.L. Nelson (eds.) Handbooks inOperations Research and Management Science:Simulation, vol. 13, chap. 20, pp. 617–631.Elsevier (2006)

[9] Andradottir, S.: Simulation optimization.In: Handbook of Simulation: Principles,Methodology, Advances, Applications andPractice, pp. 307–333. John Wiley & Sons(2006)

[10] Andradottir, S., Kim, S.H.: Fully sequentialprocedures for comparing constrained systemsvia simulation. Naval Research Logistics 57(5),403–421 (2010)

[11] de Angelis, V., Felici, G., Impelluso, P.:Integrating simulation and optimisation inhealth care centre management. EuropeanJournal of Operational Research 150, 101–114(2003)

[12] Angun, E.: Black box simulation optimization:Generalized response surface methodology.Ph.D. thesis, Tilburg University (2004)

[13] Angun, E., Kleijnen, J.P.C., Hertog, D.D.,Gurkan, G.: Response surface methodologywith stochastic constraints for expensive

simulation. Journal of the OperationalResearch Society 60(6), 735–746 (2009)

[14] Ayvaz, M.T.: A linked simulation-optimizationmodel for solving the unknown groundwaterpollution source identification problems. Jour-nal of Contaminant Hydrology 117(1–4), 46–59(2010)

[15] Azadivar, F.: A tutorial on simulationoptimization. In: J.J. Swain, D. Goldsman,R.C. Crain, J.R. Wilson (eds.) Proceedings ofthe 1992 Winter Simulation Conference, pp.198–204 (1992)

[16] Azadivar, J.: Simulation optimization method-ologies. In: P.A. Farrington, H.B. Nembhard,D.T. Sturrock, G.W. Evans (eds.) Proceedingsof the 1999 Winter Simulation Conference, pp.93–100 (1999)

[17] Balakrishna, R., Antoniou, C., Ben-Akiva, M.,Koutsopoulos, H.N., Wen, Y.: Calibration ofmicroscopic traffic simulation models: Methodsand application. Transportation ResearchRecord: Journal of the TransportationResearch Board 1999(1), 198–207 (2007)

[18] Bangerth, W., Klie, H., Matossian, V.,Parashar, M., Wheeler, M.F.: An autonomicreservoir framework for the stochastic opti-mization of well placement. Cluster Computing8(4), 255–269 (2005)

[19] Barton, R.R., Ivey, Jr., J.S.: Nelder-Mead simplex modifications for simulationoptimization. Management Science 42, 954–973 (1996)

[20] Barton, R.R., Meckesheimer, M.: Metamodel-based simulation optimization. In: S. Hender-son, B. Nelson (eds.) Handbook in OperationsResearch and Management Science: Simulation13, pp. 535–574. Elsevier (2006)

[21] Bechhofer, R.E., Santner, T.J., Goldsman,D.M.: Design and Analysis of Experiments forStatistical Selection, Screening, and MultipleComparisons. John Wiley & Sons (1995)

[22] van Beers, A.C., Kleijnen, J.P.C.: Kriginginterpolation in simulation: A survey. Proceed-ings of the 2004 Winter Simulation Conference1, 121–129 (2004)

[23] Bertsimas, D., Tsitsiklis, J.: Simulatedannealing. Statistical Science 8(1), 10–15(1993)


[24] Bettonvil, B., del Castillo, E., Kleijnen, J.P.C.:Statistical testing of optimality conditions inmultiresponse simulation-based optimization.European Journal of Operational Research199, 448–458 (2009)

[25] Bhatnagar, S.: Adaptive multivariate three-timescale stochastic approximation algorithmsfor simulation based optimization. ACMTransactions on Modeling and ComputerSimulation 15(1), 74–107 (2005)

[26] Bianchi, L., Dorigo, M., Gambardella, L.M.,Gutjahr, W.J.: A survey on metaheuristics forstochastic combinatorial optimization. NaturalComputing 8(2), 239–287 (2009)

[27] Birge, J.R., Louveaux, F.: Introduction toStochastic Programming, 2nd edn. Springer(2011)

[28] Box, G.E.P., Wilson, K.B.: On the experimen-tal attainment of optimum conditions. Journalof the Royal Statistical Society XIII(1), 1–35(1951)

[29] Carson, Y., Maria, A.: Simulation opti-mization: Methods and applications. In:S. Andradottir, K.J. Healy, D.H. Winters, B.L.Nelson (eds.) Proceedings of the 1997 WinterSimulation Conference, pp. 118–126 (1997)

[30] Chang, K.H.: Stochastic trust region responsesurface convergent method for continuoussimulation optimization. Ph.D. thesis, PurdueUniversity (2008)

[31] Chang, K.H.: Stochastic Nelder-Mead simplexmethod–A new globally convergent directsearch method for simulation optimization.European Journal of Operational Research220, 684–694 (2012)

[32] Chang, K.H., Hong, L.J., Wan, H.:Stochastic trust-region response-surfacemethod (STRONG)—A new response-surfaceframework for simulation optimization.INFORMS Journal on Computing 25(2),230–243 (2013)

[33] Chen, C.H.: An effective approach to smartlyallocate computing budget for discrete eventsimulation. In: Proceedings of the 34th IEEEConference on Decision and Control, pp. 2598–2605 (1995)

[34] Chen, C.H.: A lower bound for the correctsubset selection probability and its applicationto discrete event system simulations. IEEE

Transactions on Automatic Control 41, 1227–1231 (1996)

[35] Chen, C.H., Lee, L.H.: Stochastic SimulationOptimization: An Optimal Computing BudgetAllocation. System Engineering and Oper-ations Research. World Scientific PublishingCompany (2010)

[36] Chen, C.H., Yucesan, E., Dai, L., Chen,H.C.: Optimal budget allocation for discrete-event simulation experiments. IIE Transactions42(1), 60–70 (2009)

[37] Chen, H., Schmeiser, B.W.: Retrospectiveoptimization algorithms for stochastic rootfinding. In: J. Tew, S. Manivannan,D. Sadowski, A. Seila (eds.) Proceedings of1994 Winter Simulation Conference, pp. 255–261 (1994)

[38] Chick, S.E.: Subjective probability andbayesian methodology. In: S.G. Henderson,B.L. Nelson (eds.) Simulation, Handbooks in

Operations Research and Management Science,vol. 13, pp. 225–257. Elsevier, Amsterdam(2006)

[39] Cho, J., Dorfman, K.D.: Brownian dynamicssimulations of electrophoretic DNA separationsin a sparse ordered post array. Journal ofChromatography A 1217, 5522–5528 (2010)

[40] Cohn, D.A., Ghahramani, Z., Jordan, M.I.:Active learning with statistical models. Journalof Artificial Intelligence Research 4, 129–145(1996)

[41] Collins, N.E., Eglese, R.W., Golden, B.L.:Simulated annealing–an annotated bibliogra-phy. American Journal of MathematicalManagement Sciences 8, 209–308 (1988)

[42] Conn, A.R., Gould, N.I.M., Toint, P.L.:Trust-region methods. MOS-SIAM Series onOptimization (2000)

[43] Conn, A.R., Scheinberg, K., Vicente, L.N.:Introduction to derivative-free optimization.SIAM, Philadelphia, PA (2009)

[44] de Boer, P.T., Kroese, D.P., Mannor, S.,Rubinstein, R.Y.: A tutorial on the cross-entropy method. Annals of OperationsResearch 134, 19–67 (2005)

[45] Deng, G.: Simulation-based optimization.Ph.D. thesis, University of Wisconsin–Madison(2007)


[46] Deng, G., Ferris, M.C.: Adaptation of theUOBYQA algorithm for noisy functions. In:L.F. Perrone, F.P. Wieland, J. Liu, B.G.Lawson, D.M. Nicol, R.M. Fujimoto (eds.)Proceedings of the 2006 Winter SimulationConference, pp. 312–319 (2006)

[47] Deng, G., Ferris, M.C.: Extension of theDIRECT optimization algorithm for noisyfunctions. In: S.G. Henderson, B. Biller, M.H.Hsieh, J. Shortle, J.D. Tew, R.R. Barton (eds.)Proceedings of the 2007 Winter SimulationConference, pp. 497–504 (2007)

[48] Dengiz, B., Akbay, K.S.: Computer simulationof a PCB production line: Metamodelingapproach. International Journal of ProductionEconomics 63(2), 195–205 (2000)

[49] Dhivya, M., Sundarambal, M., Anand, L.N.:Energy efficient computation of data fusionin wireless sensor networks using cuckoo-basedparticle approach (cbpa). International Journalof Communications, Network and SystemSciences 4(4), 249–255 (2011)

[50] Dolan, E.D., More, J.J.: Benchmarkingoptimization software with performance pro-files. Mathematical Programming 91, 201–213(2002)

[51] Dorigo, M., Blum, C.: Ant colony optimizationtheory: A survey. Theoretical ComputerScience 344(2–3), 243–278 (2005)

[52] Dorigo, M., Stutzle, T.: Ant colony optimiza-tion. MIT Press, Cambridge, MA, USA (2004)

[53] Driessen, L.T.: Simulation-based optimizationfor product and process design. Ph.D. thesis,Tilburg University (2006)

[54] Ernst, D., Glavic, M., Stan, G.B., Mannor,S., Wehenkel, L.: The cross-entropy methodfor power system combinatorial optimizationproblems. In: Power Tech, pp. 1290–1295.IEEE (2007)

[55] Ferris, M.C., Deng, G., Fryback, D.G.,Kuruchittham, V.: Breast cancer epidemiol-ogy: Calibrating simulations via optimization.Oberwolfach Reports 2 (2005)

[56] Figueira, G., Almada-Lobo, B.: Hybridsimulation-optimization methods: A taxon-omy. Simulation Modelling Practice andTheory 46, 118–134 (2014)

[57] Frazier, P., Powell, W., Dayanik, S.: Theknowledge-gradient policy for correlated nor-mal beliefs. INFORMS Journal on Computing21(4), 599–613 (2009)

[58] Frazier, P.I.: Knowledge-gradient methods forstatistical learning. Ph.D. thesis, PrincetonUniversity (2009)

[59] Fu, M.C.: Optimization via simulation: Areview. Annals of Operations Research 53,199–247 (1994)

[60] Fu, M.C.: Optimization for simulation: Theoryvs. practice. SIAM Journal on Computing14(3), 192–215 (2002)

[61] Fu, M.C., Andradottir, S., Carson, J.S.,Glover, F.W., Harrell, C.R., Ho, Y.C., Kelly,J.P., Robinson, S.M.: Integrating optimizationand simulation: Research and practice. In:J.A. Joines, R.R. Barton, K. Kang, P.A.Fishwick (eds.) Proceedings of the 2000 WinterSimulation Conference (2000)

[62] Fu, M.C., Glover, F.W., April, J.: SimulationOptimization: A review, new developments,and applications. In: M.E. Kuhl, N.M. Steiger,F.B. Armstrong, J.A. Joines (eds.) Proceedingsof the 2005 Winter Simulation Conference, pp.83–95 (2005)

[63] Fu, M.C., Hill, S.D.: Optimization of discreteevent systems via simulataneous perturbationstochastic approximation. IIE Transactions29(233–243) (1997)

[64] Fu, M.C., Hu, J., Marcus, S.I.: Model-basedrandomized methods for global optimization.In: Proceedings of the 17th International Sym-posium on Mathematical Theory of Networksand Systems, pp. 355–363. Kyoto, Japan (1996)

[65] Fu, M.C., Hu, J.Q.: Conditional MonteCarlo: Gradient estimation and optimizationapplications. Kluwer Academic Publishers(1997)

[66] Gendreau, M., Potvin, J.Y.: Tabu search.In: Handbook of Metaheuristics, InternationalSeries in Operations Research & Management

Science, vol. 146, second edn., pp. 41–60.Springer (2010)

[67] Gerencser, L., Kozmann, G., Vago, Z.,Haraszti, K.: The use of the spsa method inecg analysis. IEEE Transactions on BiomedicalEngineering 49(10), 1094–1101 (2002)


[68] Gittins, J.C.: Multi-armed bandit allocationindices. Wiley-Interscience Series in Systemsand Optimization. John Wiley & Sons (1989)

[69] Glasserman, P.: Gradient Estimation viaPerturbation Analysis. Kluwer AcademicPublishers (1991)

[70] Glover, F.: Tabu search: A tutorial. Interfaces20(4), 77–94 (1990)

[71] Glover, F., Hanafi, S.: Tabu search and finiteconvergence. Discrete Applied Mathematics119(1–2), 3–36 (2002)

[72] Glover, F., Laguna, M.: Tabu Search. KluwerAcademic Publishers, Boston (1997)

[73] Glover, F., Laguna, M.: Fundamentals ofscatter search and path relinking. Control andCybernetics 29(3), 653–684 (2000)

[74] Goldsman, D., Nelson, B.L.: Comparingsystems via simulation. In: J. Banks(ed.) Handbook of Simulation: Principles,Methodology, Advances, Applications, andPractice, chap. 8. John Wiley & Sons (1998)

[75] Gong, W.B., Ho, Y.C., Zhai, W.: Stochasticcomparison algorithm for discrete optimizationwith estimation. SIAM Journal on Optimiza-tion 10, 384–404 (49)

[76] Griewank, A., Walther, A.: EvaluatingDerivatives: Principles and Techniques ofAlgorithmic Differentiation, 2nd edn. No. 105in Other Titles in Applied Mathematics. SIAM,Philadelphia, PA (2008). URL http://www.ec-securehost.com/SIAM/OT105.html

[77] Gurkan, G., Ozge, A.Y., Robinson, S.M.:Sample path optimization in simulation. In:J. Tew, S. Manivannan, D. Sadowski, A. Seila(eds.) Proceedings of 1994 Winter SimulationConference, pp. 247–254 (1994)

[78] Hajek, B.: Cooling schedules for optimal an-nealing. Mathematics of Operations Research13, 311–329 (1988)

[79] Hall, J.D., Bowden, R.O., Usher, J.M.: Usingevolution strategies and simulation to optimizea pull production system. Journal of MaterialsProcessing Technology 61(1–2), 47–52 (1996)

[80] Hansen, N.: The CMA Evolution Strategy: Atutorial.http://www.lri.fr/~hansen/cmaesintro.html

[81] Hansen, N.: The CMA evolution strategy:A comparing review. In: J.A. Lozano,P. Larranaga, I. Inza, E. Bengoetxea (eds.)Towards a new evolutionary computation.Advances on estimation of distribution algo-rithms, pp. 75–102. Springer (2006)

[82] Healy, K., Schruben, L.W.: Retrospectivesimulation response optimization. In: B.L.Nelson, D.W. Kelton, G.M. Clark (eds.)Proceedings of the 1991 Winter SimulationConference, pp. 954–957 (1991)

[83] Hill, S.D., Fu, M.C.: Transfer optimization viasimultaneous perturbation stochastic approxi-mation. In: C. Alexopoulos, K. Kang, W.R.Lilegdon, D. Goldsman (eds.) Proceedings ofthe 1995 Winter Simulation Conference, pp.242–249 (1995)

[84] Ho, Y.C.: An explanation of ordinal opti-mization: Soft computing for hard problems.Information Sciences 113, 169–192 (1999)

[85] Ho, Y.C., Cao, X.R.: Discrete Event DynamicSystems and Perturbation Analysis. KluwerAcademic Publishers (1991)

[86] Hochberg, Y., Tamhane, A.C.: MultipleComparison Procedures. John Wiley & Sons,New York (1987)

[87] Hong, L.J., Nelson, B.L.: Discrete optimizationvia simulation using COMPASS. OperationsResearch 54(1), 115–129 (2006)

[88] Hong, L.J., Nelson, B.L.: A brief introductionto optimization via simulation. In: M.D.Rossetti, R.R. Hill, B. Johansson, A. Dunkin,R.G. Ingalls (eds.) Proceedings of the 2009Winter Simulation Conference (2009)

[89] Hooke, R., Jeeves, T.A.: Direct search solutionof numerical and statistical problems. Journalof the Association for Computing Machinery 8,212–219 (1961)

[90] Hsu, J.C.: Multiple Comparisons: theory andmethods. CRC Press, Boca Raton (1996)

[91] Hu, J., Fu, M.C., Marcus, S.I.: Stochasticoptimization using model reference adaptivesearch. In: M.E. Kuhl, N.M. Steiger, F.B.Armstrong, J.A. Joines (eds.) Proceedings ofthe 2005 Winter Simulation Conference, pp.811–818 (2005)

[92] Hu, J., Fu, M.C., Marcus, S.I.: A modelreference adaptive search method for globaloptimization. Operations Research 55(3), 549–568 (2007)

http://www.ec-securehost.com/SIAM/OT105.html

http://www.ec-securehost.com/SIAM/OT105.html

http://www.lri.fr/~hansen/cmaesintro.html

http://www.lri.fr/~hansen/cmaesintro.html


[93] Huang, D., Allen, T.T., Notz, W.I., Zeng,N.: Global optimization of stochastic black-boxsystems via sequential kriging meta-models.Journal of Global Optimization 34, 441–466(2006)

[94] Humphrey, D.G., Wilson, J.R.: A revisedsimplex search procedure for stochastic simula-tion response-surface optimization. INFORMSJournal on Computing 12(4), 272–283 (2000)

[95] Hunter, S.R., Pasupathy, R.: Optimalsampling laws for stochastically constrainedsimulation optimization on finite sets. IN-FORMS Journal on Computing 25(3), 527–542(2013)

[96] Hutchison, D.W., Hill, S.D.: Simulationoptimization of airline delay with constraints.In: B.A. Peters, J.S. Smith, D.J. Medeiros,M.W. Rohrer (eds.) Proceedings of the 2001Winter Simulation Conference, pp. 1017–1022(2001)

[97] Huyer, W., Neumaier, A.: SNOBFIT – Stablenoisy optimization by branch and fit. ACMTransactions on Mathematical Software 35, 1–25 (2008)

[98] Irizarry, M.D.L.A., Wilson, J.R., Trevino, J.:A flexible simulation tool for manufacturing-cell design, II: Response surface analysis andcase study. IIE Transactions (2001)

[99] Jacobson, S.H., Schruben, L.W.: Techniquesfor simulation response optimization. Opera-tion Research Letters 8, 1–9 (1989)

[100] Jia, Q.S., Ho, Y.C., Zhao, Q.C.: Comparisonof selection rules for ordinal optimization.Mathematical and Computer Modelling 43(9–10), 1150–1171 (2006)

[101] Jones, D.R., Perttunen, C.D., Stuckman,B.E.: Lipschitzian optimization without theLipschitz constant. Journal of OptimizationTheory and Application 79, 157–181 (1993)

[102] Jones, D.R., Schonlau, M., Welch, W.J.:Efficient global optimization of expensiveblack-box functions. Journal of GlobalOptimization 13, 455–492 (1998)

[103] Jung, J.Y., Blau, G., Pekny, J.F., Reklaitis,G.V., Eversdyk, D.: A simulation basedoptimization approach to supply chain manage-ment under demand uncertainty. Computers &Chemical Engineering 28, 2087–2106 (2004)

[104] Kabirian, A.: Continuous optimization viasimulation using golden region search. Ph.D.thesis, Iowa State University (2009)

[105] Kabirian, A., Olafsson, S.: Allocation ofsimulation runs for simulation optimization.In: S.G. Henderson, B. Biller, M.H. Hsieh,J. Shortle, J.D. Tew, R.R. Barton (eds.)Proceedings of the 2007 Winter SimulationConference, pp. 363–371 (2007)

[106] Kabirian, A., Olafsson, S.: Continuousoptimization via simulation using golden regionsearch. European Journal of OperationalResearch (2011)

[107] Kenne, J.P., Gharbi, A.: A simulationoptimization approach in production planningof failure prone manufacturing systems. Jour-nal of Intelligent Manufacturing 12, 421–431(2001)

[108] Khan, H.A., Zhang, Y., Ji, C., Stevens,C.J., Edwards, D.J., O’Brien, D.: Optimizingpolyphase sequences for orthogonal nettedradar. Signal Processing Letters, IEEE 13(10),589–592 (2006)

[109] Kiefer, J., Wolfowitz, J.: Stochastic estimationof the maximum of a regression function. TheAnnals of Mathematical Statistics 23(3), 462–466 (1952)

[110] Kim, S.H.: Comparison with a standard viafully sequential procedures. ACM Transactionson Modeling and Computer Simulation 15(2),155–174 (2005)

[111] Kim, S.H., Nelson, B.L.: Selecting the bestsystem. In: S.G. Henderson, B.L. Nelson(eds.) Handbooks in Operations Research andManagement Science: Simulation, chap. 17, pp.501–534. Elsevier Science (2006)

[112] Kim, S.H., Nelson, B.L.: Recent advances inranking and simulation. In: S.G. Henderson,B. Biller, M.H. Hsieh, J. Shortle, J.D. Tew,R.R. Barton (eds.) Proceedings of the 2007Winter Simulation Conference, pp. 162–172(2007)

[113] Kirkpatrick, S., Gelatt, C.D., Vecchi, M.P.:Optimization by simulated annealing. Science220, 671–680 (1983)

[114] Kleijnen, J.P.C.: Simulation and optimizationin production planning: A case study. DecisionSupport Systems 9, 269–280 (1993)


[115] Kleijnen, J.P.C.: Design and Analysis ofSimulation Experiments. Springer, New York(2008)

[116] Kleijnen, J.P.C.: Kriging metamodeling insimulation: A review. European Journal ofOperational Research 192(3), 707–716 (2009)

[117] Kleijnen, J.P.C., van Beers, W.C.M.: Robust-ness of kriging when interpolating in randomsimulation with heterogeneous variances: Someexperiments. European Journal of OperationalResearch 165, 826–834 (2005)

[118] Kleijnen, J.P.C., Beers, W.C.M., van Nieuwen-huyse, I.: Expected improvement in efficientglobal optimization through bootstrappedkriging. Journal of Global Optimization 54(1),59–73 (2012)

[119] Kleinman, N.L., Hill, S.D., Ilenda, V.A.:Spsa/simmod optimization of air traffic delaycost. In: Proceedings of the 1997 AmericanControl Conference, vol. 2, pp. 1121–1125(1997)

[120] Kochel, P., Nielander, U.: Simulation-based optimisation of multi-echelon inventorysystems. International Journal of ProductionEconomics 93–94, 505–513 (2005)

[121] Kolda, T.G., Lewis, R.M., Torczon, V.J.: Op-timization by direct search: New perspectiveson some classical and modern methods. SIAMReview 45, 385–482 (2003)

[122] Kothandaraman, G., Rotea, M.A.:Simultaneous-perturbation-stochastic-approximation algorithm for parachuteparameter estimation. Journal of Aircraft42(5), 1229–1235 (2005)

[123] Kroese, D.P., Hui, K.P., Nariai, S.: Networkreliability optimization via the cross-entropymethod. Reliability, IEEE Transactions on56(2), 275–287 (2007)

[124] Kroese, D.P., Porotsky, S., Rubinstein, R.Y.:The cross-entropy method for continuousmulti-extremal optimization. Methodology andComputing in Applied Probability 8(3), 383–407 (2006)

[125] Kulturel-Konak, S., Konak, A.: Simulationoptimization embedded particle swarm opti-mization for reliable server assignment. In:B. Johansson, S. Jain, J. Montoya-Torres,J. Hugan, E. Yucesan (eds.) Proceedings of the2010 Winter Simulation Conference, pp. 2897–2906 (2010)

[126] Larranaga, P., Lozano, J.A.: Estimationof distribution algorithms: A new tool forevolutionary computation. Kluwer AcademicPublishers (2002)

[127] Lau, T.W.E., Ho, Y.C.: Universal alignmentprobabilities and subset selection for ordinaloptimization. Journal of Optimization Theoryand Applications 93(3), 455–489 (1997)

[128] Law, A.M., Kelton, W.D.: Simulationmodeling and analysis, 3rd edn. McGraw-Hill,Singapore (2000)

[129] Lee, L.H., Pujowidianto, N.A., Li, L.W., Chen,C.H., Yap, C.M.: Approximate simulationbudget allocation for selecting the best designin the presence of stochastic constraints. IEEETransactions on Automatic Control 57(11),2940–2945 (2012)

[130] Li, Y.: A simulation-based evolutionaryapproach to LNA circuit design optimiza-tion. Applied Mathematics and Computation209(1), 57–67 (2009)

[131] Lucidi, S., Sciandrone, M.: On the globalconvergence of derivative-free methods forunconstrained minimization. SIAM Journal onOptimization 13, 97–116 (2002)

[132] Lutz, C.M., Davis, K.R., Sun, M.: Determiningbuffer location and size in production linesusing tabu search. European Journal ofOperational Research 106, 301–316 (1998)

[133] Martı, R., Laguna, M., Glover, F.: Principlesof scatter search. European Journal ofOperational Research 169(2), 359–372 (2006)

[134] Maryak, J.L., Chin, D.C.: Global randomoptimization by simulataneous perturbationstochastic approximation. IEEE Transactionson Automatic Control 53, 780–783 (2008)

[135] Meketon, M.S.: Optimization in simulation:A survey of recent results. In: A. Thesen,H. Grant, W.D. Kelton (eds.) Proceedings ofthe 1987 Winter Simulation Conference, pp.58–67 (1987)

[136] Merhof, D., Soza, G., Stadlbauer, A., Greiner,G., Nimsky, C.: Correction of susceptibilityartifacts in diffusion tensor data using non-linear registration. Medical Image Analysis11(6), 588–603 (2007)

[137] Merton, R.C.: On the pricing of corporatedebt: The risk structure of interest rates. TheJournal of Finance 29(2), 449–470 (1974)


[138] Mishra, V., Bhatnagar, S., Hemachandra, N.:Discrete parameter simulation optimizationalgorithms with applications to admissioncontrol with dependent service times. In:Proceedings of the 46th IEEE Conference onDecision and Control, pp. 2986–2991. NewOrleans, LA (2007)

[139] Mockus, J.: Bayesian Approach to GlobalOptimization. Kluwer Academic Publishers(1989)

[140] Mockus, J., Tiesis, V., Zilinskas, A.: TowardsGlobal Optimisation, vol. 2, chap. Theapplication of Bayesian Methods for Seekingthe Extremum, pp. 117–128. North-Holland,Amsterdam (1978)

[141] More, J., Wild, S.: Benchmarking derivative-free optimization algorithms. SIAM Journal onOptimization 20, 172–191 (2009)

[142] Myers, R.H., Montgomery, D.C., Anderson-Cook, C.M.: Response Surface Methodology:Process and Product Optimization UsingDesigned Experiments. Wiley Series inProbability and Statistics. John Wiley & Sons(2009)

[143] Neddermeijer, H.G., Oortmarssen, G.J.V.,Piersma, N., Dekker, R.: A framework forresponse surface methodology for simulationoptimization. In: J.A. Joines, R.R. Barton,K. Kang, P.A. Fishwick (eds.) Proceedings ofthe 2000 Winter Simulation Conference, pp.129–136 (2000)

[144] Nelder, J.A., Mead, R.: A simplex method forfunction minimization. Computer Journal 7,308–313 (1965)

[145] Nelson, B.L.: Optimization via simulationover discrete decision variables. Tutorials inoperations research 7, 193–207 (2010)

[146] Nelson, B.L., Goldsman, D.: Comparisonswith a standard in simulation experiments.Management Science 47(3), 449–463 (2001)

[147] Nicolai, R., Dekker, R.: Automated responsesurface methodology for simulation optimiza-tion models with unknown variance. QualityTechnology and Quantitative Management6(3), 325–352 (2009)

[148] O. Yalcinkaya, G.M.B.: Modelling andoptimization of average travel time for ametro line by simulation and response surfacemethodology. European Journal of OperationalResearch 196, 225–233 (2009)

[149] Olafsson, S.: Metaheuristics. In: S. Henderson,B. Nelson (eds.) Handbook in OperationsResearch and Management Science: Simulation13, pp. 633–654. Elsevier (2006)

[150] http://www.orms-today.org/surveys/Simulation/Simulation.html

[151] Osorio, C., Bierlaire, M.: A simulation-basedoptimization approach to perform urban trafficcontrol. In: Proceedings of the TriennialSymposium on Transportation Analysis (2010)

[152] Pasupathy, R., Ghosh, S.: Simulationoptimization: A concise overview and im-plementation guide. Tutorials in operationsresearch 10, 122–150 (2013)

[153] Pasupathy, R., Henderson, S.G.: SIMOPT:A library of simulation-optimization problems.In: S. Jain, R.R. Creasey, J. Himmelspach,K.P. White, e. M. Fu (eds.) Proceedings of the2011 Winter Simulation Conference (2011)

[154] Pasupathy, R., Kim, S.: The stochastic rootfinding problem: Overview, solutions, and openquestions. ACM Transactions on Modeling andComputer Simulation 21(3), 19:1–19:23 (2011)

[155] Peters, J., Vijayakumar, S., Schaal, S.:Reinforcement learning for humanoid robotics.In: Third IEEE-RAS International Conferenceon Humanoid Robots, pp. 1–20. Karlsruhe,Germany (2003)

[156] Pflug, G.C.: Optimization of stochasticmodels: The interface between simulation andoptimization. Kluwer Academic Publishers(1996)

[157] Powell, W.B.: http://www.castlelab.princeton.edu/cso.htm (accessed October 23,2013)

[158] Powell, W.B., Ryzhov, I.O.: Optimal Learning.John Wiley & Sons (2012)

[159] Prakash, P., Deng, G., Converse, M.C.,Webster, J.G., Mahvi, D.M., Ferris, M.C.:Design optimization of a robust sleeve antennafor hepatic microwave ablation. Physics inMedicine and Biology 53, 1057–1069 (2008)

[160] Radac, M.B., Precup, R.E., Petriu, E.M.,Preitl, S.: Application of ift and spsa to servosystem control. IEEE Transactions on NeuralNetworks 22(12), 2363–2375 (2011)

http://www.orms-today.org/surveys/Simulation/Simulation.html

http://www.orms-today.org/surveys/Simulation/Simulation.html

http://www.castlelab.princeton.edu/cso.htm

http://www.castlelab.princeton.edu/cso.htm


[161] Rall, L.B.: Automatic Differentiation: Tech-niques and Applications, Lecture Notes in

Computer Science, vol. 120. Springer, Berlin(1981). DOI 10.1007/3-540-10861-0

[162] Ramanathan, S.P., Mukherjee, S., Dahule,R.K., Ghosh, S., Rahman, I., Tambe, S.S.,Ravetkar, D.D., Kulkarni, B.D.: Optimiza-tion of continuous distillation columns usingstochastic optimization approaches. Transac-tions of the Institution of Chemical Engineers79, 310–322 (2001)

[163] Rasmussen, C.E., Williams, C.K.I.: GaussianProcesses for Machine Learning. The MITPress (2006)

[164] Reeves, C.R.: Genetic algorithms for theoperations researcher. INFORMS Journal onComputing 9(3), 231–250 (1997)

[165] Renotte, C., Vande Wouwer, A.: Stochasticapproximation techniques applied to parameterestimation in a biological model. In: IntelligentData Acquisition and Advanced ComputingSystems: Technology and Applications, 2003.Proceedings of the Second IEEE InternationalWorkshop on, pp. 261–265. IEEE (2003)

[166] Rios, L.M., Sahinidis, N.V.: Derivative-free optimization: A review of algorithmsand comparison of software implementations.Journal of Global Optimization 56, 1247–1293(2013)

[167] Robbins, H., Monro, S.: A stochasticapproximation method. The Annals ofMathematical Statistics 22(3), 400–407 (1951)

[168] Romero, P.A., Krause, A., Arnold, F.H.:Navigating the protein fitness landscape withgaussian processes. Proceedings of the NationalAcademy of Sciences (PNAS) 110(3) (2013).DOI 10.1073/pnas.1215251110

[169] Roustant, O., Ginsbourger, D., Deville, Y.:DiceKriging, DiceOptim: Two R packagesfor the analysis of computer experiments bykriging-based metamodeling and optimization.Journal of Statistical Software 51(1), 1–55(2012)

[170] Rubinstein, R.: The cross-entropy methodfor combinatorial and continuous optimization.Methodology and Computing in AppliedProbability 1, 127–190 (1999)

[171] Rubinstein, R.Y., Kroese, D.P.: The Cross-Entropy Method: A unified approach to combi-natorial optimization, Monte-Carlo simulation,

and machine learning. Springer-Verlag, NewYork (2004)

[172] Rubinstein, R.Y., Shapiro, A.: Discrete eventsystems: Sensitivity analysis and stochasticoptimization by the score function method.John Wiley & Sons (1993)

[173] Sacks, J., Schiller, S.B., Welch, W.J.: Designsfor computer experiments. Technometrics 31,41–47 (1989)

[174] Safizadeh, M.H.: Optimization in simulation:Current issues and the future outlook. NavalResearch Logistics 37, 807–825 (1990)

[175] Sahinidis, N.V.: Optimization under uncer-tainty: State-of-the-art and opportunities.Computers & Chemical Engineering 28(6–7),971–983 (2004)

[176] Schwartz, J.D., Wang, W., Rivera, D.E.:Simulation-based optimization of process con-trol policies for inventory management insupply chains. Automatica 42, 1311–1320(2006)

[177] Scott, W., Frazier, P.I., Powell, W.: Thecorrelated knowledge gradient for simulationoptimization of continuous parameters usinggaussian process regression. SIAM Journal onOptimization 21(3), 996–1026 (2011)

[178] Settles, B.: Active learning literature survey.Tech. rep., University of Wisconsin–Madison(2010)

[179] Shapiro, A.: Asymptotic analysis of stochasticprograms. Annals of Operations Research 30,169–186 (1991)

[180] Shapiro, A.: Simulation based optimization.In: J.M. Charnes, D.J. Morrice, D.T. Brunner,J.J. Swain (eds.) Proceedings of the 1996Winter Simulation Conference, pp. 332–336(1996)

[181] Shi, L., Olafsson: Nested Partitions Op-timization: Methodology and Applications,International Series in Operations Research &

Management Science, vol. 109. Springer (2007)

[182] Shi, L., Olafsson, S.: Nested partitions methodfor stochastic optimization. Methodology andComputing in Applied Probability 2, 271–291(2000)

[183] Song, Y., Grizzle, J.W.: The extendedkalman filter as a local asymptotic observerfor discrete-time nonlinear systems. Journal


of Mathematical Systems, Estimation, andControl 5(1), 59–78 (1995)

[184] Spall, J.C.: Multivariate stochastic approx-imation using a simultaneous perturbationgradient approximation. IEEE Transactions onAutomatic Control 37, 332–341 (1992)

[185] Spall, J.C.: Chapter 7: SimultaneousPerturbation Stochastic Approximation. In:Introduction to stochastic search and opti-mization: Estimation, simulation, and control.Wiley-Interscience (2003)

[186] Spall, J.C.: Introduction to stochastic searchand optimization: Estimation, simulation, andcontrol. Wiley-Interscience (2003)

[187] Spall, J.C.: Feedback and weighting mecha-nisms for improving Jacobian estimates in theadaptive simultaneous perturbation algorithm.IEEE Transactions on Automatic Control54(6), 1216–1229 (2009)

[188] Spall, J.C.: Stochastic optimization. In: J.E.Gentle, W.K. Hardle, Y. Mori (eds.) Handbookof Computational Statistics: Concepts andMethods, second edn., chap. 7, pp. 173–201.Springer (2012)

[189] Srinivas, N., Krause, A., Kakade, S.M., Seeger,M.: Information-theoretic regret bounds forgaussian process optimization in the banditsetting. IEEE Transactions on InformationTheory 58(5), 3250–3265 (2012)

[190] Stephens, C.P., Baritompa, W.: Global opti-mization requires global information. Journalof Optimization Theory and Applications 96,575–588 (1998)

[191] Swisher, J.R., Hyden, P.D., Jacobson, S.H.,Schruben, L.W.: A survey of simulationoptimization techniques and procedures. In:J.A. Joines, R.R. Barton, K. Kang, P.A.Fishwick (eds.) Proceedings of the 2000 WinterSimulation Conference (2000)

[192] Syberfeldt, A., Lidberg, S.: Real-worldsimulation-based manufacturing optimizationusing cuckoo search. In: C. Laroque,J. Himmelspach, R. Pasupathy, O. Rose,A. Uhrmacher (eds.) Proceedings of the 2012Winter Simulation Conference (2012)

[193] Tein, L.H., Ramli, R.: Recent advancementsof nurse scheduling models and a potentialpath. In: Proceedings of the 6th IMT-GTConference on Mathematics, Statistics and itsApplications, pp. 395–409 (2010)

[194] Tekin, E., Sabuncuoglu, I.: Simulationoptimization: A comprehensive review ontheory and applications. IIE Transactions 36,1067–1081 (2004)

[195] Teng, S., Lee, L.H., Chew, E.P.: Multi-objective ordinal optimization for simulationoptimization problems. Automatica 43(11),1884–1895 (2007)

[196] Trosset, M.W.: On the use of direct searchmethods for stochastic optimization. Tech.rep., Rice University, Houston, TX (2000)

[197] Vande Wouwer, A., Renotte, ., Bogaerts, P.,Remy, M.: Application of spsa techniques innonlinear system identification. In: Proceed-ings of the European Control Conference, p.2835 (2001)

[198] Wang, H., Pasupathy, R., Schmeiser, B.W.:Integer-ordered simulation optimization us-ing R-SPLINE: Retrospective search withpiecewise-linear interpolation and neighbor-hood enumeration. ACM Transactions onModeling and Computer Simulation 23, 17:1–17:24 (2012)

[199] Wang, Q., Spall, J.C.: Discrete simultaneousperturbation stochastic approximation on lossfunctions with noisy measurements. In: Pro-ceedings of the American Control Conference,pp. 4520–4525. IEEE, San Francisco (2011)

[200] Whitley, D.: A genetic algorithm tutorial.Statistics and Computing 4, 65–85 (1994)

[201] Xie, J., Frazier, P.I.: Sequential bayes-optimalpolicies for multiple comparisons with a knownstandard. Operations Research 61(5), 1174–1189 (2013)

[202] Xie, J., Frazier, P.I., Sankaran, S., Mars-den, A., Elmohamed, S.: Optimization ofcomputationally expensive simulations withgaussian processes and parameter uncertainty:Application to cardiovascular surgery. In: 50thAnnual Allerton Conference on Communica-tion, Control, and Computing (2012)

[203] Xing, X.Q., Damodaran, M.: Assessment ofsimultaneous perturbation stochastic approxi-mation method for wing design optimization.Journal of Aircraft 39, 379–381 (2002)

[204] Xing, X.Q., Damodaran, M.: Application ofsimultaneous perturbation stochastic approxi-mation method for aerodynamic shape designoptimization. AIAA Journal 43(2), 284–294(2005)


[205] Xing, X.Q., Damodaran, M.: Inverse designof transonic airfoils using parallel simultaneousperturbation stochastic approximation. Jour-nal of Aircraft 42(2), 568–570 (2005)

[206] Xu, J., Nelson, B.L., Hong, L.J.: Indus-trial strength COMPASS: A comprehensivealgorithm and software for optimization viasimulation. ACM Transactions on Modelingand Computer Simulation 20(1), 1–29 (2010)

[207] Xu, J., Nelson, B.L., Hong, L.J.: An adaptivehypberbox algorithm for high-dimensionaldiscrete optimization via simulation problems.INFORMS Journal on Computing 25(1), 133–146 (2013)

[208] Yan, D., Mukai, H.: Stochastic discreteoptimization. SIAM Journal on Control andOptimization 30, 594–612 (1992)

[209] Yang, X.S., Deb, S.: Engineering optimisationby cuckoo search. International Journalof Mathematical Modelling and NumericalOptimisation 1(4), 330–343 (2010)

[210] Yeomans, J.S.: Solid waste planning underuncertainty using evolutionary simulation-optimization. Socio-Economic Planning Sci-ences 41, 38–60 (2007)

[211] Yun, I., Park, B.: Application of stochasticoptimization method for an urban corridor.In: L.F. Perrone, F.P. Wieland, J. Liu, B.G.Lawson, D.M. Nicol, R.M. Fujimoto (eds.)Proceedings of the 2010 Winter SimulationConference, pp. 1493–1499 (2010)

Documents

Simulation optimization: A review of algorithms and ... · Simulation optimization: A review of ... excellent tutorial on simulation optimization, and ... and also talk about estimation