1

Myopic Policies for Budgeted Optimization

with Constrained Experiments

Javad Azimi, Xiaoli Fern, Alan Fern

Oregon State University

AAAI, July 2010

2

Motivation: Electricity Production in a Microbial Fuel Cell (MFC)

AnodeCathode

bac

teri

aOxidation products

(CO2)

Fuel (organic matter)

e-

e-

O2

H2OH+

This is how an MFC works

SEM image of bacteria sp. on Ni nanoparticle enhanced carbon fibers.

Nano-structure of anode significantly impact the electricity production.

We should optimize anode nano-structure to maximize power.

3

Experiment Selection• Experiments are costly and there is a fixed budget.• How to select the best sequence of experiments.

Current Experiments Scientist selects Experiment

Run Experiment

Bayesian Optimization (BO)• Since Running experiment is very expensive we use BO.

• Select one experiment to run at a time based on results of previous experiments.Current Experiments Gaussian Process Surface Select Single Experiment

Run Experiment 4

5

Bayesian Optimization (BO)

• BO assumes that we can ask for specific experiment.• This is unreasonable assumption in many applications.

– In Fuel Cell it takes many trials to create a nano-structure with specific requested properties.

– Costly to fulfill.Space of Experiments

Average Circularity

Ave

rage

d A

rea

6

Constrained Experiments• It is less costly to fulfill a request that specifies ranges for the

nanostructure properties

• E.g. run an experiment with Averaged Area in range r1 and Average Circularity in range r2

• We will call such requests “constrained experiments”Space of Experiments

Average Circularity

Ave

rage

d A

rea

Constrained Experiment 1• large ranges • low cost• high uncertainty about which experiment will be run

Constrained Experiment 2• small ranges• high cost• low uncertainty about which experiment will be run

7

BO for Constrained Experiment

• Given a fixed budget, select the best constrained experiments.

Run Experiment

Current Experiments Gaussian Process Surface Select Single ExperimentSelect Constrained Experiment

8

Constrained Experiment• We generalized BO heuristics to constrained

experiments.

• Two challenges:– How to compute heuristics for constrained experiment?– How to take experimental cost into account?(which has

been ignored by most of the approaches in BO).

9

Standard BO Heuristics• Standard heuristics are statistics of the posterior p(y|x,D) where D is our

current observation.

• Maximum Upper bound Interval (MUI)– Select point with highest 95% upper confidence bound– Purely explorative approach.

• Maximum Probability of Improvement (MPI)– It computes the probability that the output is more than (1+m) times of the best

current observation , m>0. – Explorative and Exploitative.

• Maximum Expected of Improvement (MEI)– Similar to MPI but parameter free– It simply computes the expected amount of improvement after sampling at any

point.

10

Generalizing BO for Constrained Experiment

• Having the posterior distribution of p(y|x,D) and px(.|D) we can calculate the posterior of the output of each constrained experiment which has a closed form solution.

• Therefore we can compute standard BO heuristics for constrained experiments.– There are closed form solution for these heuristics.

Input spaceDiscretization

Level

11

Budgeted Constrained Experiments

• We are limited with Budget B.• Unfortunately heuristics will typically select the smallest and most

costly constrained experiments which is not a good use of budget.

• How can we consider the cost of each constrained experiment in making the decision?– Cost Normalized Policy (CN)– Constraint Minimum Cost Policy(CMC)

-Low uncertainty

-High uncertainty

-Better heuristic value

-Lower heuristic value

-Expensive -Cheap

12

Cost Normalized Policy

• It selects the constrained experiment achieving the highest expected improvement per unit cost.

• We report this approach for MEI policy only.

13

Constraint Minimum Cost Policy (CMC)

• Motivation:1. Approximately maximizes the heuristic value.2. Has expected improvement at least as great as spending

the same amount of budget on random experiments.• Example:

Very expensive: 10 random experiments likely to be better

Selected Constrained experiment

Poor heuristic value: not select due to 1st

condition

Cost=4 random Cost=10 random Cost=5 random

14

Experimental Results(Setup)

• Gaussian process is used as our model with squared exponential kernel.

• Cost function is defined as:

– There is a constant cost for running any constrained experiment plus an additional cost depending on the size of the experiment.

– The value of slope dictates how fast the cost increases as the size of a constrained experiment decreases.

Space of Experiments

r1

r2

15

Experimental Results(Real Applications)

• Two Real data sets:– Fuel Cell:• Fuel Cell electricity generation

– Hydrogene:• Biosolar hydrogen production

16

Experimental Results(Benchmarks Functions)

• 3 popular benchmarks used in BO literature.

Rosenbrock Discontinuous Cosines

17

Overall Performance• The normalized regret of each framework is shown which is

calculated as y*- ymax over Random performance for budget 15 where y* is the highest possible output.

Average regret of each approach over all frameworks.

Different Budget(1)

18

Random

Cosines

Fuel CellReal

Rosenbrock

CMC-MUI

Different Budget(2)

19

CN-MEI

Cosines

Fuel CellReal

Rosenbrock

Different Budget(3)

20

CMC-MPI(0.2)

Cosines

Fuel CellReal

Rosenbrock

21

Different Budget(4)

CMC-MEICosines

Fuel CellReal

Rosenbrock

22

Conclusion

• We introduced a new constrained experiment framework which asks for hyper-rectangle rather than exact point.

• We extended model-free BO heuristics to our frame work.

• We introduced two approaches to optimize our budgeted framework.

• CMC-MEI is working better than other approaches.

23

Thanks for attendance

Question?