23
Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE CS Meeting, April 15, 1999 (c) Linden Innovation Research LLC 1 Designing Like Mother Nature An Introduction to Genetic Algorithms Dr. Derek S. Linden (703) 561-3400 [email protected] 2 About the Speaker, Derek Linden B.S., USAF Academy, 1991, Applied Physics (Elec. Systems) M.S., MIT, 1993, EE (Solid State Devices/Superconductivity) Rome Lab, 1993 - 1996, basic research on superconductors at microwave frequencies, antennas, GAs Ph.D., MIT, 1997, Thesis: “Automated Design and Optimization of Wire Antennas Using Genetic Algorithms” Current research: Increasing GA efficiency, applying GAs to new problems Linden Innovation Research LLC Automated Design and Optimization Consulting, Training and Software

Designing Like Mother Nature - IEEEewh.ieee.org/r2/wash_nova/computer/archives/ieee/archives/apr99.pdf · Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE

Embed Size (px)

Citation preview

Page 1: Designing Like Mother Nature - IEEEewh.ieee.org/r2/wash_nova/computer/archives/ieee/archives/apr99.pdf · Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE

Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE CS Meeting, April 15, 1999

(c) Linden Innovation Research LLC 1

Designing Like Mother NatureAn Introduction to Genetic Algorithms

Dr. Derek S. Linden(703) 561-3400 [email protected]

2

About the Speaker, Derek Linden

• B.S., USAF Academy, 1991, Applied Physics (Elec. Systems)

• M.S., MIT, 1993, EE (Solid State Devices/Superconductivity)

• Rome Lab, 1993 - 1996, basic research on superconductors atmicrowave frequencies, antennas, GAs

• Ph.D., MIT, 1997, Thesis: “Automated Design andOptimization of Wire Antennas Using Genetic Algorithms”

• Current research: Increasing GA efficiency, applying GAs tonew problems

• Linden Innovation Research LLC

– Automated Design and Optimization Consulting, Trainingand Software

Page 2: Designing Like Mother Nature - IEEEewh.ieee.org/r2/wash_nova/computer/archives/ieee/archives/apr99.pdf · Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE

Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE CS Meeting, April 15, 1999

(c) Linden Innovation Research LLC 2

3

Introduction to GAs: Overview

• Goal: to introduce the fundamental concepts of a GA– What is a GA?

– GA basics

– Examples

4

What is a GA?

• A probabilistic, iterative search and optimizationstrategy

• Mimics biological intra-species adaptation andevolution through mating and survival-of-the-fittest

• Finds optima for many types of numerical problems

• Requires:– A coding strategy

– An objective function

– A mating and mutation scheme

Page 3: Designing Like Mother Nature - IEEEewh.ieee.org/r2/wash_nova/computer/archives/ieee/archives/apr99.pdf · Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE

Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE CS Meeting, April 15, 1999

(c) Linden Innovation Research LLC 3

5

The GA Iterative Process

Choose mates andcreate children

Mutate children

Initialize newpopulation

Simulate and evaluatenew members

Rank-order allmembers

Is convergencecriteria met?

Output results

YES

NO

6

GA Terms

0 0 1 1 1 0 1 0 1 0 0.546 0.010 0.530 0.223 0.750 0.456

0 0 1 1 0 0 1 1 0 0 0.754 0.122 0.822 0.564 0.438 0.990

0 1 1 1 0 1 1 0 0 1 0.945 0.678 0.800 0.442 0.901 0.198

1 0 1 0 1 1 0 0 1 0 0.248 0.548 0.401 0.881 0.058 0.451

1 1 1 0 0 1 1 1 0 0 0.700 0.890 0.540 0.111 0.878 0.002

Alleles

Genes

Population

Chromosomes

Page 4: Designing Like Mother Nature - IEEEewh.ieee.org/r2/wash_nova/computer/archives/ieee/archives/apr99.pdf · Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE

Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE CS Meeting, April 15, 1999

(c) Linden Innovation Research LLC 4

7

An Example Design Problem

h

d

Mt

4 Variables, with Constraints:Material: ceramic, glass, plasticDiameter: 2”-5”Height: 3”-6”Thickness: 0.1”-0.5”

Dependent constraint:Weight < 1.5 lbs.

Optimize for:Heat Retention = f(M,d,h,t)Cost = f(M,d,h,t)Volume = f(d,h)

8

Setup for GA Optimization

h

d

Mt

Objective Function = Heat Retention + Volume - Cost - Penalty * Weight

(Penalty is non-zero only if Weight above 1.5 lbs.)

HeightDiameter ThicknessMaterial

01 1010 0100 1101

Glass 4.0” 2.8” 0.447”

Chromosome:

Page 5: Designing Like Mother Nature - IEEEewh.ieee.org/r2/wash_nova/computer/archives/ieee/archives/apr99.pdf · Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE

Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE CS Meeting, April 15, 1999

(c) Linden Innovation Research LLC 5

9

Snapshot During the GA Process

P

G

C P

G

C

10

Brief History

• Before the GA, algorithms based on mutation weretried

• John Holland (University of Michigan)– Holland had the basic GA by the mid-1960s

– Monograph in 1975—“Adaptation in Natural andArtificial Systems”

– Purpose: to understand adaptive processes in naturalsystems and design artificial systems that mimicnatural system behavior

• David Goldberg—textbook in 1989

Page 6: Designing Like Mother Nature - IEEEewh.ieee.org/r2/wash_nova/computer/archives/ieee/archives/apr99.pdf · Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE

Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE CS Meeting, April 15, 1999

(c) Linden Innovation Research LLC 6

11

Current Areas of Application

• Mechanical Engineering

• Software Design

• Electromagnetics

• Electrostatics

• Artificial Intelligence/Artificial Life

• Robotics

• Aeronautical Engineering

• Financial

12

The GA Process

• Set up simulator/equations to evaluate members of population

• Define problem—constraints, unknowns, variables

• Determine objective function

• Determine chromosome mapping

• Determine genetic algorithm characteristics– mating selection, crossover, mutation, population size, etc.

• Run the GA optimization process

• Output the optimal design characteristics

Page 7: Designing Like Mother Nature - IEEEewh.ieee.org/r2/wash_nova/computer/archives/ieee/archives/apr99.pdf · Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE

Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE CS Meeting, April 15, 1999

(c) Linden Innovation Research LLC 7

13

Objective Function

• Gives a single score based on simulation results

• Used to rank-order the members of the population

• Single criteria or multi-criteria

• Include any penalty terms for violating constraints

Fitness = -c1 * gain + c2 * mismatch + c3 * distortion+ c4 * (amount of power violation)

14

1-D Binary and Real Chromosomes

• Binary: 0 0 1 1 1 0 1 0 1 0

– Usually each variable consists of several bits

– Most commonly used by far, good for most problems

• Real: 0.546 0.010 0.530 0.223 0.750 0.456 0.555

– Usually each variable consists of only one number

– Use for problems involving mostly real, continuousvariables

Page 8: Designing Like Mother Nature - IEEEewh.ieee.org/r2/wash_nova/computer/archives/ieee/archives/apr99.pdf · Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE

Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE CS Meeting, April 15, 1999

(c) Linden Innovation Research LLC 8

15

Chromosome Mapping: Example

GroundPlane

(0.05 - 0.50)λ

(0.01 - 0.10)λ

(0.05 - 0.50)λ

(0.05 - 0.50)λ

(0.01 - 0.10)λ

Z4

Z1

Z2Z3 X1

X2

(0.03 - 0.35)λ

00000 11111 00000 11111 00000 11111 Z1 Z2 Z3 Z4 X1 X2

0.452 0.335 0.102 0.873 0.525 0.651

Binary 1-D

Real 1-D

16

Mating Process

• The basic mating process:– Eliminate poor performers (total population remains

constant)

– Choose chromosomes to mate

– Create offspring

• Simple GA: replaces whole population with newchildren, though some are copies of parents

• Steady-State GA: saves a portion of the populationeach generation

• Elitist: saves top chromosome

Page 9: Designing Like Mother Nature - IEEEewh.ieee.org/r2/wash_nova/computer/archives/ieee/archives/apr99.pdf · Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE

Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE CS Meeting, April 15, 1999

(c) Linden Innovation Research LLC 9

17

• In biology, mates are chosen through natural selection– Brightest flower, strongest male, most attractive call

• Most common GA method: weighted roulette wheel

• Usually weighted by fitness, or qualities like similarity

Mating Selection

Spin 1: First Parent

Spin 2: Second Parent

1 = Most fit member

10 = Least fit member1

23

4

56 7 8 9 10

18

Concept: Crossover

A ba B

c

C

A bA

Baa

c

C

Ab

A B

a b

a B

cc

C

C

A b c

a b c

A B C

a B C

Chiasma

Page 10: Designing Like Mother Nature - IEEEewh.ieee.org/r2/wash_nova/computer/archives/ieee/archives/apr99.pdf · Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE

Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE CS Meeting, April 15, 1999

(c) Linden Innovation Research LLC 10

19

1-D Binary Mating—Single-Point Crossover

• Parent chromosomes:

[00011110]

{11001100}

• Let the crossover point be between the 5th and 6th bit(but could be between any two bits)

• Children: [00011]{100}

{11001}[110]

• Works the same way for real chromosomes, exceptno functional genes are able to be split

20

1-D Real Chromosome Mating

• Heuristic crossover

• Quadratic crossover

• Many other methods existAdewuya, 1996

gene value

;;

12

3fitnesschild

;;

1

2fitness child

gene value

Page 11: Designing Like Mother Nature - IEEEewh.ieee.org/r2/wash_nova/computer/archives/ieee/archives/apr99.pdf · Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE

Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE CS Meeting, April 15, 1999

(c) Linden Innovation Research LLC 11

21

1-D Binary and RealChromosome Mutation

• Binary: Bit flip– Flip a randomly selected 1 0 or 0 1

• Real: Uniform mutation

• Real: Gaussian mutation

lower limit upper limit

lower limit upper limit

22

The GA Parameters

• Population Size: 30 - 10,000(most I’ve heard of: 1,000,000)

• Parent pool size (overlap): 10%-50%

• Probability of mutation: < 2%

• Convergence criteria:– # generations

– # simulations

– non-improvement

– loss of diversity

– when I choose to stop it

Page 12: Designing Like Mother Nature - IEEEewh.ieee.org/r2/wash_nova/computer/archives/ieee/archives/apr99.pdf · Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE

Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE CS Meeting, April 15, 1999

(c) Linden Innovation Research LLC 12

23

Putting the Process Together

Choose mates andcreate children

Mutate children

Initialize newpopulation

Simulate and evaluatenew members

Rank-order allmembers

Is convergencecriteria met?

Output results

YES

NO

24

Typical GA Behavior: Fitness

• Best fitness, Average fitness vs. Generation

0

2000

4000

6000

8000

10000

12000

14000

1 11 21 31 41 51 61 71 81 91

Fitn

ess

Generation

Page 13: Designing Like Mother Nature - IEEEewh.ieee.org/r2/wash_nova/computer/archives/ieee/archives/apr99.pdf · Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE

Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE CS Meeting, April 15, 1999

(c) Linden Innovation Research LLC 13

25

GA Advantages

• Properly implemented, it can lead to optimal solutionsrelatively rapidly and efficiently

• Prevents the solution from getting trapped in local minimathrough parallelism

• Is zeroth-order/blind—requires no information other than theobjective function value for each chromosome

• Can optimize very complicated systems with no humanintervention (not even an initial guess!)

• Very robust to parameters, coding, etc.

• Able to be implemented in a parallel manner GA

Sim 3Sim 2

Sim 1

26

GA Examples

• Discrete problems– Truss topology design

– VLSI connection design

– Job Shop Process Planning

• Continuous problems

– Turbine engine design

– Pattern nesting (Parts layout)

– Simple wire antenna

– Folded monopole & Crooked-wire antennas

Page 14: Designing Like Mother Nature - IEEEewh.ieee.org/r2/wash_nova/computer/archives/ieee/archives/apr99.pdf · Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE

Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE CS Meeting, April 15, 1999

(c) Linden Innovation Research LLC 14

27

Truss Topology Design

• Use a GA to determine an optimal truss structure withthe least amount of material given a load

Chapman et al.,1994

28

Truss Topology Design

• Example optimized designs at differing resolutions

Page 15: Designing Like Mother Nature - IEEEewh.ieee.org/r2/wash_nova/computer/archives/ieee/archives/apr99.pdf · Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE

Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE CS Meeting, April 15, 1999

(c) Linden Innovation Research LLC 15

29

Job Shop Process Planning

• Minimize the cost and hassle in machining custom parts

• Many different combinations of machine, tool, and setup arepossible to create the same part

Zhang, et al.,1997

30

Job Shop Process Planning

Minimize Machine Setup Tool Cost

Machine 0 11 13 2664

Setup 8 0 10 3799

Tool 12 5 0 5014

Cost 1 3 8 1739

Page 16: Designing Like Mother Nature - IEEEewh.ieee.org/r2/wash_nova/computer/archives/ieee/archives/apr99.pdf · Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE

Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE CS Meeting, April 15, 1999

(c) Linden Innovation Research LLC 16

31

VLSI Connection Design

• Rather complex GA technique compared with 2-4 other standardVLSI techniques in each of 10 classic benchmarks

• GA was best method for each benchmark in numbers of vias

• Best for overall wire length in 7 of 10 benchmarks

• 2nd best in the other 3 benchmarks, and usually a close second

• Crosstalk requirements can be added, which none of the othertechniques can handle

Leinig, 1997

32

Turbine Engine Design

At least 100 variables, eachwith a continuous range

Search space of 10387 points

Fitness: compliance withabout 50 constraints +performance measures

Engineer: 8 weeks for a satisfactory designEngineer + Expert system: less than 1 day w/ 2x improvementGA + Expert system: 2 days w/ 3x improvement over engineer alone

Holland, 1992

Pratt&Whitney Engine

Page 17: Designing Like Mother Nature - IEEEewh.ieee.org/r2/wash_nova/computer/archives/ieee/archives/apr99.pdf · Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE

Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE CS Meeting, April 15, 1999

(c) Linden Innovation Research LLC 17

33

Dighe & Jakiela, 1996

Pattern Nesting

• Applications in many industries– Clothing

– Shipbuilding

– Automobile part manufacturing

34

Pattern Nesting

• Minimizing rectangular enclosure

68.4% 69.0%

Page 18: Designing Like Mother Nature - IEEEewh.ieee.org/r2/wash_nova/computer/archives/ieee/archives/apr99.pdf · Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE

Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE CS Meeting, April 15, 1999

(c) Linden Innovation Research LLC 18

35

Pattern Nesting

• Minimizing height

70.4% 72.4%

36

Simple Wire Antenna

• The design

Drive point(in center of element)

Reflector element 0 - 4 λ

Driven element 0.5 λ

Separation distance 0.04 - 2 λ

Linden, 1997

Page 19: Designing Like Mother Nature - IEEEewh.ieee.org/r2/wash_nova/computer/archives/ieee/archives/apr99.pdf · Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE

Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE CS Meeting, April 15, 1999

(c) Linden Innovation Research LLC 19

37

0.02

0.22

0.42

0.62

0.82

1.02

1.22

1.42

1.62

1.82

2.02

2.22

2.42

2.62

2.82

3.02

3.22

3.42

3.62

3.82

0.08

0.28

0.48

0.68

0.88

1.08

1.28

1.48

1.68

1.88

-2

0

2

4

6

8

10

Gain

Length

Separation

Simple Wire Antenna

• The search space

38

Simple Wire Antenna

• The objective function– Maximize gain in forward direction (already a single

number)

• The chromosome– Two real values for length and separation

• GA parameters– 20 chromosomes, 50% overlap, 0.6% mutation

Page 20: Designing Like Mother Nature - IEEEewh.ieee.org/r2/wash_nova/computer/archives/ieee/archives/apr99.pdf · Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE

Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE CS Meeting, April 15, 1999

(c) Linden Innovation Research LLC 20

39

Simple Wire Antenna

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

1 2 5

8 11 14

17 20 23

40

Folded Monopole, Crooked-WireAntennas

• The problem: Our goal in each case was to achieve asingle objective: the broadest beam possible over theupper hemisphere

– Folded monopole — power gain only

– Crooked wire antennas — RH circular polarizationgain

Score = Σover all θ,φ(Gain(θ,φ) - Avg. Gain)2

Page 21: Designing Like Mother Nature - IEEEewh.ieee.org/r2/wash_nova/computer/archives/ieee/archives/apr99.pdf · Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE

Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE CS Meeting, April 15, 1999

(c) Linden Innovation Research LLC 21

41

The Folded Monopole Chromosome

GroundPlane

(0.05 - 0.50)λ

(0.01 - 0.10)λ

(0.05 - 0.50)λ

(0.05 - 0.50)λ

(0.01 - 0.10)λ

Z4

Z1

Z2Z3 X1

X2

(0.03 - 0.35)λ

00000 11111 00000 11111 00000 11111 Z1 Z2 Z3 Z4 X1 X2

Goal: Hemispherical coverage, regardless of polarization

42

Folded Monopole Results

5

0

-10

-5

10

Gain

-70 -50 -30 -10 10 30 50 70θ

f = 1600MHz

-90 90

(dB)

(deg.)

45°(dash)

90°(solid)

φ=0°(dot)

θ

φ

Altshuler & Linden, 1997

Page 22: Designing Like Mother Nature - IEEEewh.ieee.org/r2/wash_nova/computer/archives/ieee/archives/apr99.pdf · Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE

Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE CS Meeting, April 15, 1999

(c) Linden Innovation Research LLC 22

43

Crooked-Wire Genetic AntennaSpace

Goal: Coverage over hemisphere 10 above the horizonwith right-hand circular polarization

0.5λ

0.5λ

0.5λ

Ground

Ground

(X1,Y1,Z1)(X2,Y2,Z2)(X3,Y3,Z3)...(X7,Y7,Z7) 105 bits

44

Crooked-Wire Genetic AntennaResults

-5

-1070

5

0

10

Gain

-70 -50 -30 -10 10 30 50

θ

f = 1600MHz

-90 90

(dB

(deg.)

φ=0°

45°

135° 90°

Linden & Altshuler, 1996

Page 23: Designing Like Mother Nature - IEEEewh.ieee.org/r2/wash_nova/computer/archives/ieee/archives/apr99.pdf · Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE

Designing Like Mother Nature: An Introduction to Genetic Algorithms IEEE CS Meeting, April 15, 1999

(c) Linden Innovation Research LLC 23

45

References

• A. Adewuya, “ New Methods in Genetic Search with Real-valued Chromosomes,” Master'sThesis, Mech. Engr. Dept., MIT, 1996

• C.D. Chapman, K. Saitou, M.J. Jakiela. “Genetic Algorithms as an Approach to Configurationand Topology Design.” Journal of Mechanical Design,Vol. 116, December 1994.

• F. Zhang, Y.F. Zhang, A.Y.C Nee. “Using Genetic Algorithms in Process Planning for JobShop Machining.” IEEE Trans. on Evolutionary Computation, Vol. 1, No. 4, November 1997.

• J. Lienig. “A Parallel Genetic Algorithm for Performance-Driven VLSI Routing.” IEEETrans. on Evolutionary Computation, Vol. 1, No. 1, April 1997.

• J.H. Holland. “Genetic Algorithms.” Scientific American, July 1992.

• R. Dighe and M.J. Jakiela. “Solving Pattern Nesting Problems with Genetic AlgorithmsEmploying Task Decomposition and Contact Detection.” Evolutionary Computation, Vol. 3,No. 3, 1996.

• D.S. Linden. “Automated Design and Optimization of Wire Antennas using GeneticAlgorithms.” Ph.D. Thesis, MIT, September 1997.

• E.E. Altshuler and D.S. Linden. “Design of a Loaded Monopole Having HemisphericalCoverage Using a Genetic Algorithm.” IEEE Trans. on Antennas and Propagation., Vol. 45,No. 1, January 1997.

• D.S. Linden and E.E. Altshuler. “Automating Wire Antenna Design using GeneticAlgorithms.” Microwave Journal, Vol. 39, No. 3, March 1996.