MULTIPLE GRADIENT DESCENT ALGORITHM (MGDA ...Engineering, Vienna (Austria), September 10-14, 2012 TO16: Computational inverse problems and optimization 1/102 MGDA applied to compressible

MGDA applied tocompressible aerodynamics

J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities

Mathematical test-cases

Aircraft-Wing Aerodynamics

Automobile Cooling SystemDesign

Problem description and numericaltools

Software framework

Numerical results

Domain-Partitioning ModelProblem

Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions

Cooperation and Competition

MULTIPLE GRADIENT DESCENTALGORITHM (MGDA)

FOR MULTIOBJECTIVE OPTIMIZATION

Jean-Antoine DésidériINRIA Project-Team OPALE

Sophia Antipolis Méditerranée Centrehttp://www-sop.inria.fr/opale

ISMP 201221st International Synposium on Mathematical Programming,

Berlin, August 19-24, 2012

PDE-Constrained Optimization and Multi-Level/Multi-Grid Methods

ECCOMAS 2012European Congress on Computational Methods in Applied Sciences and

Engineering, Vienna (Austria), September 10-14, 2012

TO16: Computational inverse problems and optimization

1 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Outline1 Foreword2 MGDA

Pareto-stationarityDescent directionMain resultsPracticalitiesMathematical test-cases

3 Aircraft-Wing Aerodynamics4 Automobile Cooling System Design

Problem description and numerical toolsSoftware frameworkNumerical results

5 Domain-Partitioning Model ProblemModel problemQuasi-Newton MethodBasic MGDAMGDA-II

6 MGDA-IIIBird of paradiseAlgorithmProperties

7 MGDA-IV8 Conclusions9 Cooperation and Competition

2 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Foreword: Pareto optimality• Design vector:

Y ∈Υ⊆ RN

• Objective functions to be minimized (reduced):

Ji (Y ) (i = 1, . . . ,n)

Dominance in efficiency(see e.g. K. Miettinen: Nonlinear Multiobjective Optimization, Kluwer Academic Publishers)

Design-point Y 1 dominates design-point Y 2 in efficiency, Y 1 Y 2,iff

Ji (Y 1)≤ Ji (Y 2) (∀i = 1, . . . ,n)

and at least one inequality is strict. (relationship of partial order)

The designer’s Holy Grail: the Pareto setset of "non-dominated design-points" or "Pareto-optimal solutions"

The Pareto frontimage of the Pareto set in the objective function space(bounds the domain of attainable performance)

3 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Pareto front identificationFavorable situation: continuous and convex Pareto frontGradient-based optimization can be used efficiently, if number n ofobjective functions is not too large

Non-convex or discontinuous Pareto fronts do existEvolutionary strategies or GA’s most commonly-used forrobustness: NSGA-II (Deb), PAES

From: A Fast and Elitist Multiobjective Ge-netic Algorithm: NSGA-II, K. Deb, A. Pratap,S. Agarwal, and T. Meyarivan, IEEE Trans-actions in Evolutionary Computation, Vol. 6,No. 2, April 2002.

Nondominated solutions with NSGA-II on KUR (testcase by F. Kursawe, 1990)

n = 3 f1(x) =n−1

∑i=1

−10exp

−√

x2i + x2

i+1

5

f2(x) =n

∑i=1

(|xi |

45 + 5sinx3

i

)4 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Our general objective

Construct a robust gradient-based method for Paretofront identification

Note : in shape optimization, N is often the dimension of a parameterization, meant to be large

as the dimension of a discretization usually is; thus, often, N n. However, the following

theoretical results also apply to cases where n > N which can be the case of other situations in

engineering optimization.

5 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions









6 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Constructing a gradient-basedcooperative algorithm

The basic question :Knowing the gradients, ∇Ji (Y 0), of n criteria, assumed to besmooth functions of the design vector Y ∈ RN , at a given startingdesign-point Y 0, can we define a nonzero vector ω ∈ RN , in thedirection of which the Fréchet derivatives of all criteria have thesame sign, (

∇Ji (Y 0),ω)≥ 0 (∀i = 1,2, ...,n)?

Answer:Yes, if Y 0 is not Pareto-optimal!

Then, −ω is locally a descent direction common to allcriteria.

7 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Notion of Pareto-stationarityClassically, for smooth real-valued functions of Nvariables

Optimality (+ regularity) =⇒ Stationarity

Now, for smooth Rn-valued functions of N variables,which stationarity requirement is to be enforced forPareto-optimality?

Proposition 1: Pareto Stationarity at a design point Y 0

in an open domain B in which the objective-functions are smoothand convex, if there exists a convex combination of the gradientsequal to 0:

∃α = αi(i=1,,n) s.t.n

∑i=1

αi∇Ji (Y 0) = 0 , αi ≥ 0 (∀i) ,n

∑i=1

αi = 1

Then (to be established subsequently):

Pareto-optimality (+ regularity) =⇒ Pareto-stationarity

8 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


General principle

Proposition 2: Minimum-norm element in the convex hullLet ui(i=1,...,n) be a family of n vectors in RN , and U the so-called convexhull of this family, i.e. the following set of vectors :

U =

u ∈ RN / u =

n

∑i=1

αi ui ; αi ≥ 0 (∀i) ;n

∑i=1

αi = 1

Then, U admits a unique element of minimal norm, ω, and the followingholds :

∀u ∈ U : (u,ω)≥ ‖ω‖2

in which (u,v) denotes the usual scalar product of the vectors u and v .

9 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Existence and uniqueness of ω

Existence⇐= U closed. Uniqueness⇐= U convex.To establish uniqueness, let ω1 and ω2 be two realisations of the minimum:

‖ω1‖= ‖ω2‖= Argminu∈U ‖u‖

Then, since U is convex, ∀ε ∈ [0,1], ω1 + εr12 ∈ U (r12 = ω2−ω1) and:

‖ω1 + εr12‖ ≥ ‖ω1‖

Square both sides, and remplace by scalar products:

∀ε ∈ [0,1] : (ω1 + εr12,ω1 + εr12)− (ω1,ω1)≥ 0

Then∀ε ∈ [0,1] : 2ε(r12,ω1) + ε

2(r12, r12)≥ 0

As ε→ 0+, this condition requires that (r12,ω1)≥ 0; but then, for ε = 1:

‖ω2‖2−‖ω1‖2 = 2(r12,ω1) + (r12, r12) > 0

unless r12 = 0, i.e. ω2 = ω1.

Remark: The identification of the element ω is equivalent to the constrained minimization of aquadratic form in Rn , and not RN .

10 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


First consequence

∀u ∈ U : (u,ω)≥ ‖ω‖2

Proof:Let u ∈ U, and r = u−ω.∀ε ∈ [0,1], ω + εr ∈ U (convexity); hence:

‖ω + εr‖2−‖ω‖2 = 2ε(r ,ω) + ε2(r , r)≥ 0

and this requires that:

(r ,ω) = (u−ω,ω)≥ 0

11 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Proof of Pareto-stationarityat Pareto-optimal design-point Y 0 ∈ B (open ball) ⊆ RN

Hyp : Ji (Y ) (∀i ≤ n) smooth and convex in B ; ui = ∇Ji (Y 0) (∀i ≤ n)

Without loss of generality, assume Ji (Y 0) = 0 (∀i). Then:

Y 0 = ArgminY

Jn(Y ) subject to: Ji (Y )≤ 0 (∀i ≤ n−1) (1)

Let Un−1 be the convex hull of the gradients u1,u2, . . . ,un−1 and ωn−1 = Argminu∈Un−1‖u‖.

The vector ωn−1 exists, is unique, and such that:(ui ,ωn−1

)≥ ‖ωn−1‖2 (∀i ≤ n−1).

Two possible situations:1. ωn−1 = 0, and the objective-functions J1,J2, . . . ,Jn−1 satisfy the Pareto stationarity

condition at Y = Y 0. A fortiori, the condition is also satisfied by the whole set ofobjective-functions.

2. Otherwise ωn−1 6= 0. Let ji (ε) = Ji (Y 0− εωn−1) (i = 1, . . . ,n−1) so that ji (0) = 0 andj ′i (0) =−

(ui ,ωn−1

)≤−‖ωn−1‖2 < 0, and for sufficiently small strictly-positive ε:

ji (ε) = Ji (Y 0− εωn−1) < 0 (∀i ≤ n−1)

and Slater’s qualification condition is satisfied for (1). Thus, the LagrangianL = Jn(Y ) + ∑

n−1i=1 λi Ji (Y ) is stationary w.r.t. Y at Y = Y 0:

un +n−1

∑i=1

λi ui = 0

in which λi > 0 (∀i ≤ n−1) since equality constraints hold Ji (Y 0) = 0 (KKT condition).Normalizing this equation results in the Pareto-stationarity condition.

12 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


RN Affine and Vector StructuresNotation

Usually indistinct

When distinction necessary, a dotted symbol is used foran affine subset/subspace

In particular for the convex hull Uthe set of points vectors of a given origin O and equipollent to u ∈ Upoint to.

Rigorously speaking, the term "convex hull" applies to U.

13 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


RN Affine and Vector StructuresConvex hulls

A case where O⊥ /∈ U

U (affine)

U(vector)

A2

O

O⊥u1

u2

u3

u ∈ U

14 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Second consequenceNote that ∀u ∈ U:

u−un =n

∑i=1

αiui −

(n

∑i=1

αi

)un =

n−1

∑i=1

αiun,i (un,i = ui −un)

Thus U⊆ An−1 (or in affine space, U⊆ An−1),where An−1 is a set of vectors pointing to an affine sub-space An−1

of dimension at most n−1.

Define the orthogonal projection O⊥ of O onto An−1

Since−−→OO⊥ ⊥ An−1 ⊇ U:

O⊥ ∈ U⇐⇒ ω =−−→OO⊥

In this case:∀u ∈ U : (u,ω) = const. = ‖ω‖2

15 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Second consequence

ProofSince O⊥ is the orthogonal projection of O onto An−1,

−−→OO⊥ ⊥ An−1, and in particular

−−→OO⊥ ⊥ U.

If additionally O⊥ ∈ U, vector−−→OO⊥ is the minimum-norm element of U: ω =

−−→OO⊥.

In this case, ω can be calculated by ignoring the inequality constraints, ∀i, αi ≥ 0, that areautomatically satisfied by the solution.

Hence, vector ω realizes the minimum of the quadratic form ‖ω‖2 = ‖∑ni=1 αi ui‖2 uniquely

subject to the equality constraint: ∑ni=1 αi = 1.

The Lagrangian is formed with a unique multiplier λ:

L(α,λ) = 12

(n

∑i=1

αi ui ,n

∑i=1

αi ui

)+ λ

(n

∑i=1

αi −1

)

The stationarity condition requires that:

∀i, ∂L∂αi

= (ui ,ω) + λ = 0 =⇒ (ui ,ω) =−λ = const. = ‖ω‖2

By convex combination, the result extends straightforwardly to the whole U.

16 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Application to the case ofgradients

Suppose ui = ∇Ji(Y 0) (∀i); then:

• either ω 6= 0, and all criteria admit positive Fréchet derivativesin the direction of ω (all equal if ω belongs to the interior U)

• or ω = 0, and the current design point Y 0 is Pareto stationary:

∃αii=1,2,...,n (αi ≥ 0,∀i;n

∑i=1

αi = 1) so thatn

∑i=1

αiui = 0

17 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Proposition 3: Common Descent DirectionBy virtue of Propositions 1 and 2 in the particular case where

ui = J ′i =∇Ji(Y 0)

Si

(Si : user-supplied scaling constant; Si > 0), two situations are possible atY = Y 0 :

• either ω = 0, and the design point Y 0 is Pareto-stationary (orPareto-optimal);

• or ω 6= 0, and −ω is a descent direction common to all criteriaJi(x)(i=1,...,n); additionally, if ω⊥ U, the scalar product (u,ω)

(u ∈ U), and the Frechet derivatives (ui ,ω) are all equal to ‖ω‖2.

MGDA : substitute vector ω to the single-criterion gradientin the steepest-descent method.

Proposition 4: ConvergenceCertain normalization provisions being made, in RN , the MultipleGradient Descent Algorithm (MGDA) converges to aPareto-stationary point.

18 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Convergence proof

Define the criteria to be positive (possibly apply an exp-transform), and ∞ at ∞ (possibly add anexploding term outside of the working ball):

∀i, Ji (Y )→ ∞ as ‖Y‖→ ∞.

Assume these criteria to be continuous.

If the iteration stops in a finite number of steps, a Pareto-stationary point has been reached.

Otherwise, the iteration continues indefinitely, generating an infinite sequence of design points,Y k. The corresponding sequences of criteria are infinite, positive and monotone decreasing.They are bounded. Hence, the sequence of iterates, Y k, is itself bounded and it admits asubsequence converging to say Y ?.Necessarily, Y ? is a Pareto-stationary point. To establish this, assume instead that ω?, whichcorresponds to Y ∗, is nonzero. Then for each criterion, there exists a stepsize ρ for which, thevariation is finite. These criteria are in finite number. Hence, the smallest ρ wlll cause a finitevariation to all criteria, and this is in contradiction with the fact that only infinitely-small variationsof the criteria are realized from Y ?.

19 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Remark 1 : practicaldetermination of vector ω

in the convex hull

Problem to be solved in U⊂ RN

ω = Argminu∈U ‖u‖

U =

u ∈ RN / u =

n

∑i=1

αi ui ; αi ≥ 0 (∀i) ;n

∑i=1

αi = 1

Usually, but not necessarily : N ≥ n.

ParameterizationLet

αi = σ2i (i = 1, ...,n)

to satisfy trivially the inequality constraints (αi ≥ 0 ,∀i), and transform the equality constraint,∑

ni=1 αi = 1 into

n

∑i=1

σ2i = 1⇐⇒ σ = (σ1,σ2, ...,σn) ∈ Sn

where Sn is the unit sphere of Rn and precisely not RN .

20 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Determining vector ω (cont’d)The sphere is easily parameterizedusing trigonometric functions of n−1 independent arcs φ1,φ2, ...,φn−1:

σ1 = cosφ1 . cosφ2 . cosφ3 . ... . cosφn−1σ2 = sinφ1 . cosφ2 . cosφ3 . ... . cosφn−1σ3 = 1 . sinφ2 . cosφ3 . ... . cosφn−1

.

.

....

σn−1 = 1 . 1 . ... . sinφn−2 . cosφn−1σn = 1 . 1 . ... . 1 . sinφn−1

(Consider only : φi ∈ [0,π/2] ,∀i and set : φ0 = π

2 .)

Let ci = cos2 φi (i = 1, ...,n)and get:

α1 = c1 . c2 . c3 . ... . cn−1α2 = (1− c1) . c2 . c3 . ... . cn−1α3 = 1 . (1− c2) . c3 . ... . cn−1

.

.

....

αn−1 = 1 . 1 . ... . (1− cn−2) . cn−1αn = 1 . 1 . ... . 1 . (1− cn−1)

(c0 = 0, and ci ∈ [0,1] for all i ≥ 1).

21 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Determining vector ω (end)

The convex hull is thus parameterized in

[0,1]n−1 (n : number of criteria)

independently of N (dimension of design space).

For example, with n = 4 criteria :α1 = c1c2c3

α2 = (1− c1)c2c3

α3 = (1− c2)c3

α4 = (1− c3)

(c1,c2,c3) ∈ [0,1]3

22 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Remark 2 : appropriatenormalization of the gradients

may reveal to be essential : ui = ∇Ji (Y 0)/SiCase n = 2

• Without gradient normalizationThen, ω =

−−→OO⊥ unless the angle < u1,u2 > is acute, and the norms ||u1||, ||u2|| are very

different; in that case, ω is equal the one of smaller norm.

u1u2 ω =−−→OO⊥

u1

u2

ω =−−→OO⊥

u1u2 = ω−−→OO⊥

• With normalization: The equality ω =−−→OO⊥ holds automatically =⇒

EQUAL FRÉCHET DERIVATIVES

General CaseIF the vectors ui are NOT NORMALIZED, those of smallernorms are more influential to determine the direction of vector ω

23 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Normalizations being examinedRecall

ui = J ′i = ∇Ji (Y 0)/Si (Si > 0) =⇒ ω

δY =−ρω =⇒ δJi = (∇Ji ,δY ) =−ρSi (ui ,ω)

(ui ,ω) = const. if ω does not lie on the “edge” of convex hull

• Standard :

ui =∇Ji (Y 0)

‖∇Ji (Y 0)‖• Equal logorithmic variations (whenever ω is not on edge) :

ui =∇Ji (Y 0)

Ji (Y 0)

• Newton-inspired when limJi = 0 :

ui =Ji

‖∇Ji (Y 0)‖2 ∇Ji (Y 0)

• Newton-inspired when limJi 6= 0 :

ui =max

(J(k−1)

i − J(k)i ,δ

)‖∇Ji (Y 0)‖2 ∇Ji (Y 0) , k : iteration no. , δ > 0 (small)

24 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Is standard normalizationsufficient to guarantee that

ω =−−→OO⊥?

(yielding equal Fréchet derivatives)

No, if n > 2 :

A2

u1

u2 u3O⊥

−−→OO⊥ /∈ U

0

Unit Sphere

25 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Experimenting MGDAFrom Adrien ZERBINATI, on-going doctoral thesis

Basic testcase : minimize the functions

f (x ,y) = 4x2 + y2 + xy g(x ,y) = (x−1)2 + 3(y−1)2

26 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Basic testcaseConvergence from different initial design points

DESIGN SPACE FUNCTION SPACE

(0.5,2)

(1.5,2.5)

(-1.5,-2.5)

27 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Fonseca testcase

Minimize 2 fonctions of 3 variables :

f1,2(x1,x2,x3) = 1−exp

(−

3

∑i=1

(xi ±

1√3

)2)

28 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Fonseca testcase (cont’d)Convergence from initial design points over a sphere


Continuous but nonconvex front

29 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Fonseca testcase (end)Compare MGDA with Pareto Archived Evolution Strategy

(PAES)


−0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 −1

0

1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

x2

x1

PAES 2x50 & MGDA 6x12

x3

MGDA iterate

PAES generate

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1PAES 2x50 & MGDA 6x12

MGDA iterate

PAES generate

30 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions









31 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Wing-shape optimizationWave-drag minimization in conjunction with

lift maximization

Metamodel-assisted MGDA (Adrien Zerbinati)

• Wing-shape geometryextruded from airfoil geometry (initial airfoil : NACA0012), 10 parameters

• Two-point/two-objective optimization

1 Subsonic Eulerian flow : M∞ = 0.3, α = 8o , for CL maximization2 Transonic Eulerian flow : M∞ = 0.83, α = 2o , for CD

minimization

• Algorithm - Initial database of 40 design points evolves as follows :

1 Evaluate each new point by two 3D Eulerian flow computations2 Construct surrogate models for CL and CD (using the entire

dataset), and perform MGDA to convergence3 Enrich the database with new points, if appropriate

After 6 loops, the database is made of 227 design points(554 flow computations)

32 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Wing-shape optimization2

Convergence of dataset

6 loops

0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 0.045−0.85

−0.8

−0.75

−0.7

−0.65

−0.6

−0.55

−0.5

−0.45

drag coefficient

−lif

t coeffic

ient

initial database

1st step

2nd step

3rd step

4th step

5th step

6th step

DRAG (transonic)

-LIFT(subsonic)

33 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions



Low-drag quasi-Pareto-optimal shape

SUBSONIC TRANSONIC

M∞ = 0.3, α = 8o M∞ = 0.83, α = 2o

34 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions



High-lift quasi-Pareto-optimal shape

SUBSONIC TRANSONIC

M∞ = 0.3, α = 8o M∞ = 0.83, α = 2o

35 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions



Intermediate quasi-Pareto-optimal shape

SUBSONIC TRANSONIC

M∞ = 0.3, α = 8o M∞ = 0.83, α = 2o

36 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions









37 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Description of the test case :

In-house CFD Solver(Num3sis) :

• Compressible Navier-Stokes(FV/FE)

• Parallel computing (MPI)

• CPU cost : 4h on 32 cores

• CAD and mesh by GMSH

Test case :• Mach number : 0.1

• Reynolds number : Re ≈ 2800

• Two-criterion shapeoptimization

• Fine mesh (some 700,000points)

38 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


First objective function

Velocity Variance

• Umean =1

Vtot∑

d(cel,Po )≤ε

u(cel)∗Vol(cel)

• σ2vel,x =

1Vtot

∑d(cel,Po )≤ε

(Umean,x −ux (cel))2 ∗Vol(cel)

• σ2 = σ2vel,x + σ2

vel,y + σ2vel,z

39 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Second objective function

Pressure loss• pk = pmean for inlet (k = i), or outlet (k = o)

• uk = ‖u‖mean for in- or out-let

• ∆p = pi −po +ρi u2

i

2− ρou2

o

2

40 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Design parameters

41 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


MGDA on surrogate model

42 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Software framework

43 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Identifying the Pareto set

3

21

Nominal PointInitial database3rd step6th step9th stepInitial Pareto frontFinal Pareto front

Pres

sure

loss

0,5

1

1,5

2

2,5

Velocity variance0,6 0,8 1 1,2 1,4

Figure: MGDA metamodel assisted step by step evolution. Initial and finalnon-dominated sets. 223 solver calls.

44 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Velocity Magnitude Streamlines

1: BEST VELOCITY VARIANCE 2 : NOMINAL SHAPE

SHAPE COMPARISON 3 : BEST PRESSURE LOSS

45 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Velocity magnitude in a sectionjust after the bend :

1: BEST VELOCITY VARIANCE 2 : NOMINAL SHAPE

SECTION POSITION 3 : BEST PRESSURE LOSS

46 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Velocity Magnitude in outputsection :

1 : BEST VELOCITY VARIANCE 2 : NOMINAL SHAPE

SECTION POSITION 3 : BEST PRESSURE LOSS

47 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


NOMINAL versus LOWESTVELOCITY VARIANCE

48 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


NOMINAL versus LOWESTPRESSURE LOSS

49 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


LOWEST VELOCITYVARIANCE versus LOWEST

PRESSURE LOSS

50 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions









51 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Dirichlet problem

Discrete solution by standard 2nd-order finite-differencemethod over 40×40 qradrangular mesh

−∆u = f over Ω = [−1,1]× [−1,1]

u = 0 (Γ = ∂Ω)

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 -1-0.8

-0.6-0.4

-0.2 0

0.2 0.4

0.6 0.8

1

-0.5

0

0.5

1

1.5

u_h

u_h

’fort.20’ 1.3 1.1 0.9 0.7 0.5 0.3 0.1 0

-0.1 -0.2 -0.3 -0.4 -0.5

x

y

u_h

u_h contour lines 1.3 1.1 0.9 0.7 0.5 0.3 0.1 0

-0.1 -0.2 -0.3 -0.4 -0.5

-1 -0.5 0 0.5 1

x

-1

-0.5

0

0.5

1

y

52 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Domain partitioningFunction value controls at 4 interfaces

yielding 4 sub-domain Dirichlet problems with 2controlled interfaces each

53 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Interface jumps

Formal expression

• over γ1 (0≤ x ≤ 1 ; y = 0):

s1(x) =∂u∂y

(x ,0+)− ∂u∂y

(x ,0−) =

[∂u1

∂y− ∂u4

∂y

](x ,0);

• over γ2 (x = 0 ; 0≤ y ≤ 1):

s2(y) =∂u∂x

(0+,y)− ∂u∂x

(0−,y) =

[∂u1

∂x− ∂u2

∂x

](0,y);

• over γ3 (−1≤ x ≤ 0 ; y = 0):

s3(x) =∂u∂y

(x ,0+)− ∂u∂y

(x ,0−) =

[∂u2

∂y− ∂u3

∂y

](x ,0);

• over γ4 (x = 0 ; −1≤ y ≤ 0):

s4(y) =∂u∂x

(0+,y)− ∂u∂x

(0−,y) =

[∂u4

∂x− ∂u3

∂x

](0,y).

Approximationby 2nd-order one-sided finite-differences

54 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Functionals and matchingcondition

Interface functionals

Ji =∫

γi

12 s2

i w dγi := Ji (v)

that is, explicitly:

J1 =∫ 1

0

12 s1(x)2 w(x)dx J2 =

∫ 1

0

12 s2(y)2 w(y)dy

J3 =∫ 0

−1

12 s3(x)2 w(x)dx J4 =

∫ 0

−1

12 s4(y)2 w(y)dy

(w(t) (t ∈ [0,1]) is an optional weighting function, andw(−t) = w(t).)

Matching condition

J1 = J2 = J3 = J4 = 0

55 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Adjoint problems

Eight Dirichlet sub-problems

∆pi = 0

pi = 0

pi = siw

(Ωi )

(∂Ωi\γi )

(γi )

∆qi = 0

qi = 0

qi = si+1w

(Ωi )

(∂Ωi\γi+1)

(γi+1)

Green’s formula∫γi

si u′i nw =∫

γi

pi n v ′i +∫

γi+1

pi n v ′i+1

and ∫γi+1

si+1 u′i nw =∫

γi

qi n v ′i +∫

γi+1

qi n v ′i+1

56 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Functional gradients

J ′1 =∫ 1

0s1(x)s′1(x)w(x)dx =

∫ 1

0s1(x)

[∂u′1∂y− ∂u′4

∂y

](x ,0)w(x)dx

=∫ 1

0

∂(p1−q4)

∂y(x ,0)v ′1(x)dx +

∫ 1

0

∂p1

∂x(0,y)v ′2(y)dy +

∫ 0

−1

∂q4

∂x(0,y)v ′4(y)dy

J ′2 =∫ 1

0s2(y)s′2(y)w(y)dy =

∫ 1

0s2(y)

[∂u′1∂x− ∂u′2

∂x

](0,y)w(y)dy

=∫ 1

0

∂q1

∂y(x ,0)v ′1(x)dx +

∫ 1

0

∂(q1−p2)

∂x(0,y)v ′2(y)dy +

∫ 0

−1

∂p2

∂y(x ,0)v ′3(x)dx

J ′3 =∫ 0

−1s3(x)s′3(x)w(x)dx =

∫ 0

−1s3(x)

[∂u′2∂y− ∂u′3

∂y

](x ,0)w(x)dx

=−∫ 1

0

∂q2

∂x(0,y)v ′2(y)dy +

∫ 0

−1

∂(q2−p3)

∂y(x ,0)v ′3(x)dx−

∫ 0

−1

∂p3

∂x(0,y)v ′4(y)dy

J ′4 =∫ 0

−1s4(y)s′4(y)w(y)dy =

∫ 0

−1s4(y)

[∂u′4∂x− ∂u′3

∂x

](0,y)w(y)dy

=−∫ 1

0

∂p4

∂y(x ,0)v ′1(x)dx−

∫ 0

−1

∂q3

∂y(x ,0)v ′3(x)dx +

∫ 0

−1

∂(p4−q3)

∂x(0,y)v ′4(y)dy

Conclusion:

J ′i =4

∑j=1

∫γj

Gi,j v ′j dγj (i = 1, ...,4) ; Gi,j : partial gradients.

57 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Other technical details

Dirichlet sub-problemsAll 12 (4 direct, 4×2 adjoint) sub-problems are solved by directinverse (discrete sine-transform)

uh = (ΩX ⊗ΩY ) (ΛX ⊕ΛY )−1 (ΩX ⊗ΩY ) fh

Integralsare approximated by the trapezoidal rule

58 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Reference methodQuasi-Newton Method applied to agglomerated criterion

J =4

∑i=1

Ji

Gradient

∇J =4

∑i=1

∇Ji

Iterationv (`+1) = v (`)−ρ`∇J(`)

Stepsize

δJ = ∇J(`).δv (`) =−ρ`

∥∥∥∇J(`)∥∥∥2

is set to −εJ(`) by fixing

ρ` =εJ(`)∥∥∇J(`)

∥∥2

59 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Quasi-Newton steepest-descentε = 1

Convergence history

Asymptotic Global

1e-06

1e-05

0.0001

0.001

0.01

0.1

0 2 4 6 8 10 12 14 16 18 20

J_1J_2J_3J_4

J = SUM TOTAL

1e-09

1e-08

1e-07

1e-06

1e-05

0.0001

0.001

0.01

0.1

1

10

100

0 20 40 60 80 100 120 140 160 180 200

J_1J_2J_3J_4

J = SUM TOTAL

60 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Quasi-Newton steepest descentε = 1

History of gradients over 200 iterations

∂J/∂v1 ∂J/∂v2

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

0 2 4 6 8 10 12 14 16 18 20

DISCRETIZED FUNCTIONAL GRADIENT OF J = SUM_i J_i W.R.T. V1

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

0 2 4 6 8 10 12 14 16 18 20


61 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions



History of gradients over 200 iterations

∂J/∂v3 ∂J/∂v4

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0 2 4 6 8 10 12 14 16 18 20


-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

0 2 4 6 8 10 12 14 16 18 20


62 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions



Discrete solution

-1-0.5

0 0.5

1 -1

-0.5

0

0.5

1

-0.5

0

0.5

1

1.5

u_h

u_h

u_h(x,y) 1.3 1.1 0.9 0.7 0.5 0.3 0.1 0

-0.1 -0.2 -0.3 -0.4 -0.5

x

y

u_h

u_h contour lines 1.3 1.1 0.9 0.7 0.5 0.3 0.1 0

-0.1 -0.2 -0.3 -0.4 -0.5

-1 -0.5 0 0.5 1

x

-1

-0.5

0

0.5

1

y

63 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Basic MGDA

Practical determination of minimum-norm element ω

ω =4

∑i=1

αiui

ui = ∇Ji (Y 0)

using the following parameterization of the convex hull:α1 = c1c2c3

α2 = (1− c1)c2c3

α3 = (1− c2)c3

α4 = (1− c3)

and(c1,c2,c3) ∈ [0,1]3

are discretized by step of 0.01. Best set of coefficients retained.

64 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Basic MGDAStepsize

Iteration

v (`+1) = v (`)−ρ`ω(`)

Stepsize

δJ = ∇J(`).δv (`) =−ρ` ∇J(`).ω

is set to −εJ(`) by fixing

ρ` =εJ(`)

∇J(`).ω

In practice: ε = 1.

65 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Basic MGDAε = 1

Asymptotic Convergence

Unscaled Scaledui = ∇Ji ui = ∇Ji

Ji

66 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Basic MGDAε = 1

Global Convergence


Ji

67 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


First conclusions

Basic MGDA• Scaling essential

• Somewhat deceiving convergence

Who is to blame?• the insufficiently accurate determination of ω;

• the non-optimality of the scaling of gradients;

• the non-optimality of the step-size, the parameter ε beingmaintained equal to 1 throughout;

• the large dimension of the design space, here 76 (4 interfacesassociated with 19 d.o.f.’s).

68 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


MGDA-II : a direct algorithmValid when gradients are linearly independent

User-supplied scaling factors Si(i=1,,n), and scaledgradients

J ′i =∇Ji (Y 0)

Si

(Si > 0; e.g. Si = Ji for logarithmic gradients).

Perform Gram-Schmidt with special normalization

• Set u1 = J ′1

• For i = 2, . . . ,n, set: ui =J ′i −∑k<i ci,k uk

Aiwhere:

∀k < i : ci,k =

(J ′i ,uk

)(uk ,uk

) , and:

Ai =

1−∑k<i

ci,k if nonzero

εi otherwise (εi arbitrary, but small)69 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Consequences

Element ω always in the interior of convex hull

ω =n

∑i=1

αiui αi =1

‖ui‖2∑

nj=1

1

‖uj‖2

=1

1 + ∑j 6=i‖ui‖2

‖uj‖2

< 1

This implies equal projections of ω onto uk

∀k :(uk ,ω

)= αk ‖uk‖2 = ‖ω‖2

and finally: (J ′i ,ω

)= ‖ω‖2 (∀i)

(with a possibly-modified scale S′i = (1 + εi )Si ), that is:the same positive constant.

EQUAL FRÉCHET DERIVATIVESFORMED WITH SCALED GRADIENTS

70 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Geometrical demonstration

u1

u2O

u3

O⊥

u ∈ U

Given J ′i = ∇Ji (Y 0)/Si , perform G.-S. process

to compute ui = [J ′i −∑k<i ci,k uk ]/Ai ; then:

O⊥ ∈ U =⇒ ω =−−→OO⊥ =⇒(

u1,ω)

=(u2,ω

)=(u3,ω

)= ‖ω‖2

J ′i = Aiui + ∑k<i ci,k uk =⇒(J ′i ,ω

)=(Ai + ∑

k<ici,k)

︸︷︷︸= 1 by

normalization

thru Ai

‖ω‖2 = ‖ω‖2 = const.

71 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


MGDA-II b: automatic rescaleDo not let constant Ai become negative; instead define

Ai = 1−∑k<i

ci,k

only if this number is strictly-positive. Otherwise, use modified scale:

S′i =

(∑k<i

ci,k

)Si

(“automatic rescale”), so that:

c′i,k =

(∑k<i

ci,k

)−1

ci,k , ∑k<i

c′i,k = 1 ,

and set Ai = εi , for some small εi .

Same formal conclusion: the Fréchet derivatives are equal; but the

value is much larger, and (at least) one criterion has been rescaled

72 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Some open questions

• ωII = ω?

• Yes, if n = 2 and angle obtuse

• No, in general.

• Is the following implication

limωII = 0 =⇒ limω = 0

true? (assuming the limit is the result of the MGDA iteration)

In which order should we perform the Gram-Schmidtprocess?

• Etc

73 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


MGDA-IIε = 1

Global convergence


Ji

74 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


MGDA-II bε = 1

Global convergence


Ji

automatic rescale on automatic rescale on

75 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


MGDA-II : Recap

Convergence over 500 iterations


Ji

automatic rescale off automatic rescale on

76 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions









77 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


MGDA-IIIBird of paradise (Strelitzia reginae)

78 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions



79 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions



u1

u2

ω

80 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions



ω

81 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


MGDA-III - definition1

Start from scaled gradients, J ′i , and duplicates, gi,J ′i =

∇Ji (Y 0)

Si(Si : user-supplied scale)

and proceed in 3 steps: A,B and C.

A: Initialization

• Set

u1 := g1 = J ′k / k = Argmaxi minj

(J ′j ,J

′i

)(J ′i ,J

′i

)(justified afterwards)

• Set n×n lower-triangular array c = ci,j (i ≥ j) to 0.

• Set, conservatively, I := n(expected number of computed orthogonal basis vectors).

• Assign some appropriate value to a cut-off constant a :0≤ a < 1.

(Note: main diagonal of array c is to contain cumulative row-sums.)82 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions



B: Main G.-S. loop; for i = 2,3, . . . , (at most) n, do:

1. Calculate the i−1st column of coefficients:

cj,i−1 =

(gj ,ui−1

)(ui−1,ui−1

) (∀j = i, . . . ,n)

and update the cumulative row-sums:

cj,j := cj,j + cj,j−1 = ∑k<i

cj,k (∀j = i, . . . ,n)

2. Test:• If the following condition is satisfied

cj,j > a (∀j = i, . . . ,n)

set I := i−1, and interrupt the Gram-Schmidt process(go to 3.).

• Otherwise, compute next orthogonal vector ui as follows:

83 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions



- Identify index ` = Argminj cj,j / i ≤ j ≤ n(Note that c`,` ≤ a < 1.)

- Permute information associated with i and ` :g-vectors gi g`, rows i and ` of array c and cumulative row-sums ci,i c`,`.

- Set Ai = 1− ci,i ≥ 1−a > 0 (ci,i = former-c`,` ≤ a), and calculate

ui =gi −∑k<i ci,k uk

Ai

(in which gi = former-g`, ci,k = former-c`,k ).

- If ui 6= 0, return to 1. with incremented i; otherwise:

gi = ∑k<i

ci,k uk = ∑k<i

c′i,k gk

where the c′i,k are calculated by backward substitution.Then, if c′i,k ≤ 0 (∀k < i):

Pareto-stationarity detected: STOP MGDA iteration;otherwise (ambiguous exceptional case):

STOP G.-S. process; compute original ω and go to C.

84 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions



3. Calculate ω as the minimum-norm element in the convex hullof u1,u2, . . . ,uI:

ω =I

∑i=1

αiui 6= 0

αi =1

‖ui‖2∑

Ij=1

1

‖uj‖2

=1

1 + ∑j 6=i‖ui‖2

‖uj‖2

(Note that here all computed ui 6= 0, and 0 < αi < 1.)

C: If ‖ω‖< TOL, STOP MGDA iteration; otherwise,perform descent step and return to B.

85 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


MGDA-III - consequencesIf I = n :MGDA-III = MGDA-II b + automatic ordering in Gram-Schmidtprocess making ’rescale’ unnecessary since, by construction

∀i : Ai ≥ 1−a > 0

Otherwise, I < n (incomplete Gram-Schmidt process):

• First I Fréchet derivatives:(gi ,ω

)=(ui ,ω

)= ‖ω‖2 > 0 (∀i = 1, . . . , I)

• Subsequent ones (i > I):

ω =I

∑j=1

αjuj gi =I

∑k=1

ci,k uk + vi vi ⊥ u1,u2, . . . ,uI

⇓(gi ,ω

)=

I

∑k=1

ci,k(uk ,ω

)=

I

∑k=1

ci,k ‖ω‖2 = ci,i ‖ω‖2 > a‖ω‖2 > 086 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Choice of g1Justification

At initialization, we have set

u1 := g1 = J ′k / k = Argmaxi minj

(J ′j ,J

′i

)(J ′i ,J

′i

)This was equivalent to maximizing c2,1 = c2,2, that is,maximizing the least cumulative row-sum, at firstestimation.

(Recall that the favorable situation is when all cumulative row-sums are positive, or > a.)

87 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Expected benefits

Automatic ordering and rescale

When gradients exhibit a general trend, ω is found infewer steps, and has a larger norm

Larger ‖ω‖ implies larger directional derivatives, andgreater efficiency of descent step(

gi ,ω

‖ω‖)

= ‖ω‖ or a‖ω‖

88 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions









89 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Adressing the question ofscaling

When Hessians are known• For objective function Ji (Y ) alone, Newton’s iteration writes

Y 1 = Y 0−pi Hipi = ∇Ji (Y 0)

• Split pi into orthogonal components

pi = qi + ri

where: qi =

(pi ,∇Ji (Y 0)

)‖∇Ji (Y 0)‖2 ∇Ji (Y 0) and ri ⊥ ∇Ji (Y 0)

• Define scaled gradientJ ′i = qi

• Apply MGDA-III

Equivalently, set scaling factor as follows: Si =‖∇Ji (Y 0)‖2(pi ,∇Ji (Y 0)

)90 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


VariantMGDA-IV b

Use BFGS iterative estimates in place of exact Hessians

k : MGDA iteration index

(∀i = 1, . . . ,n) H(0)i = Id

H(k+1)i = H(k)

i −1

s(k)T H(k)i s(k)

H(k)i s(k)s(k)T

H(k)T

i +1

z(k)T

i s(k)z(k)

i z(k)T

i

in which:

s(k) = Y (k+1)−Y (k)

z(k)i = ∇Ji (Y (k+1))−∇Ji (Y (k))

91 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Recommended Step-Size

Ji (Y 0−ρω) = Ji (Y 0)−ρ(∇Ji (Y 0),ω

)+ 1

2 ρ2(Hiω,ω

)+ . . . .

|δJi | := Ji (Y 0)− Ji (Y 0−ρω) := Aiρ− 12 Biρ

2 ,

where: Ai = Siai , Bi = Sibi , and:

ai =(J ′i ,ω

)bi =

(Hiω,ω

)/Si .

If the scales Si are truly relevant for the objective-functions

ρ? = Argmax

ρmin

iδi .

where:

δi =|δJi |Si

= aiρ− 12 biρ

2 .

92 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Recommended Step-Size2

I = n

ρ? = ρ

?I :=

‖ω‖2

bI,

where bI = maxi≤I bi is assumed to be positive.

I < n

ρ?II :=

a‖ω‖2

bII,

where bII = maxi>I bi is assumed to be positive.Also

ρ× =2(1−a)‖ω‖2

bI−bII

at which point the two bounding parabolas intersect:

‖ω‖2ρ− 1

2 bIρ2 = a‖ω‖2

ρ− 12 bIIρ

2 .

93 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Recommended Step-Size3

a) I = n

||ω||2δ

ρρ?I

b) I < n, ρ× > max(ρ?I ,ρ

?II )

||ω||2

a||ω||2

δ

ρρ?I ρ?

II ρ×

c) I < n, ρ× ∈ (ρ?I ,ρ

?II )

||ω||2

a||ω||2

δ

ρρ?I ρ× ρ?

II

d) I < n, ρ× < min(ρ?I ,ρ

?II )

||ω||2

δ

ρρ?I

ρ×

ρ?II

94 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Recommended Step-Sizeend

If I = n, ρ? = ρ?I .

If I < n, ρ? is the element of the triplet ρ?I ,ρ

?II ,ρ×

which separates the other two.

95 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions









96 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


CONCLUSIONSTheory

Multiple-Gradient Descent Algorithm

• Notion of Pareto stationarity introduced to characterize"standard Pareto-optimal points"

• Basic MGDA : permits to identify a descent direction commonto arbitrary number of criteria, knowing the local gradients innumber at most equal to the number of design parameters

• Proof of convergence of MGDA to Pareto-stationary points

• Capability to identify the Pareto front demonstrated formathematical test-cases; non-convexity of Pareto front not aproblem

• MGDA-II : a direct procedure, faster and more accurate, whengradients are linearly independent; variant MGDA-II b (withautomatic rescale) recommended

• On-going research: scaling, preconditioning, step-sizeadjustment, what to do to extend MGDA-II to cases oflinearly-dependent gradients, . . .

97 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


CONCLUSIONSApplications

Implementation and first experiments

• Validation with analytical test cases and comparison with aPareto capturing method (PAES)

• Extension to Meta-Model-Assisted MGDA

Aircraft-Wing Aerodynamics and AutomobileCooling-System Design

• Successful design experiments with 3D compressible flow(Euler and Navier-Stokes)

• On-going development of meta-model-assisted MGDA andtesting of 3D external aerodynamics case to be presented atECCOMAS 2012, Vienna, Austria)

98 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


CONCLUSIONSApplications

Domain-partitioning model problemThe method works, but it is not as cost-efficient as the standardquasi-Newton method applied to the agglomerated criterion; but thetest-case was peculiar in several ways:

• Pareto front and set reduced to a singleton

• large dimension of the design space (4 criteria, 76 design variables)

• robust procedure to adjust the step-size needed

• determination of ω in the basic method should be made more accurately, perhaps usingiterative refinement

• scaling of gradients found important but never really analyzed completely (logarithmicgradients appropriate for vanishing criteria)

• general preconditioning not yet clear (on-going)

The MGDA-II direct variant found faster and more robust; withautomatic rescaling procedure on, MGDA-II b seems to exhibitquadratic convergence.

99 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions









100 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Flirting with the Pareto frontMGDA is combined with a locally-adapted Nash game

based on a territory splitting that preserves tosecond-order the Pareto-stationarity condition

Cooperation AND Competition

• COOPERATION (MGDA) :The design point is steered to the Pareto front.

• COMPETITION (Nash game) : at start (from the Pareto front) :α1∇J1 +α2∇J2 = 0; then let : JA = α1J1 +α2J2, et JB = J2, andadapt territory-splitting to best preserve JA; then :the trajectory remains tangent to the Pareto front.

101 / 102


J.-A. Désidéri

Foreword

MGDA

Pareto-stationarity

Descent direction

Main results

Practicalities





Software framework

Numerical results


Model problem

Quasi-Newton Method

Basic MGDA

MGDA-II

MGDA-III

Bird of paradise

Algorithm

Properties

MGDA-IV

Conclusions


Some references

Reports from INRIA Open Archive:

• M. G. D. A., INRIA Research Report No. 6953 (2009),Revised version, October 2012(Open Archive : http://hal.inria.fr/inria-00389811)

• Related INRIA Research Reports:Nos. 7667 (2011), 7922 (2012), 7968 (2012), 8068 (2012)

Other• C. R. Acad. Sci. Paris, Ser. I 350 (2012) 313-318

102 / 102

http://hal.inria.fr/inria-00389811

Documents

MULTIPLE GRADIENT DESCENT ALGORITHM (MGDA ...Engineering, Vienna (Austria), September 10-14, 2012 TO16: Computational inverse problems and optimization 1/102 MGDA applied to compressible