Zhen Lu CPACT University of Newcastle MDC Technology Reduced Hessian Sequential Quadratic Programming(SQP)

Zhen Lu

CPACT

University of Newcastle

MDC Technology

Centre for Process Analytics and Control Technology (CPACT)University of Newcastle, UK

Reduced Hessian Sequential Quadratic Programming(SQP)

Background

• BSc in Automatic Control, Tsinghua University, China

• MSc in Automatic Control, Tsinghua University, China

• The first year PhD student, CPACT, University of Newcastle, UK


Research area

• My research area : Process optimization;

On-line optimization;

Optimizing control

• Research Project - Optimization of Batch Reactor Operations (MDC)


Introduction

• Disadvantages of SQP• Advantages of reduced

Hessian SQP• Description of rSQP• Implementation of

rSQP

• Summarize• Numerical examples• Conclusion• Future work


Disadvantages of SQP methods

• In the SQP method, large and sparse QP sub-problems must be solved at each iteration. This can be computationally intensive to solve.

• Many chemical process optimization problems have a small number of degrees of freedom.

• A mixture of analytical second derivatives and many small, dense quasi-Newton updates are used to approximate the Hessian matrix of the Lagrangian function in the full space of the variables.


Advantages of reduced Hessian SQP

• The reduced Hessian SQP is designed for large Non-linear Programming(NLP) problems with few degrees of freedom.

• The approach only requires projected second derivative information and this can often be approximated efficiently with quasi-Newton update formulae.

• This feature makes rSQP especially attractive for process systems where second derivative information may be difficult or computationally intensive to obtain.


Advantages of reduced Hessian SQP

• Reduced Hessian SQP methods project the quadratic programming sub-problem into the reduced space of independent variables.

• Refinements of the reduced Hessian SQP approach guarantee a one-step super-linear convergence rate.


Description of rSQP

• Optimization problems of the form:

)(min xfnRx

0)( xc

RRf n : mn RRc :


Description of rSQP

• Quadratic sub-problem:


dxWddxg kTT

kRd n

)()(min 21

0)()( dxAxc Tkk

)](,),([)( 1 xcxcxA m

Description of rSQP

• To compute the search direction , the null-space approach is used. The solution is written as:

• Where is an matrix spanning the null space of , is an matrix spanning the range of .

• and


ZkYkk pZpYd

kZ )( mnn TkA kY mn

kA

0kTk ZA

Description of rSQP

• The QP sub-problem can be expressed by:

• The solution is :


ZkkTk

TZZ

TYkk

Tkk

Tk

RppZWZpppYWZgZ

mnZ

)()(min 21

][)( 1Ykk

Tkk

Tkkk

TkZ pYWZgZZWZp

Description of rSQP

• The components of x are grouped into m basic, or dependent variables and non-basic or control variables. The columns of A are grouped accordingly:


mn

])()([)( xNxCxA T

)()(

)(1 xNxC

xZ

0)(xY

Description of rSQP

• When the number of variables n is large and the number of degrees of freedom n-m is small, it is attractive to approximate the reduced Hessian .

• To ensure that good search directions are always generated, the algorithm approximates the cross term by a vector :


kkTk ZWZ

YkkTk pYWZ kw

kYkkTk wpYWZ ][

Description of rSQP

• is approximated by a quasi-Newton matrix

• The reduced Hessian matrix is approximated by a positive definite quasi-Newton matrix


kTkWZ kS

Ykkk pYSw

kkTk ZWZ

kB

Implementation of rSQP

• Update S


kTk

Tkkkk

kk ss

ssSySS

)(1

kTkk

Tkk gZgZy 11

kkk xxs 1

Implementation of rSQP

• Update B


kTk

Tkk

kkTk

kTkkk

kk sy

yy

sBs

BssBBB 1

Zkk ps

kkTkk

Tkk wgZgZy 11

Summarize

• The algorithm does not require the computation of the Hessian of the Lagrangian.

• The algorithm only makes use of first derivatives of objective function and constraints.

• The reduced Hessian matrix is approximated by a positive definite quasi-Newton matrix.


Numerical examples• Model 1:

• degrees of freedom = 1


2ixf

111 10)1( jjj xxxg 99,,1 j

100,,1 i

Method Iterations Convergence Time

rSQP 5 4 sec

SQP 5 20 sec




2ixf

jnjnjj xxxg 2/2/ 10)1( 50,,1 j

100,,1 i


rSQP 11 5 sec

SQP 4 10 sec


• x0=[1.1, 1.1, ……, 1.1]



2ixf

111 10)1( xxxg jj 99,,1 j

100,,1 i


rSQP 3 3 sec

SQP 10 20 sec


• x0=[0.1, 0.1, ……, 0.1]



2ixf

111 10)1( xxxg jj 99,,1 j

100,,1 i


rSQP 3 2 sec

SQP 3 8 sec


• x0=[2.1, 2.1, ……, 2.1]




rSQP 4 4 sec

SQP 15 33 sec

2ixf

111 10)1( xxxg jj 99,,1 j

100,,1 i

Conclusion

• The algorithm is well-suited for large problems with few degrees of freedom.

• Reduced Hessian SQP approach saves the time of computing Hessian matrix, cuts down the cost of computation.

• Reduced Hessian SQP algorithm is at least as robust as SQP method.


Future work

• Use differential algebraic equations as constraints.

• Apply reduced Hessian SQP method to batch and continuous processes.


Documents

Zhen Lu CPACT University of Newcastle MDC Technology Reduced Hessian Sequential Quadratic Programming(SQP)