AN AUTOMATED METHOD FOR SENSITIVITY ANALYSIS USING

AIAA-2000-0689

AN AUTOMATED METHOD FOR SENSITIVITY ANALYSIS USINGCOMPLEX VARIABLES

Joaquim R. R. A. Martins ∗ , Ilan M. Kroo † and Juan J. Alonso ‡

Department of Aeronautics and AstronauticsStanford University, Stanford, CA

Abstract

The complex-step method for calculating sensitivi-ties and its use in numerical algorithms is presented.A general procedure for the implementation of thismethod is described in detail and a script is devel-oped that automates its implementation. The nu-merical examples include the automatic conversionof a structural finite element and a two-dimensionalcomputational fluid dynamics code. In both ofthese examples, the complex-step method is com-pared with other existing methods, namely finite-differencing, automatic differentiation and an ana-lytic method. The complex-step method is shownto have implementation advantages over automaticdifferentiation and computational advantages overfinite-differencing.

Introduction

Sensitivity analysis has been an important area ofengineering research, being particularly useful in de-sign optimization. In choosing a method for com-puting sensitivities, one is mainly concerned withits accuracy and computational expense. In certaincases it is also important that the method be easilyimplemented.

One method that is very commonly used is finite-differencing. Although it is not known for being par-ticularly accurate or computationally efficient, thismethod’s biggest advantage resides in the fact thatit is extremely easy to implement.

Analytic methods, on the other hand, are typicallymuch more accurate and efficient but they require

∗Graduate Student, AIAA Student Member†Professor, AIAA Associate Fellow‡Assistant Professor, AIAA Member

Copyright c�2000 by the authors. Published by the AmericanInstitute of Aeronautics and Astronautics, Inc., with permis-sion.

the derivation and development of a program that isspecific to each case.

Yet another technique, called Automatic Differ-entiation for Fortran (ADIFOR),1 uses a script thatautomatically differentiates an existing code by cre-ating a new one that calculates the required sensitiv-ities. This approach has the advantages of the firsttwo methods mentioned above: being both accurateand relatively easy to implement.

The use of complex variables to develop estimatesof derivatives originated with the work of Lyness andMoler2 and Lyness.3 Their papers introduced sev-eral methods that made use of complex variables,including a reliable method for calculating the nth

derivative of an analytic function. However, onlyrecently has some of this theory been rediscoveredby Squire and Trapp4 and used to obtain a verysimple expression for estimating the first derivative.This estimate is suitable for use in modern numer-ical computing and has shown to be very accurate,extremely robust and surprisingly easy to imple-ment, while retaining a reasonable computationalcost. The potential of this technique is now start-ing to be recognized and it has been used in CFDsensitivity analysis by Anderson5 and in a MDO en-vironment by Newman.6

The objective of this paper is to shed new lighton the theory behind the complex-step derivativeapproximation, demonstrate its advantages in com-puting sensitivities and provide a script that can im-plement it automatically. There will also be a com-parison with other methods, with emphasis on finite-differencing and ADIFOR, since these are also in theclass of methods that have a relatively straightfor-ward implementation.

1

American Institute of Aeronautics and Astronautics

Theory

Analyticity

The use of complex variables in the evaluation offunction sensitivities is sometimes better understoodif one makes the following analogy by Saff andSnider.7 Consider the set of rational numbers andthe fact that no rational number can solve x2 = 2.If we want to extend the set of rational numbers sothat are able to solve this equation we can write:

w = x +√

2y (1)

where x and y are rational numbers and a numberof this form can now solve x2 = 2.

Now suppose that we already have the set of realnumbers. With real numbers alone, we still cannotsolve x2 = −1. If we want to solve this equation,we can adopt the same approach as before and ex-tend the set of real numbers by defining a set of newnumbers,

z = x + iy (2)

where x and y are real and i =√−1. This defines

the set of complex numbers which not only enablesus to solve x2 = −1, but in fact any polynomial ofdegree n.

Now lets consider to our extension of the set of ra-tional numbers represented by w. If a function f(w)is differentiable, its derivative must be the same,whether we differentiate it with respect to the ra-tional part or the irrational one, i.e.,

∂f

∂w=

∂f

∂x=√

2∂f

∂y(3)

The same is true for a function of a complex vari-able that is differentiable in the complex plane andsimilarly we can write,

∂f

∂y= i

∂f

∂x(4)

The real and imaginary parts of the function can beseparated and we can define,

f = u + iv. (5)

Comparing the real and imaginary parts ofEquation(4) we obtain the familiar Cauchy-Riemannequations,

∂u

∂x=

∂v

∂y(6)

∂u

∂y= −∂v

∂x. (7)

A complex function that satisfies these equations issaid to be analytic, which is to say that it is differ-entiable in the complex plane.

The main goal of this analogy between complexnumbers and an extension of rational numbers is toremind ourselves that a complex number is reallyjust one number. It is not unusual to forget this andthink instead of a complex number as two numbersjust because we do not have a way of representingit with a single axis or by a single term. This isan important realization that will be useful in ex-plaining some of the implementation issues that aredescribed later.

First Derivative Approximations

Finite-differencing formulas are a common methodfor estimating derivatives. These formulas can be de-rived by truncating a Taylor series which has beenexpanded about a given point x. A common esti-mate for the first derivative is the forward-differenceformula,

f�(x) ≈ f(x + h)− f(x)

h, (8)

where h is finite-difference interval. The truncationerror is O(h), and therefore it is a first-order approx-imation. For a second-order estimate the we can usethe central-difference formula,

f�(x) ≈ f(x + h)− f(x− h)

2h. (9)

As with any divided-difference approximation, weare faced in this case with the “step size dilemma”,i.e. wanting to choose a small step size that min-imizes truncation error while avoiding the use of astep size so small that subtractive cancellation errorsbecome inevitable. Note that this approximation isa discretization of the mathematical definition of thefirst derivative.

We will now see that an equally simple first deriva-tive estimate for real functions can be obtained usingcomplex calculus. If f is an analytic function, thenthe Cauchy-Riemann equations apply, establishingthe exact relationship between the real and imagi-nary parts of the function. We can use the defini-tion of a derivative in the right hand side of the firstCauchy-Riemann Equation(6) to obtain,

∂u

∂x= lim

h→0

v(x + i(y + h))− v(x + iy)h

. (10)

2


Since the functions that we are interested in are realfunctions of a real variable, we restrict ourselves tothe real axis and then we know that y = 0, u(x) =f(x) and v(x) = 0. Equation (10) can then be re-written as,

∂f

∂x= lim

h→0

Im [f (x + ih)]h

. (11)

For a small discrete h, this can be approximated by,

∂f

∂x≈ Im [f (x + ih)]

h, (12)

which we will call the complex-step derivative ap-proximation. This estimate is not subject to sub-tractive cancellation error, since it does not involvea difference operation. This constitutes a tremen-dous advantage over the finite-difference approachexpressed in Equation (8).

The property that was used to derive this approx-imation is a direct consequence of the analyticity ofthe function and it is therefore necessary that thefunction f be analytic in the complex plane. In alater section we will discuss to what extent a genericnumerical algorithm can be considered to be an an-alytic function.

In order to determine the error involved in thisapproximation, we repeat the derivation by Squireand Trapp4 which is based on a Taylor series expan-sion. Rather than using a real step h, we use a pureimaginary step, ih. If f is a real function in realvariables and it is also analytic, we can expand it ina Taylor series about a real point x as follows,

f(x + ih) = f(x) + ihf�(x)−

h2 f ��(x)

2!− ih

3 f ��(x)3!

+ . . . (13)

Taking the imaginary parts of both sides of Equa-tion (13) and dividing the equation by h yields

f�(x) =

Im [f(x + ih)]h

+ h2 f ��(x)

3!+ . . . (14)

Hence the approximations is a O(h2) estimate of thederivative of f . The reason we achieve a second-order approximation with just one function evalua-tion has to do with the fact that the imaginary partof the result is an odd function with respect to theimaginary part of the independent variable.

Numerical Example

Because the complex-step approximation does notinvolve a difference operation, we can choose ex-tremely small steps sizes with no loss of accuracydue to subtractive cancellation.

To illustrate this, consider the following analyticfunction:

f(x) =ex

√sin3x + cos3x

(15)

The exact derivative at x = 1.5 was computed an-alytically to 16 digits and then compared to the re-sults given by the complex-step (12) and the forwardand central finite-difference formulas (8,9).

Figure 1: Normalized error in the sensitivity esti-mates given by finite-difference and the complex-step with the analytic result as the reference; ε =|f �−f �ref ||f �ref |

.

The forward-difference estimate initially con-verges to the exact result at a linear rate since itstruncation error is O(h), and the central-differenceconverges quadratically, as expected. However, asthe step is reduced to a value of about 10−8 forthe forward-difference and 10−5 for the central-difference, subtractive cancellation errors become anissue and the estimates become unreliable. Whenthe interval h is so small that no difference exists inthe output (for steps smaller than 10−16) the finite-difference estimates eventually yields zero and thenε = 1.

The complex-step estimate converges quadrati-cally with decreasing step size, as predicted by thetruncation error estimate. The estimate is practi-cally insensitive to small step sizes and below an h

of the order of 10−8 it achieves the accuracy of thefunction evaluation.

Comparing the best accuracy of each of these ap-proaches, we see that by using finite-difference weonly achieve a fraction of the accuracy that is ob-tained by using the complex-step approximation.

3


As we can see the complex-step size can be madeextremely small. However, there is a lower limiton the step size when using finite precision arith-metic. The range of real numbers that can be han-dled in numerical computing is dependent on theparticular compiler that is used. Usually, however,when double-precision complex numbers are used,the smallest non-zero number that can be repre-sented is 10−308. If a number falls below this value,underflow occurs and the number drops to zero.Note that the estimate is still accurate down to astep of the order of 10−307. Below this, underflowoccurs and the estimate results in NaN. Therefore,the smallest possible h is the one below which un-derflow occurs somewhere in the algorithm.

When it comes to comparing the relative accu-racy of complex and real computations, analysisshows that there is an increased error in basic arith-metic operations when using complex numbers, morespecifically when dividing and multiplying.8

Higher Derivative Approximations

The derivative of order n of a given analytic functioncan be calculated by Cauchy’s Integral Formula inits general form:7

f(n)(z) =

n!2πi

�

Γ

f(ξ)(ξ − z)n+1

dξ, (16)

where Γ is a simple closed positively oriented contourthat encloses z. This integral can be numericallycomputed by means of a mid-point trapezoidal ruleapproximation around a circle of radius r yielding,9

f(n)(z) ≈ n!

mr

m−1�

j=0

f

�z + rei 2πj

m

�

ei 2πjnm

, (17)

where if m is the number of points used in the in-tegration, we can approximate a derivative of ordern = 0, 1, . . . ,m− 1.

When comparing conventional finite-differencemethods and the complex integration expressed inEquation (17), we observe that both use formulas ofthe type

�aif(xi) where the coefficients have differ-

ent signs. However, there is a significant differencebetween the two. In conventional methods the step h

has to be decreased in order to reduce the truncationerror of the approximation, making it susceptible tosubtractive cancellation. If we want to reduce thetruncation error of the complex integration methodall we need to do is to increase the number of func-tion evaluations, i.e. m in Equation (17). This keeps

the subtractive cancellation error constant and it isthen possible to calculate a bound on the error in-volved in the approximation.2

The complex-step first derivative approxima-tion (12) can be derived from Equation (17). Fromcomplex variable theory, for a real function of thereal variable that is analytic,

f(x + iy) = u + iv ⇒ f(x− iy) = u− iv. (18)

By setting m = 2 in Equation (17) and starting theintegration at the top of the circle (z + rei π

2 ) ratherthan on the left side (z + r) we obtain the second-order approximation that we arrived at previously.This is the only approximation that can be obtainedfrom Equation (17) that does not involve subtractionand it is only valid for functions whose imaginarypart is zero on the real axis.

Implementation

Intrinsic Complex Functions and Operatorsin Fortran

In the derivation of the complex-step derivative ap-proximation (12) for a function f we have assumedthat f was an analytic function, i.e. that theCauchy-Riemann equations apply. It is therefore im-portant to examine to what extent this assumptionholds when the value of the function is calculated bya numerical algorithm. In addition it is also usefulto explain how we can convert real functions and op-erators such that they can take complex numbers asarguments. Fortunately, in the case of Fortran, com-plex numbers are a standard data type and manyintrinsic functions are already defined for them.

Any algorithm can be broken down into a se-quence of basic operations. Two main types of oper-ations are relevant when converting a real algorithmto a complex one:

• Relational operators

• Arithmetic functions and operators.

Relational logic operators such as “greater than”and “less than” are not defined for complex num-bers in Fortran. These operators are usually usedin conjunction with if statements in order to redi-rect the execution thread. The original algorithmand its “complexified” version must obviously fol-low the same execution thread. Therefore, definingthese operators to compare only the real parts of thearguments is the correct approach.

4


Functions that choose one argument such as maxand min are based on relational operators. There-fore, according to our previous discussion, we shouldonce more choose a number based on its real partalone and let the imaginary part “tag along”.

Any algorithm that uses conditional statements islikely to be a discontinuous function of its inputs.Either the function value itself is discontinuous orthe discontinuity is in the first or higher derivatives.When using a finite-difference method, the deriva-tive estimate will be incorrect if the two functionevaluations are within h of the discontinuity loca-tion. However, if the complex-step is used, the re-sulting derivative estimate will be correct right upto the discontinuity. At the discontinuity, a deriva-tive does not exist by definition, but if the functionis defined a that point, the approximation will stillreturn a value that will depend on how the functionis defined at that point.

Arithmetic functions and operators include addi-tion, multiplication, and trigonometric functions, toname only a few, and most of these have a standardcomplex definition that is analytic almost every-where. Many of these definitions are implementedin Fortran. Whether they are or not depends onthe compiler and libraries that are used. The usershould check the documentation of the particularFortran compiler being used in order to determinewhich intrinsic functions need to be redefined. Afull description of Fortran’s standard intrinsic func-tions and their complex definitions can be found inTable 5.

Functions of the complex variable are merely ex-tensions of their real counterparts. By requiring thatthe extended function satisfy the Cauchy-Riemannequations, i.e. analyticity, and that its propertiesbe the same as those of the real function, we canobtain a unique complex function definition.7 Sincethese complex functions are analytic, the complex-step approximation is valid and will yield the correctresult.

Some of the functions, however, have singulari-ties or branch cuts on which they are not analytic.This does not pose a problem since, as previouslyobserved, the complex-step approximation will re-turn a correct one-sided derivative. As for the caseof a function that is not defined at a given point,the algorithm will not return a function value, so aderivative cannot be obtained. However, the deriva-tive estimate will be correct in the neighborhood ofthe discontinuity.

The only standard complex function definitionthat is non-analytic is the absolute value function

or modulus. When the argument of this function isa complex value, the function returns a positive realnumber, |z| =

�x2 + y2. This function’s definition

was not derived by imposing analyticity and there-fore it will not yield the correct derivative when us-ing the complex-step estimate. In order to derive ananalytic definition of abs we start by satisfying theCauchy-Riemann equations. From the Equation (6),since we know what the value of the derivative mustbe, we can write,

∂u

∂x=

∂v

∂y=

�−1 ⇐ x < 0+1 ⇐ x > 0

. (19)

From Equation (7), since ∂v/∂x = 0 on the real axis,we get that ∂u/∂y = 0 on the axis, so the real partof the result must be independent of the imaginarypart of the variable. Therefore, the new sign of theimaginary part depends only on the sign of the realpart of the complex number, and an analytic “abso-lute value” function can be defined as:

abs(x + iy) =

�−x− iy ⇐ x < 0+x + iy ⇐ x > 0

. (20)

Note that this is not analytic at x = 0 since a deriva-tive does not exist for the real absolute value. Onceagain, the complex-step approximation will give thecorrect value of the first derivative right up to thediscontinuity. Later the x > 0 condition will besubstituted by x ≥ 0 so that we not only obtain afunction value for x = 0, but also we are able to cal-culate the correct right-hand-side derivative at thatpoint.

Automatic Implementation

In order to implement the complex-step method, onemust have access to the source code that computesthe value of f . The implementation procedure canbe summarized as follows:

• Substitute all real type variable declarationswith complex declarations. It is not strictlynecessary to declare all variables complex, butit is much easier to do so.

• Define all functions and operators that are notdefined for complex arguments.

• A complex-step can then be added to the de-sired x and ∂f

∂x can be estimated using Equa-tion (12).

5


The method has been successfully implemented byhand on a Fortran three-dimensional CFD code.5,6This was done by writing a set of subroutines witha different name from the intrinsic functions thatneeded to be defined for complex arguments. Thenthe code was processed line by line, and some of theintrinsic function names where substituted with thenew function names. Also, in some if statementsthe complex arguments in the comparisons must becast to real. This procedure requires an advancedknowledge of the type of arguments, which is notalways a trivial matter.

Fortunately, in Fortran 90, intrinsic functions andoperators (including comparison operators) can beoverloaded. This means that if a particular func-tion or operator does not take complex arguments,one can extend it by writing another definition thattakes this type of arguments. This feature makes itmuch easier to implement the complex-step methodsince once we overload the functions and operators,there is no need to change the function calls or condi-tional statements. The compiler will automaticallydetermine the argument type and choose the correctfunction or operation.

All the functions that need to be overloaded andtheir new definitions are listed in Table 5. Addingthe extended function definitions is simple, since ithas been written as a Fortran 90 module that canbe used by any subroutine in a program.

In order to automate the implementation, a scriptthat processes Fortran source files automatically wasdeveloped. The script declares the complex func-tions module in every existing subroutine, substi-tutes all the real type declarations by complex onesand adds implicit complex statements when ap-propriate. The script was written in Python10,11

and uses the Perl regular expressions module. Thelatest versions of both the script and the Fortran 90module, are available from the first author.12

The third step of the implementation, adding thecomplex-step to the variable of interest and using thecomplex-step approximation, is left to the user. An-other task that must be done manually is the changeof file I/O statements, since the user might want tohave I/O files containing either full complex num-bers or just their real parts.

Implementation in Other Programming Lan-guages

C/C++: Neither C nor C++ perform complexarithmetic by default, although there are com-plex arithmetic libraries than one can use. In

C++, all operators and functions can be over-loaded just as in Fortran 90, so automatic im-plementation seems to be perfectly feasible.

Matlab: As in the case of Fortran, one must re-define functions such as abs, max and min. Alldifferentiable functions are defined for complexvariables. Results for the simple example in theprevious section were computed using Matlab.The standard transpose operation representedby an apostrophe (’) poses a problem as ittakes the complex conjugate of the elements ofthe matrix, so one should use the non-conjugatetranspose represented by “dot apostrophe” (.’)instead.

Java: Complex arithmetic is not standardized atthe moment but there are plans for its imple-mentation. Although function overloading ispossible, operator overloading is currently notsupported.

Python: When using the Numerical Python mod-ule (NumPy), we have access to complex num-ber arithmetic and implementation should be asstraightforward as in Fortran.

Results

The complex-step method was implemented auto-matically in two different analysis codes in order tocalculate sensitivities. The first one is a structuralfinite element solver and the second one is a two-dimensional CFD code. The calculated sensitivitiescan then be used to perform design optimization ina multidisciplinary environment.

Structural Sensitivities

For purposes of comparison of the complex-step ap-proximation with other sensitivity analysis methods,we used a structural solver based on a finite elementcode, fesmeh, developed by Holden.13 The sensitiv-ities are calculated in order to perform design opti-mization in a multidisciplinary environment and thecomplex-step method was automatically.

In this example, a transonic transport wing ismodeled with multiple spars, shear webs, and ribslocated at various spanwise stations, together withthe skins of the upper and lower surfaces of the wingbox.

Two types of finite elements are used: truss andtriangular plane-stress plate elements. Both elementtypes have 3 translational degrees of freedom per

6


node, so the truss has a total of 6 degrees of freedomwhile the plate has 9 degrees of freedom. Figure 2shows an expanded view of the finite element model.

Figure 2: Structural finite element model of wing

In a first study, the sensitivity estimates given bythe complex-step and forward-finite-difference meth-ods are compared for a varying step size. The samplesensitivity chosen for this study was the derivativeof the stress in a spar cap with respect to its owncross-sectional area. This kind of derivative is veryimportant in structural optimization, where stressconstraints are usually required.

Note that in this case, stress is a non-linear func-tion of the cross-sectional area and, therefore, weshould be able to observe the rate convergence ofthe estimates for decreasing step sizes.

Figure 3 shows a comparison of results – analo-gous to that of Figure 1 – with a reference derivativewhich is obtained by an analytic method. The ana-lytic method used here is the direct method appliedto a discrete linear set of equations as described byAdelman.14 This method is included here only toprovide a benchmark, since it has an implementa-tion that is far more involved and code-specific thanthe other ones.

As expected, the error of the finite-difference esti-mate initially decreases at a linear rate. As the stepis reduced to a value of about 10−6, subtractive can-cellation errors become increasingly significant andthe estimate error increases. For even smaller per-turbations – of the order of 10−17 – no difference ex-ists in the output and the finite-difference estimateeventually goes to zero (ε = 1).

The complex-step estimate converges quadrati-cally with decreasing step size, converging to thecode’s precision when h is of the order of 10−7. Note

Figure 3: Error of sensitivity estimates given byfinite-difference and complex-step with the analytic

method result as the reference; ε = |f �−f �ref ||f �ref |

.

that the estimate is still accurate down to a step ofthe order of 10−306. Below this, underflow starts tooccur, the estimate is corrupted, and eventually theresult becomes meaningless.

A second study compares the accuracy and com-putational cost between the three methods men-tioned above and an additional method: ADIFOR.1This is a package that automatically processes agiven Fortran program, producing a new programthat in addition to the original computations alsocalculates the desired sensitivities. A sample of thesensitivity results is shown in Table 1, and a com-putational comparison is made in Table 2. The costvalues are normalized with respect to the compu-tation time and memory usage of the complex-stepmethod.

The computations were performed on a SGI Oc-tane and correspond to the calculation of the sen-sitivities of the stress in all of the 60 trusses withrespect to their cross-sectional areas, i.e., a total of3600 sensitivities. The sample sensitivity is the sameone that was used to produce the results shown pre-viously in Figure 3. When compiled using doubleprecision, the finite element solver has an accuracyof about 13 digits, so the last 4 digits should be ig-nored.

The finite-difference sensitivity estimate – shownat the bottom of Tables 1 and 2 – is obtained us-ing an optimal step (h = 10−7). Even then, theestimate is shown to be only half as accurate as theother ones. Although this method is extremely easyto implement, finding a step that gives reasonably

7


Method Sample SensitivityComplex −39.049760045804646ADIFOR −39.049760045809059Analytic −39.049760045805281FD −39.049724352820375

Table 1: Sensitivity estimate accuracy comparison

Method Time MemoryComplex 1.00 1.00ADIFOR 2.33 8.09Analytic 0.58 2.42FD 0.88 0.72

Table 2: Relative computational cost comparison forthe calculation of the complete Jacobian

accurate estimates is usually a problem and, there-fore, the total computation time in practice is muchhigher than the one that is shown in these results.

The analytic method was accurate, as expected,and by far the fastest. The computation using thismethod required considerably more memory since ahost of new variables were introduced in the algo-rithm. As mentioned before, its implementation ismuch more involved than in the other cases and thisplaces this method in a class of its own.

ADIFOR produced very accurate estimates but itwas the costliest, with respect to both computationtime and memory usage. This has to do with thefact that that ADIFOR produces a code which isis much larger than the original one which containsmany more statements and variables. This fact con-stitutes an implementation disadvantage as it be-comes impractical to debug this new code. One hasto work with the original source and every time itis changed (or when we want to compute differentderivatives) one must first run ADIFOR and thencompile the new version.

The complex-step method was also accurate to thecode’s precision, was reasonably fast, and used lessmemory than any other method with the exceptionof the finite-difference method. The results were ob-tained using h = 10−100. In general, the memoryrequirement will always be greater than in the caseof finite-differencing, but never more than twice asmuch. As opposed to ADIFOR, the new “complexi-fied” code is practically identical to the original oneand can therefore be worked on directly. With thehelp of a few compiler flags one can even produce asingle code that can be chosen to be real or complexat compilation.

Finally, note that the relative cost values givenin Table 2 may vary for different problems, sincethey depend heavily on the ratio of the number ofoutputs we want to differentiate to the number ofdesign variables we want to differentiate with respectto. However, the costs associated with the complex-step method will always be proportional to those ofthe finite-difference method.

Aerodynamic Sensitivities

To further validate the complex-step method andtest the automatic implementation process on aCFD code, we chose to “complexify” FLO82. This isa two-dimensional, cell-centered, finite volume solverfor the complete Euler equations. It achieves fastturnaround through the use of several techniquesfor convergence acceleration, namely multigrid, im-plicit residual smoothing, enthalpy damping, and lo-cal time-stepping. The solution is updated explic-itly at every multigrid iteration using a modifiedRunge-Kutta algorithm which treats the convectiveand dissipative portions of the residual separatelyfor optimum error damping properties. FLO82 hasmultiple options for artificial dissipation schemes,ranging from the baseline Jameson-Schmidt-Turkel(JST) algorithm to advanced schemes such as CUSP,ECUSP, and HCUSP.15 FLO82 was chosen as an ex-ample for this work, since it contains the essentialnumerical schemes present in more complicated pro-grams such as FLO87 and FLO107 that solve three-dimensional Euler and Navier-Stokes flows, and arecurrently used in our design work.

Figure 4: RAE 2822 airfoil and grid geometry.

8


The implementation process, even in this rela-tively large program, was carried out very quicklywith the use of the “complexifying” script. Afterless than one hour, the new solver was calculatingthe correct sensitivities.

The test case was chosen to be the RAE 2822 air-foil shown in Figure 4 at a an angle-of-attack of 3degrees with a freestream Mach number of 0.7. Theresultant pressure coefficient distribution is plottedin Figure 5.

1.

20

0.80

0.

40

0.00

-0.

40 -

0.80

-1.

20 -

1.60

-2.

00

Cp

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

+

+

+

+

+

+

+

+

+

+++++++++++++++++++++++++++++++++++++++++++++++++++

+

++

+++++++++++++++++++++++++++

Figure 5: Pressure coefficient distribution, Mach =0.7, α = 3.0o.

As we have seen in the previous numerical ex-amples, it is rather useful to study the variationof the sensitivity estimates with step size. For thisstudy, we computed the derivative of drag coefficientwith respect to the free stream Mach number, i.e.∂CD/∂M∞. The results are shown in Figure 6, andthe reference estimate used to calculate the errorwas, in this case, the last estimate calculated by thecomplex-step method, corresponding to h = 10−38.The behavior of the estimates follows once more, thesame general pattern: both estimates converge ini-tially, the forward-finite-difference estimate then re-bounds to zero, and the complex-step estimate main-

tains the accuracy of the code until underflow takesplace. Again, note that the range of h for which thefinite-difference gives a reasonable estimate is verysmall, and therefore a convergence study would benecessary if we were to use this method in designoptimization. Even then, there would be no guar-antee that the optimal step would remain the samethroughout the design process.

10−3010−2010−10

10−8

10−6

10−4

10−2

100

Step Size, h

Nor

mal

ized

Erro

r, ε Complex−Step

Finite−difference

Figure 6: Normalized error of sensitivity estimates of∂CD/∂M∞ The reference value is the last complex-

step estimate; ε = |f �−f �ref ||f �ref |

.

In Table 3 we list the sensitivity matrix of thelift, drag and moment coefficients with respect toangle-of-attack and freestream Mach number. Forthe complex-step, we used an arbitrary small stepsize of 10−20, while for the finite-difference, the op-timum step sizes were found to be 10−8 for α and10−6 for M∞. The digits that are shown in italicsin the table are the ones that have not converged.

Since the solver is iterative, we also analyzed theconvergence of the sensitivity estimates and com-pared it to the convergence of the algorithm. Fig-ure 7 shows the evolution of the sensitivity residualfor both the finite-difference and the complex-stepestimates of ∂CD/∂M∞ during the process of a typi-cal calculation lasting 150 iterations. The sensitivityresidual is defined as the absolute value of the dif-ference between the current estimate of ∂CD/∂M∞and its value at the previous iteration. To measurethe convergence of the algorithm, we used the aver-age residual of the density equation. As expected,the rate of convergence of the sensitivity estimate ofthe finite-difference method tracks that of the flowsolver. The fact that the rate of convergence of the

9


Sensitivity Complex FD∂CL/∂α 12.37054751691092 12.370726665267277∂CD/∂α 0.8602380269441042 0.86024234610682058∂CM/∂α −0.5026301652982372 −0.5026313670830616∂CL/∂M∞ 3.2499722985150532 3.2499447577549727∂CD/∂M∞ 0.43978671505102005 0.43978998888818932∂CM/∂M∞ −0.99037388394690016 −0.9903747405504145

Table 3: Lift, drag and moment coefficient sensitivities with respect to angle-of-attack and Mach number.

complex-step estimate is the same as the one for theother two is rather reassuring.

0 50 100 15010−10

10−8

10−6

10−4

10−2

100

102

104

106

108

Number of Iterations

Res

idua

l

Complex−Step Finite−difference Average density residual

Figure 7: Residual for the estimates of ∂CD/∂M∞compared to the convergence of the code.

The last set of results presents the calculation ofthe sensitivity of the drag coefficient with respect toa series of airfoil shape parameters. The use of thesesensitivities is commonplace in aerodynamic shapeoptimization problems. The shape parameters thatare used in this study are the amplitudes of a seriesof Hicks-Henne “bump” functions that are centeredat various points along the upper surface of the air-foil, from the leading edge to the 90% chord loca-tion. Hicks-Henne “bump” function have been usedin previous work16 and have the desirable propertythat, when applied to a smooth baseline geometry,they produce a smooth surface. The plot in Figure 8shows the sensitivities at 55 grid points distributedalong the top surface of the airfoil. The values ofthe sensitivities are interpolated using cubic splines.The complex-step size was set to h = 10−10 andfor the finite-difference step the optimal value wasfound to be h = 10−9. The plot shows no discernibledifference bettween the two sets of results and themaximum difference between the two was calculated

5 10 15 20 25 30 35 40 45 50 550

1

2

3

4

5

6

7

8

9

10

11

Design Points

Dra

g C

oeffi

cien

t Sen

sitiv

ity

Complex−Step Finite−difference

Figure 8: Comparison of the estimates for the shapesensitivities of the drag coeffiecient, ∂CD/∂bi.

Method Time MemoryComplex 1.00 1.00FD 0.31 0.55

Table 4: Relative computational cost comparison forthe calculation of the complete shape sensitivity vec-tor.

to be 2.1× 10−4.A comparison of the relative computational cost

of the two methods was also made for the aerody-namic sensitivities, namely for the calculation of thecomplete shape sensitivity vector. Table 4 lists thesecosts, normalized with respect to the complex-stepresults.

The complex-step exhibits, once more, a highercost than the finite-difference method. However, inthis case the difference is significantly higher thanthe one observed for the structural solver. In spiteof this substantial difference, it is still usually ad-vantageous to use the complex-step method, sinceno additional computational effort is necessary in

10


order to find an adequate step.

Conclusions

The theory behind the application of the complex-step method to real-world numerical algorithms wasintroduced. The Cauchy-Riemann equations andthe fact that a complex number is really just onenumber, were vital in the rationalization of the prac-tical implementation of this method.

The implementation process of the complex-step derivative approximation was successfully au-tomated for two large solvers, including an iterativeone. The resulting derivative estimates were val-idated by comparing them to results obtained byother known methods.

The complex-step method, unlike the finite-difference method, has the advantage of being stepsize insensitive and for small enough steps, the ac-curacy of the sensitivity estimates is only limited bythe numerical precision of the algorithm.

This method is now at least as easy to implementas ADIFOR and once it is implemented it is mucheasier to maintain the resulting code.

Acknowledgements

The authors would like to thank Professor JamesLyness for his invaluable help and support.

References

[1] Bischof, C., A. Carle, G. Corliss, A. Grienwank,P. Hoveland, “ADIFOR: Generating DerivativeCodes from Fortran Programs”, Scientific Pro-gramming, Vol. 1, No. 1, 1992, pp. 11-29.

[2] Lyness, J. N., and C. B. Moler,, “Numericaldifferentiation of analytic functions”, SIAM J.Numer. Anal., Vol. 4, 1967, pp. 202-210.

[3] Lyness, J. N., “Numerical algorithms based onthe theory of complex variables”, Proc. ACM22nd Nat. Conf., Thompson Book Co., Wash-ington DC, 1967, pp. 124-134.

[4] Squire, W., and G. Trapp, “Using ComplexVariables to Estimate Derivatives of Real Func-tions”, SIAM Review, Vol. 10, No. 1, March1998, pp. 100-112.

[5] Anderson, W. K., J. C. Newman, D. L. Whit-field, E. J. Nielsen, “Sensitivity Analysis for

the Navier-Stokes Equations on UnstructuredMeshes Using Complex Variables”, AIAA Pa-per No. 99-3294, Proceedings of the 17th Ap-plied Aerodynamics Conference, 28 Jun. 1999.

[6] Newman, J. C. , W. K. Anderson, D. L. Whit-field, “Multidisciplinary Sensitivity DerivativesUsing Complex Variables”, MSSU-COE-ERC-98-08, Jul. 1998.

[7] Saff, E. B., A. D. Snider, Fundamentals of Com-plex Analysis, Prentice Hall, New Jersey, 1976.

[8] Olver, F. W. J., “Error Analysis of ComplexArithmetic”, Computational Aspects of Com-plex Analysis, pp. 279-292, 1983.

[9] Lyness, J. N., and G. Sande, “ENTCAF andENTCRE Evaluation of Normalized Taylor Co-efficients of an Analytic Function”, Com. ACM,Vol. 14, No. 10, 1971, Washington DC, 1967,pp. 124-134.

[10] Lutz, M., Programming Python, O’Reilly, NewYork, Cambridge, 1996.

[11] http://www.python.org

[12] http://aero-comlab.stanford.edu/jmartins

[13] Holden, M. E., “Aeroelastic Optimization usingthe Collocation Method”, PhD Thesis, Stan-ford, May 1999.

[14] Adelman, H. M., R. T. Haftka, “Sensitiv-ity Analysis of Discrete Structural Systems”,AIAA Journal, Vol. 24, No. 5, May 1986.

[15] Jameson, A., W. Schmidt and E. Turkel, “Nu-merical Solution of the Euler Equations by Fi-nite Volume Methods Using Runge-Kutta TimeStepping Schemes”, AIAA Paper No. 81-1259,June, 1981.

[16] J. Reuther, J. J. Alonso and A. Jameson, “Con-strained Multipoint Aerodynamic Shape Op-timization Using an Adjoint Formulation andParallel Computers: Part I”, Journal of Air-craft, 36(1):51-60, 1999.

11


FortranFunction

MathematicalStandard Definition Analytic Continuation

abs ✘ abs(z) =

�−z ⇐ x < 0+z ⇐ x ≥ 0

First derivative discontinuous atx = 0.

exp ✔ ez = ex(cos y + i sin y)

sqrt ✔√

z =�|z|

�cos

�Arg(z)

2

�+ i sin

�Arg(z)

2

��Non-continuous on non-positivereal axis (x ≤ 0, y = 0).

sin ✔ sin(z) = eiz−e−iz

2i

cos ✔ cos(z) = eiz+e−iz

2

tan ✔ tan(z) = e−iz−eiz

e−iz+eiz Not defined when cos(z) = 0

log ✔ log(z) = log |z| + iArg(z)Not defined for z = 0. Non-continuous on non-positive realaxis (x ≤ 0, y = 0).

log10 ✔ log10(z) = log(z)log(10)

Not defined for z = 0. Non-continuous on non-positive realaxis (x ≤ 0, y = 0).

asin ✔ arcsin(z) = −i log[iz + (1− z2) 12 ]

acos ✔ arccos(z) = −i log[z + (z2 − 1) 12 ]

atan ✔ arctan(z) = i2 log

�1−iz1+iz

�Not defined when z = ±i.

atan2 ✔ arctan2(z1, z2) = arctan(z2/z1)

sinh ✔ sinh(z) = ez−e−z

2

cosh ✔ cosh(z) = ez+e−z

2

tanh ✔ tanh(z) = ez−e−z

ez+e−z

dim ✘ dim(z1, z2) =

�z1 − z2 ⇐ x1 > x2

0 ⇐ x1 ≤ x2

Non-continuous derivatives whenx1 = x2.

sign ✘ sign(z1, z2) =

�+|x1| ⇐ x2 ≥ 0−|x1| ⇐ x2 < 0

Non-continuous derivatives whenx2 = 0.

max ✘ max(z1, z2) =

�z1 ⇐ x1 ≥ x2

z2 ⇐ x1 < x2


min ✘ min(z1, z2) =

�z1 ⇐ x1 ≤ x2

z2 ⇐ x1 > x2


Table 5: Fortran intrinsic functions and their complex definitions. ✔ means the presented definition is thestandard mathematical one, ✘ mean the definition is new, z = x+ iy. Note that in the case of abs, althoughit has a standard definition for complex arguments, that definition is not analytic.

12


Documents

AN AUTOMATED METHOD FOR SENSITIVITY ANALYSIS USING