Upload
others
View
11
Download
0
Embed Size (px)
Citation preview
Hindawi Publishing CorporationMathematical Problems in EngineeringVolume 2013 Article ID 548487 8 pageshttpdxdoiorg1011552013548487
Research ArticleAugmented Arnoldi-Tikhonov Regularization Methods forSolving Large-Scale Linear Ill-Posed Systems
Yiqin Lin1 Liang Bao2 and Yanhua Cao3
1 Department of Mathematics and Computational Science Hunan University of Science and Engineering Yongzhou 425100 China2Department of Mathematics East China University of Science and Technology Shanghai 200237 China3Department of Mathematics North China Electric Power University Beijing 102206 China
Correspondence should be addressed to Liang Bao nlbaoyahoocn
Received 1 November 2012 Revised 18 March 2013 Accepted 19 March 2013
Academic Editor Hung Nguyen-Xuan
Copyright copy 2013 Yiqin Lin et alThis is an open access article distributed under the Creative CommonsAttribution License whichpermits unrestricted use distribution and reproduction in any medium provided the original work is properly cited
We propose an augmented Arnoldi-Tikhonov regularization method for the solution of large-scale linear ill-posed systems Thismethod augments the Krylov subspace by a user-supplied low-dimensional subspace which contains a rough approximation ofthe desired solution The augmentation is implemented by a modified Arnoldi process Some useful results are also presentedNumerical experiments illustrate that the augmented method outperforms the corresponding method without augmentation onsome real-world examples
1 Introduction
We consider the iterative solution of a large system of linearequations
119860119909 = 119887 (1)
where 119860 isin R119899times119899 is nonsymmetric and nonsingular and119887 isin R119899 We further assume that the coefficient matrix 119860 isof ill-determined rank that is all its singular values decaygradually to zero with no gap anywhere in the spectrumSuch systems are often referred to as linear discrete ill-posed problems and arise from the discretization of ill-posedproblems such as Fredholm integral equations of the first kindwith a smooth kernelThe right-hand side 119887 of (1) is assumedto be contaminated by an error 119890 isin R119899 which may stem fromdiscretization or measurement inaccuracies Thus 119887 = + 119890where is the unknown error-free right-hand side vector
We would like to compute the solution 119909 of the linearsystem of equations with the error-free right-hand side
119860119909 = (2)
However since the right-hand side in (2) is not availablewe seek to determine an approximation of 119909 by solving the
available system (1) or a modification Due to the ill condi-tioning of 119860 the system (1) has to be regularized in orderto compute a useful approximation of 119909 Perhaps the bestknown regularization method is Tikhonov regularization [1ndash3] which in its simplest form is based on the minimizationproblem
min119909isinR119899
119860119909 minus 1198872
+1
1205831199092
(3)
where 120583 gt 0 is a regularization parameter Here andthroughout this paper sdot denotes the Euclidean vectornorm or the associated induced matrix norm
After regulating the system (1) we need to compute thesolution 119909
120583of the minimization problem (3) Such a vector
119909120583is also the solution of
(119860119879
119860 +1
120583119868)119909 = 119860
119879
119887 (4)
Here and in the following 119868 denotes the identity matrixwhose dimension is conformed with the dimension used inthe context If 120583 is far away from zero then due to theill conditioning of 119860 119909
120583is badly computed while if 120583 is
close to zero 119909120583is well computed but the error 119909
120583minus 119909 is
2 Mathematical Problems in Engineering
quite large Thus the choice of a good value for 120583 is fairlyimportant Several methods have been proposed to obtain aneffective value for 120583 For example if the norm of the error119890 or a fairly accurate estimate is known the regularizationparameter is quite easy to determine by application of thediscrepancy principle The discrepancy principle proposesthat the regularization parameter 120583 can be chosen so that thediscrepancy 119887 minus 119860119909
120583satisfies
10038171003817100381710038171003817119887 minus 119860119909
120583
10038171003817100381710038171003817= 120578120576 (5)
where 120576 = 119890 and 120578 gt 1 is a constant see for example [4]for further details on the discrepancy principle
The singular value decomposition [5] of 119860 can be usedto determine the solution 119909
120583of the minimization problem
(3) For an overview of numerical methods for computingthe SVD we refer to [6] We remark that the computationaleffort required to compute the SVD is quite high even formoderately sized matrices
Many numerical methods using Krylov subspaces havebeen proposed for the solution of large-scale Tikhonovregularization problems (3)Themain idea of such algorithmshas been to first project the large problems onto some Krylovsubspace to produce problems with small size and then solvethe small-sized problems by the SVD For instance severalwell-established methods based on the Lanczos bidiagonal-ization process have been proposed for the solution of theminimization problem (3) see [7ndash10] and references thereinThese methods use the Lanczos bidiagonalization process toconstruct a basis of the Krylov subspace
K119898(119860119879
119860119860119879
119887)
= span 119860119879119887 119860119879119860119860119879119887 (119860119879119860)119898minus1
119860119879
119887
(6)
We remark that each Lanczos bidiagonalization step needstwo matrix-vector product evaluations one with 119860 and theother with 119860119879 Other methods using the Krylov subspace
K119898(119860 119860119887) = span 119860119887 1198602119887 119860119898119887 (7)
as the projection subspace have been also designed Forexample Lewis and Reichel [11] proposed to exploit theArnoldi process to produce a basis of the Krylov subspaceK119898(119860 119860119887) to obtain an approximation of the solution of
the Tikhonov regularization problem (3) Since each Arnoldidecomposition step requires only one matrix-vector evalua-tion with 119860 the approach based on the Arnoldi process mayrequire fewer matrix-vector product evaluations than thatbased on the Lanczos bidiagonalization process Moreoverthe methods based on the Arnoldi process do not requirethe adjoint matrix 119860119879 and hence are more appropriateto problems for which the adjoint is difficult to evaluateFor such problems we refer to [12] A similar Tikhonovregularizationmethod based on generalized Krylov subspaceis proposed in [13]
Some numerical methods without using the Tikhonovregularization technique have been already proposed to solvethe large-scale linear discrete ill-posed problem (1) Thesemethods include the range-restricted GMRES (RRGMRES)method [14 15] the augmented range-restricted GMRES(ARRGMRES) method [16] and the flexible GMRES (FGM-RES) method [17] The RRGMRES method determines the119898th approximation 119909
119898of (1) by solving the minimization
problem
min119909isinK119898(119860119860119887)
119860119909 minus 119887 119898 = 1 2 1199090= 0 (8)
The regularization is implemented by choosing a suitabledimension number 119898 see for example [18] The ARRGM-RES method augments the Krylov subspace K
119898(119860 119860119887)
by a low-dimensional user-supplied subspace The low-dimensional subspace is determined by vectors that are ableto represent the known features of the desired solutionThe augmented method can yield approximate solutions ofhigher accuracy than the RRGMRES method if the Krylovsubspace K
119898(119860 119860119887) does not allow representation of the
known featuresIn this paper we propose a new iterative method named
augmented Arnoldi-Tikhonov regularization method forsolving large-scale linear ill-posed systems (1) The newmethod is deduced by combining the Tikhonov regulariza-tion technique and the augmentation technique The aug-mentation is implemented by a modified Arnoldi process
The following summarizes the structure of this paperSection 2 describes the augmented Arnoldi-Tikhonov regu-larization method and some useful results Some real-worldexamples are presented in Section 3 Section 4 contains theconclusions
2 Augmented Arnoldi-TikhonovRegularization Method
We attempt to improve the Arnoldi-Tikhonov regularizationmethod proposed in [11] by augmenting the Krylov subspaceK119898(119860 119860119887) by a 119896-dimensional subspaceW which contains
a rough approximation of the desired solution of (2) Thenthe subspace of projection we will exploit in the following isof the form
K119898(119860 119860119887) +W = span 119860119887 1198602119887 119860119898119887 +W (9)
Let 119882 be the 119899 by 119896 matrix whose columns form anorthonormal basis of the space W For the purpose ofaugmentation by W we apply the modified Arnoldi process[16] to construct the modified Arnoldi decomposition
119860 [119882119881119898] = [119880 119881
119898+1]119867 (10)
where 119880 isin R119899times119896 119881119898+1
isin R119899times(119898+1) [119880 119881119898+1] has orthonor-
mal columns and119867 isin R(119898+119896+1)times(119898+119896) is an upperHessenbergmatrix We point out that the leading principal 119896 by 119896submatrix of 119867 is the upper triangular matrix in the QRfactorization [5] of 119860119882 that is119867 is of the form
Mathematical Problems in Engineering 3
Input 119860 isin R119899times119899 119887 isin R119899119882 isin R119899times119896119898Output 119880 isin R119899times119896 119881
119898+1isin R119899times(119898+1)119867 isin R(119898+119896+1)times(119898+119896)
(1) Compute QR-factorization 119860119882 = 119880119867(2) Compute V
1= 119860119887 minus 119880(119880
119879
119860119887) V1= V11003817100381710038171003817V11003817100381710038171003817 and set 119881
1= [V1]
(3) For 119895 = 1 2 119898 DoV119895+1= 119860V
119895
For 119894 = 1 2 119896 Doℎ119894119895+119896
= 119880( 119894)119879
V119895+1
V119895+1= V119895+1minus ℎ119894119895+119896119880( 119894)
End ForFor 119894 = 1 2 119895 Do
ℎ119894+119896119895+119896
= V119879119894V119895+1
V119895+1= V119895+1minus ℎ119894+119896119895+119896
V119894
End Forℎ119895+119896+1119895+119896
= V119895+110038171003817100381710038171003817V119895+1
10038171003817100381710038171003817
V119895+1= V119895+1ℎ119895+119896+1119895+119896
119881119895+1= [119881119895 V119895+1]
End For
Algorithm 1 Modified Arnoldi process
119867 =
[[[[[[[[[[[[[[[
[
ℎ11ℎ12
sdot sdot sdot ℎ1119896
ℎ1119896+1
ℎ1119896+2
sdot sdot sdot ℎ1119896+119898
d d
sdot sdot sdot
ℎ119896minus1119896minus1
ℎ119896minus1119896
ℎ119896minus1119896+1
ℎ119896minus1119896+2
sdot sdot sdot ℎ119896minus1119896+119898
ℎ119896119896
ℎ119896119896+1
ℎ119896119896+2
sdot sdot sdot ℎ119896119896+119898
ℎ119896+1119896+1
ℎ119896+1119896+2
sdot sdot sdot ℎ119896+1119896+119898
ℎ119896+2119896+1
ℎ119896+2119896+2
sdot sdot sdot ℎ119896+2119896+119898
d d
ℎ119896+119898119896+119898minus1
ℎ119896+119898119896+119898
ℎ119896+119898+1119896+119898
]]]]]]]]]]]]]]]
]
(11)
ThemodifiedArnoldi relation (10) is shared by othermethodssuch as GMRES-E [19] and FGMRES [20] which are alsoaugmented type methods
The modified Arnoldi process is outlined in Algorithm 1We remark that the modified Arnoldi process in this paperis slightly different from the one used by Morgan [19]to augment the Krylov subspace with some approximateeigenvectors In his method the augmenting vectors are putafter the Krylov vectors while in Algorithm 1 the augment-ing subspace containing a rough approximation solution isincluded in the projection subspace from the beginningIn general this can give better results at the start of theregularization method proposed in this paper
We remark that a loss of orthogonality can occur whenthe algorithm progresses see [21] A remedy is the so-called reorthogonalization where the current vector has tobe orthogonalized against previously created vectors Onecan choose between a selective reorthogonalization or a full
reorthogonalization against all vectors in the current aug-mented subspace In this paper we only use the full reorthog-onalization The full reorthogonalization can be done as aclassical or modifed Gram-Schmidt orthogonalization see[21] for details
We now seek to determine an approximate solution 119909120583119898
of (3) in the augmented Krylov subspace K119898(119860 119860119887) +W
After computing theQR factorization [119882119881119898] = 119876119877with119876 isin
R119899times(119898+119896) having orthonormal columns and 119877 isin R(119898+119896)times(119898+119896)
being an upper triangularmatrix we substitute119909 = [119882119881119898]119910
119910 isin R119898+119896 into (3) It yields the reduced minimizationproblem
min119910isinR119898+119896
1003817100381710038171003817119860 [119882119881119898] 119910 minus 119887
1003817100381710038171003817
2
+1
120583
1003817100381710038171003817[119882119881119898] 1199101003817100381710038171003817
2
= min119910isinR119898+119896
1003817100381710038171003817[119880 119881119898+1]119867119910 minus 119887
1003817100381710038171003817
2
+1
120583
10038171003817100381710038171198761198771199101003817100381710038171003817
2
4 Mathematical Problems in Engineering
= min119910isinR119898+119896
10038171003817100381710038171003817119867119910 minus [119880119881
119898+1]119879
11988710038171003817100381710038171003817
2
+(119868 minus 119875)1198872
+1
120583
10038171003817100381710038171198771199101003817100381710038171003817
2
(12)
in which 119875 = [119880119881119898+1][119880 119881
119898+1]119879 is an orthogonal projector
onto span([119880 119881119898+1]) Obviously the reduced minimization
problem (12) is equivalent to
min119910isinR119898+119896
10038171003817100381710038171003817119867119910 minus [119880119881
119898+1]119879
11988710038171003817100381710038171003817
2
+1
120583
10038171003817100381710038171198771199101003817100381710038171003817
2
= min119910isinR119898+119896
100381710038171003817100381710038171003817100381710038171003817100381710038171003817
[
[
119867
1
radic120583119877]
]
119910 minus [[119880119881119898+1]119879
119887
0]
100381710038171003817100381710038171003817100381710038171003817100381710038171003817
2
(13)
The normal equations of the minimization problem (13) is
(119867119879
119867 +1
120583119877119879
119877)119910 = 119867119879
[119880 119881119898+1]119879
119887 (14)
We denote the solution of the minimization problem (13) by119910120583119898
Then from (14) it follows that
119910120583119898= (119867
119879
119867 +1
120583119877119879
119877)
minus1
119867119879
[119880 119881119898+1]119879
119887 (15)
The approximate solution of (3) is
119909120583119898= [119882119881
119898] 119910120583119898 (16)
Since the matrix119867119879119867+ (1120583)119877119879119877 has a larger conditionnumber than the matrix [119867119879 (1radic120583)119877119879]119879 we apply the QRfactorization of [119867119879 (1radic120583)119877119879]119879 to obtain the solution 119910
120583119898
of (13) insteadTheQR factorization of [119867119879 (1radic120583)119877119879]119879 canbe implemented by a sequence of Givens rotations
Define
120601119898(120583) =
10038171003817100381710038171003817119887 minus 119860119909
120583119898
10038171003817100381710038171003817
2
(17)
Substituting 119909120583119898
= [119882119881119898]119910120583119898
into (17) and using themodified Arnoldi decomposition (10) yield
120601119898(120583) =
10038171003817100381710038171003817119887 minus 119860 [119882119881
119898] 119910120583119898
10038171003817100381710038171003817
2
=10038171003817100381710038171003817119887 minus [119880119881
119898+1]119867119910120583119898
10038171003817100381710038171003817
2
=10038171003817100381710038171003817[119880 119881119898+1]119879
119887 minus 119867119910120583119898
10038171003817100381710038171003817
2
+ (119868 minus 119875)1198872
(18)
Substituting 119910120583119898
in (15) into (18) we obtain
120601119898(120583) =
100381710038171003817100381710038171003817100381710038171003817
[119868 minus 119867(119867119879
119867 +1
120583119877119879
119877)
minus1
119867119879
] [119880119881119898+1]119879
119887
100381710038171003817100381710038171003817100381710038171003817
2
+ (119868 minus 119875) 1198872
(19)
Note that
119868 minus 119867(119867119879
119867 +1
120583119877119879
119877)
minus1
119867119879
= [120583119867119877minus1
(119867119877minus1
)119879
+ 119868]
minus1
(20)
Therefore the function 120601119898(120583) can be expressed as
120601119898(120583) =
10038171003817100381710038171003817100381710038171003817
[120583119867119877minus1
(119867119877minus1
)119879
+ 119868]
minus1
[119880 119881119898+1]119879
119887
10038171003817100381710038171003817100381710038171003817
2
+ (119868 minus 119875) 1198872
(21)
Concerning the properties of 120601119898(120583) we have the follow-
ing results which are similar to those of Theorem 21 in [11]for the Arnoldi-Tikhonov regularization method
Theorem 1 The function 120601119898(120583) has the representation
120601119898(120583) = 119887
119879
[119880 119881119898+1] [120583119867119877
minus1
(119867119877minus1
)119879
+ 119868]
minus2
times [119880119881119898+1]119879
119887 + (119868 minus 119875) 1198872
(22)
Assume that 119860119887 = 0 and (119867119877minus1)119879[119880 119881119898+1]119879
119887 = 0 Then 120601119898is
strictly decreasing and convex for 120583 ge 0 with 120601119898(0) = 119887
2Moreover the equation
120601119898(120583) = 120591 (23)
has a unique solution 120583120591119898
such that 0 lt 120583120591119898lt infin for any 120591
with100381710038171003817100381710038171003817119875N((119867119877minus1)119879)[119880 119881119898+1]
119879
119887100381710038171003817100381710038171003817
2
+ (119868 minus 119875) 1198872
lt 120591 lt 1198872
(24)
where 119875N((119867119877minus1)119879) denotes the orthogonal projector ontoN((119867119877minus1)
119879
)
Proof The proof follows the same argument of the proof ofTheorem 21 in [11] and therefore is omitted
We easily obtain the following theorem of which theproof is almost the same as that of Corollary 22 in [11]
Theorem 2 Assume that the modified Arnoldi process breaksdown at step 119895 Then the sequence 119904
119894119895
119894=0defined by
1199040= 1198872
119904119894=100381710038171003817100381710038171003817119875N((119867119877minus1)
119879
)[119880 119881119898+1]119879
119887100381710038171003817100381710038171003817
2
+ (119868 minus 119875)1198872
0 lt 119894 lt 119895
119904119895= 0
(25)
is decreasing
We apply the discrepancy principle to the discrepancy 119887minus119860119909120583119898
to determine an appropriate regularization parameter120583 so that it satisfies (5) To make the equation
120601119898(120583) = 120578
2
1205762 (26)
Mathematical Problems in Engineering 5
Input 119860 isin R119899times119899 119887 isin R119899119882 isin R119899times119896 120576 1205781198980
Output 119909120583119898
120583119898119898 = 119898min + 1198980
(1) Compute the modified Arnoldi decomposition 119860[119882119881119898] = [119880 119881
119898+1]119867 with119898 = 119898min + 1198980
where119898min is the smallest number such that (119868 minus 119875)1198872 lt 12057821205762(2) Compute the solution 120583
119898of the equation 120601
119898(120583) = 120578
2
1205762 by Newtonrsquos method
(3) Compute the solution 119910120583119898
of the least-squares problem (13) and obtain the approximatesolution 119909
120583119898= [119882119881
119898]119910120583119898
Algorithm 2 Augmented Arnoldi-Tikhonov regularization method
have a solution it follows from Theorem 1 that the inputparameter 119898 of the modified Arnoldi should be chosen sothat 119904
119898lt 1205782
1205762 To simplify the computations we ignore
the first term in 119904119898 Then the smallest iterative step number
denoted by 119898min of the modified Arnoldi process is chosenso that
(119868 minus 119875) 1198872
lt 1205782
1205762
(27)
We can improve the quality of the computed solution bychoosing the practical iterative step number 119898 somewhatlarger than119898min
After choosing the number 119898 of the modified Arnoldiiterative steps the regularization parameter 120583
119898is determined
by solving the nonlinear equation 120601119898(120583) = 120578
2
1205762 Many
numerical methods have been proposed for the solutionof a nonlinear equation including Newtonrsquos method [22]super-Newtonrsquos [23] method and Halleyrsquos method [24] Forthe specific nonlinear equation 120601
119898(120583) = 120578
2
1205762 Reichel and
Shyshkov proposed a new zero-finder method in their newpaper [25] In this paper we still make use of Newtonrsquosmethod to obtain the regularization parameter 120583
119898
In Algorithm 2 we outline the augmented Arnoldi-Tikhonov regularizationmethod which is used to solve large-scale linear ill-posed systems (1)
Newtonrsquos method requires to evaluate 120601119898(120583) and its first
derivative with respect to 120583 for computing approximations120583(119895)
119898of 120583119898for 119895 = 0 1 2 Let
119911120583119898= [120583119867119877
minus1
(119867119877minus1
)119879
+ 119868]
minus1
[119880 119881119898+1]119879
119887 (28)
that is 119911120583119898
satisfies the system of linear equations
[120583119867119877minus1
(119867119877minus1
)119879
+ 119868] 119911 = [119880119881119898+1]119879
119887 (29)
Note that the above linear system is the normal equations ofthe least-squares problem
min119911isinR119898+119896+1
1003817100381710038171003817100381710038171003817100381710038171003817
[12058312
(119867119877minus1
)119879
119868] 119911 minus [
0
(U 119881119898+1)119879
119887]
1003817100381710038171003817100381710038171003817100381710038171003817
(30)
For numerical stability we compute the vector 119911120583119898
by solvingthe least-squares problem Then the 120601
119898(120583) is evaluated by
computing
120601119898(120583) = 119911
119879
120583119898119911120583119898+ (119868 minus 119875) 119887
2
(31)
It is easy to show that the first derivative of 1206011015840119898(120583) can be
written as
1206011015840
119898(120583) = minus2119911
119879
120583119898119908120583119898 (32)
where
119908120583119898= [120583119867119877
minus1
(119867119877minus1
)119879
+ 119868]
minus1
119867119877minus1
(119867119877minus1
)119879
119911120583119898 (33)
Hence we may compute 119908120583119898
by solving a least-squaresproblem analogous to the above with the vector [119880 119881
119898+1]119879
119887
replaced by119867119877minus1(119867119877minus1)119879119911120583119898
The algorithm for implementing the Newton iteration for
solving the nonlinear equation 120601119898(120583) = 120578
2
1205762 is presented in
Algorithm 3Let 119909119898be the 119898th iterate produced by the augmented
RRGMRES in [16] Then 119909119898satisfies
1003817100381710038171003817119860119909119898 minus 1198871003817100381710038171003817
2
= min119909isinK119898(119860119860119887)+W
119860119909 minus 1198872
= min119910isinR119898+119896
1003817100381710038171003817119860 [119882119881119898] 119910 minus 1198871003817100381710038171003817
2
= min119910isinR119898+119896
1003817100381710038171003817[119880 119881119898+1]119867119910 minus 1198871003817100381710038171003817
2
= min119910isinR119898+119896
10038171003817100381710038171003817119867119910 minus [119880119881
119898+1]119879
11988710038171003817100381710038171003817
2
+ (119868 minus 119875)1198872
(34)
As 120583 rarr infin the reduced minimization problem (12) is thesame as the above minimization problem which shows that
119909119898= lim120583rarrinfin
119909120583119898 (35)
Therefore we obtain the following result
Theorem 3 Let 119909119898
be the 119898th iterate determined by aug-mented RRGMRES applied to (1) with initial iterate 119909
0= 0
Then1003817100381710038171003817119860119909119898 minus 119887
1003817100381710038171003817
2
= 120601119898(infin) (36)
3 Numerical Experiments
In this section we present some numerical examplesto illustrate the performance of the augmented Arnoldi-Tikhonov regularization method for the solution of large-scale linear ill-posed systems We compare the augmented
6 Mathematical Problems in Engineering
(1) Set 1205830= 0 and 119894 = 0
(2) Solve the least-squares problem
min119911isinR119898+119896+1
10038171003817100381710038171003817100381710038171003817
[12058312
119894(119867119877minus1
)119879
119868] 119911 minus [
0
(119880119881119898+1)119879
119887]
10038171003817100381710038171003817100381710038171003817
to obtain 119911120583119894119898
(3) Compute 120601119898(120583119894) = 119911119879
120583119894119898119911120583119894119898+ (119868 minus 119875)119887
2(4) Solve the least-squares problem
min119911isinR119898+119896+1
100381710038171003817100381710038171003817100381710038171003817
[12058312
119894(119867119877minus1
)119879
119868] 119911 minus [
0
119867119877minus1
(119867119877minus1
)119879
119911120583119894119898
]
100381710038171003817100381710038171003817100381710038171003817to obtain 119908
120583119894119898
(5) Compute 1206011015840119898(120583119894) = minus2119911
119879
120583119894119898119908120583119894119898
(6) Compute the new approximation
120583119894+1= 120583119894minus120601119898(120583119894) minus 1205782
1205762
1206011015840119898(120583119894)
(7) If |120583119894+1minus 120583119894| lt 10
minus6 stop else 120583119894= 120583119894+1 119894 = 119894 + 1 and go to 2
Algorithm 3 Newtonrsquos method for 120601119898(120583) = 120578
2
1205762
Arnoldi-Tikhonov regularization method implemented byAlgorithm 2 to the Arnoldi-Tikhonov regularization methodproposed in [11] The Arnoldi-Tikhonov regularizationmethod is denoted by ATRM while the augmented Arnoldi-Tikhonov regularization method is denoted by AATRMIn all the following tables we denote by MV the numberof matrix-vector products and by RERR the relative error119909120583119898
minus 119909119909 where 119909 is the exact solution of the linearerror-free system of (2) Note that the number of matrix-vector products in Algorithm 2 is 119896 + 119898min + 1198980 In all theexamples 120578 = 101
All the numerical experiments are performed in Matlabon a PC with the usual double precision where the floatingpoint relative accuracy is 222 sdot 10minus16
Example 4 The first example considered is the Fredholmintegral equation of the first kind which takes the genericform
int
1
0
119896 (119904 119905) 119909 (119905) 119889119905 = 119887 (119904) 0 le 119904 le 1 (37)
Here both the kernel 119896(119904 119905) and the right-hand side 119887(119904) areknown functions while 119909(119905) is the unknown function Fortest the kernel 119896(119904 119905) and the right-hand side 119887(119904) are chosenas
119896 (119904 119905) = 119904 (119905 minus 1) 119904 lt 119905
119905 (119904 minus 1) 119904 ge 119905119887 (119904) =
1199043
minus 119904
6 (38)
With this choice the exact solution of (37) is 119905 We use theMatlab program deriv2 from the regularization package [26]to discretize the integral equation (37) and to generate asystem of linear equations (2) with the coefficient matrix 119860 isinR200times200 and the solution 119909 isin R200 The condition number of119860 is 49 sdot 104 The right-hand side 119887 is given by 119887 = 119860119909 + 119890where the elements of the error vector 119890 are generated fromnormal distribution with mean zero and the norm of 119890 is10minus2
sdot 119860119909 The augmentation subspace is one-dimensional
Table 1 Computational results of Example 4
Method 1198980
MV RERRAATRM 0 2 195 sdot 10
minus2
AATRM 1 3 117 sdot 10minus2
AATRM 2 4 625 sdot 10minus2
AATRM 3 5 119 sdot 10minus1
ATRM 0 4 314 sdot 10minus1
ATRM 1 5 294 sdot 10minus1
ATRM 2 6 287 sdot 10minus1
ATRM 3 7 289 sdot 10minus1
and is spanned by the vector 119908 = [1 2 200]119879 Numericalresults for the example are reported in Table 1 for severalchoice of the number119898
0of additional iterations
FromTable 1 we can see that for 0 le 1198980le 3 AATRMhas
smaller relative errors than ATRM and the smallest relativeerror is given by AATRM with 119898
0= 1 The exact solution
119909 and the approximate solutions generated by AATRM andARTM with119898
0= 1 are depicted in Figure 1
Example 5 This example comes again from the regulariza-tion package [26] and is the inversion of Laplace transform
int
infin
0
119890minus119904119905
119909 (119904) 119889119904 = 119887 (119905) 119905 ge 1 (39)
where the right-hand side 119887(119905) and the exact solution 119909(119905) aregiven by
119887 (119905) =1
119905 + 12 119909 (119905) = 119890
minus1199052
(40)
The system of linear equations (2) with the coefficient matrix119860 isin R250times250 and the solution 119909 isin R250 is obtained byusing the Matlab program ilaplace from the regularizationpackage [26] In the same way as Example 4 we construct
Mathematical Problems in Engineering 7
0 50 100 150 2000
001
002
003
004
005
006
007
008
Exact solutionAATRMATRM
Figure 1 Example 4 Exact solution119909 approximation solutionswith1198980= 1
Table 2 Computational results of Example 5
Method 1198980
MV RERRAATRM 1 5 33 sdot 10
minus1
AATRM 2 6 91 sdot 10minus2
AATRM 3 7 27 sdot 10minus1
ATRM 1 6 57 sdot 10minus1
ATRM 2 7 31 sdot 10minus1
ATRM 3 8 45 sdot 10minus1
Table 3 Computational results of Example 6
Method 1198980
MV RERRAATRM 1 10 22 sdot 10
minus1
AATRM 2 11 11 sdot 10minus1
AATRM 3 12 21 sdot 10minus1
ATRM 1 10 49 sdot 10minus1
ATRM 2 11 12 sdot 10minus1
ATRM 3 12 23 sdot 10minus1
the right-hand side 119887 of the system of linear equations (1)The augmentation subspace is also one-dimensional and isspanned by the vector 119908 = [1 12 1250]
119879 Numericalresults for the example are reported in Table 2 for119898
0= 1 2 3
Table 2 shows that AATRM with 1198980= 2 has the smallest
relative error and AARTM works slightly better than ATRMfor this problem
Example 6 This example considered here is the same asExample 5 except that the right-hand side 119887(119905) and the exactsolution 119909(119905) are given by
119887 (119905) =2
(119905 + 12)3 119909 (119905) = 119905
2
119890minus1199052
(41)
By using the sameMatlab program as Example 5 we generatea system with 119860 isin R1000times1000 The augmentation subspace
is one-dimensional and is spanned by the vector 119908 =
[1 1 1]119879 In Table 3 we report numerical results for119898
0=
1 2 3
We observe from Table 3 that for this example AATRMhas almost the same relative errors as ATRM and the approx-imate solution can be slightly improved by the augmentationspace spanned by 119908 = [1 1 1]119879
4 Conclusions
In this paper we propose an iterative method for solvinglarge-scale linear ill-posed systems The method is based onthe Tikhonov regularization technique and the augmentedArnoldi technique The augmentation subspace is a user-supplied low-dimensional subspace which should containa rough approximation of the desired solution Numericalexperiments show that the augmented method is effective forsome practical problems
Acknowledgments
Yiqin Lin is supported by the National Natural ScienceFoundation of China under Grant 10801048 the NaturalScience Foundation ofHunanProvince underGrant 11JJ4009the Scientific Research Foundation of Education Bureau ofHunan Province for Outstanding Young Scholars in Univer-sity under Grant 10B038 the Science and Technology Plan-ning Project of Hunan Province under Grant 2010JT4042and the Chinese Postdoctoral Science Foundation underGrant 2012M511386 Liang Bao is supported by the NationalNatural Science Foundation of China under Grants 10926150and 11101149 and the Fundamental Research Funds for theCentral Universities
References
[1] H W Engl M Hanke and A Neubauer Regularization ofInverse Problems vol 375 of Mathematics and Its ApplicationsKluwer Academic Dordrecht The Netherlands 1996
[2] A N Tikhonov and V Y Arsenin Solutions of Ill-PosedProblems John Wiley amp Sons New York NY USA 1977
[3] A N Tikhonov A V Goncharsky V V Stepanov and AG Yagola Numerical Methods for the Solution of Ill-PosedProblems vol 328 of Mathematics and Its Applications KluwerAcademic Dordrecht The Netherlands 1995
[4] P C Hansen Rank-Deficient and Discrete Ill-Posed ProblemsSIAM Monographs on Mathematical Modeling and Computa-tion SIAM Philadelphia Pa USA 1998
[5] G H Golub and C F van Loan Matrix Computations JohnsHopkins Studies in the Mathematical Sciences Johns HopkinsUniversity Press Baltimore Md USA 3rd edition 1996
[6] A Bjorck Numerical Methods for Least Squares ProblemsSIAM Philadelphia Pa USA 1996
[7] A Bjorck ldquoA bidiagonalization algorithm for solving large andsparse ill-posed systems of linear equationsrdquo BIT NumericalMathematics vol 28 no 3 pp 659ndash670 1988
[8] D Calvetti S Morigi L Reichel and F Sgallari ldquoTikhonovregularization and the 119871-curve for large discrete ill-posed
8 Mathematical Problems in Engineering
problemsrdquo Journal of Computational and Applied Mathematicsvol 123 no 1-2 pp 423ndash446 2000
[9] D Calvetti and L Reichel ldquoTikhonov regularization of largelinear problemsrdquo BIT NumericalMathematics vol 43 no 2 pp263ndash283 2003
[10] D P OrsquoLeary and J A Simmons ldquoA bidiagonalization-regularization procedure for large scale discretizations of ill-posed problemsrdquo SIAM Journal on Scientific and StatisticalComputing vol 2 no 4 pp 474ndash489 1981
[11] B Lewis and L Reichel ldquoArnoldi-Tikhonov regularizationmethodsrdquo Journal of Computational and Applied Mathematicsvol 226 no 1 pp 92ndash102 2009
[12] T F Chan and K R Jackson ldquoNonlinearly preconditionedKrylov subspace methods for discrete Newton algorithmsrdquoSIAM Journal on Scientific and Statistical Computing vol 5 no3 pp 533ndash542 1984
[13] L Reichel F Sgallari andQ Ye ldquoTikhonov regularization basedon generalized Krylov subspace methodsrdquo Applied NumericalMathematics vol 62 no 9 pp 1215ndash1228 2012
[14] D Calvetti B Lewis and L Reichel ldquoOn the choice of subspacefor iterative methods for linear discrete ill-posed problemsrdquoInternational Journal of Applied Mathematics and ComputerScience vol 11 no 5 pp 1069ndash1092 2001 Numerical analysisand systems theory (Perpignan 2000)
[15] L Reichel and Q Ye ldquoBreakdown-free GMRES for singularsystemsrdquo SIAM Journal onMatrixAnalysis andApplications vol26 no 4 pp 1001ndash1021 2005
[16] J Baglama and L Reichel ldquoAugmented GMRES-type methodsrdquoNumerical Linear Algebra with Applications vol 14 no 4 pp337ndash350 2007
[17] K Morikuni L Reichel and K Hayami ldquoFGMRES for lineardiscrete problemsrdquo Tech Rep NII 2012
[18] D Calvetti B Lewis and L Reichel ldquoOn the regularizingproperties of the GMRES methodrdquo Numerische Mathematikvol 91 no 4 pp 605ndash625 2002
[19] R B Morgan ldquoA restarted GMRES method augmented witheigenvectorsrdquo SIAM Journal on Matrix Analysis and Applica-tions vol 16 no 4 pp 1154ndash1171 1995
[20] Y Saad ldquoA flexible inner-outer preconditioned GMRES algo-rithmrdquo SIAM Journal on Scientific Computing vol 14 no 2 pp461ndash469 1993
[21] B N Parlett The Symmetric Eigenvalue Problem vol 20 ofClassics in Applied Mathematics SIAM Philadelphia Pa USA1998
[22] C T Kelley Solving Nonlinear Equations with Newtonrsquos Methodvol 1 of Fundamentals of Algorithms SIAM Philadelphia PaUSA 2003
[23] J A Ezquerro andM A Hernandez ldquoOn a convex accelerationof Newtonrsquos methodrdquo Journal of Optimization Theory andApplications vol 100 no 2 pp 311ndash326 1999
[24] W Gander ldquoOn Halleyrsquos iteration methodrdquo The AmericanMathematical Monthly vol 92 no 2 pp 131ndash134 1985
[25] L Reichel and A Shyshkov ldquoA new zero-finder for Tikhonovregularizationrdquo BIT Numerical Mathematics vol 48 no 3 pp627ndash643 2008
[26] P C Hansen ldquoRegularization tools a Matlab package foranalysis and solution of discrete ill-posed problemsrdquoNumericalAlgorithms vol 6 no 1-2 pp 1ndash35 1994
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
2 Mathematical Problems in Engineering
quite large Thus the choice of a good value for 120583 is fairlyimportant Several methods have been proposed to obtain aneffective value for 120583 For example if the norm of the error119890 or a fairly accurate estimate is known the regularizationparameter is quite easy to determine by application of thediscrepancy principle The discrepancy principle proposesthat the regularization parameter 120583 can be chosen so that thediscrepancy 119887 minus 119860119909
120583satisfies
10038171003817100381710038171003817119887 minus 119860119909
120583
10038171003817100381710038171003817= 120578120576 (5)
where 120576 = 119890 and 120578 gt 1 is a constant see for example [4]for further details on the discrepancy principle
The singular value decomposition [5] of 119860 can be usedto determine the solution 119909
120583of the minimization problem
(3) For an overview of numerical methods for computingthe SVD we refer to [6] We remark that the computationaleffort required to compute the SVD is quite high even formoderately sized matrices
Many numerical methods using Krylov subspaces havebeen proposed for the solution of large-scale Tikhonovregularization problems (3)Themain idea of such algorithmshas been to first project the large problems onto some Krylovsubspace to produce problems with small size and then solvethe small-sized problems by the SVD For instance severalwell-established methods based on the Lanczos bidiagonal-ization process have been proposed for the solution of theminimization problem (3) see [7ndash10] and references thereinThese methods use the Lanczos bidiagonalization process toconstruct a basis of the Krylov subspace
K119898(119860119879
119860119860119879
119887)
= span 119860119879119887 119860119879119860119860119879119887 (119860119879119860)119898minus1
119860119879
119887
(6)
We remark that each Lanczos bidiagonalization step needstwo matrix-vector product evaluations one with 119860 and theother with 119860119879 Other methods using the Krylov subspace
K119898(119860 119860119887) = span 119860119887 1198602119887 119860119898119887 (7)
as the projection subspace have been also designed Forexample Lewis and Reichel [11] proposed to exploit theArnoldi process to produce a basis of the Krylov subspaceK119898(119860 119860119887) to obtain an approximation of the solution of
the Tikhonov regularization problem (3) Since each Arnoldidecomposition step requires only one matrix-vector evalua-tion with 119860 the approach based on the Arnoldi process mayrequire fewer matrix-vector product evaluations than thatbased on the Lanczos bidiagonalization process Moreoverthe methods based on the Arnoldi process do not requirethe adjoint matrix 119860119879 and hence are more appropriateto problems for which the adjoint is difficult to evaluateFor such problems we refer to [12] A similar Tikhonovregularizationmethod based on generalized Krylov subspaceis proposed in [13]
Some numerical methods without using the Tikhonovregularization technique have been already proposed to solvethe large-scale linear discrete ill-posed problem (1) Thesemethods include the range-restricted GMRES (RRGMRES)method [14 15] the augmented range-restricted GMRES(ARRGMRES) method [16] and the flexible GMRES (FGM-RES) method [17] The RRGMRES method determines the119898th approximation 119909
119898of (1) by solving the minimization
problem
min119909isinK119898(119860119860119887)
119860119909 minus 119887 119898 = 1 2 1199090= 0 (8)
The regularization is implemented by choosing a suitabledimension number 119898 see for example [18] The ARRGM-RES method augments the Krylov subspace K
119898(119860 119860119887)
by a low-dimensional user-supplied subspace The low-dimensional subspace is determined by vectors that are ableto represent the known features of the desired solutionThe augmented method can yield approximate solutions ofhigher accuracy than the RRGMRES method if the Krylovsubspace K
119898(119860 119860119887) does not allow representation of the
known featuresIn this paper we propose a new iterative method named
augmented Arnoldi-Tikhonov regularization method forsolving large-scale linear ill-posed systems (1) The newmethod is deduced by combining the Tikhonov regulariza-tion technique and the augmentation technique The aug-mentation is implemented by a modified Arnoldi process
The following summarizes the structure of this paperSection 2 describes the augmented Arnoldi-Tikhonov regu-larization method and some useful results Some real-worldexamples are presented in Section 3 Section 4 contains theconclusions
2 Augmented Arnoldi-TikhonovRegularization Method
We attempt to improve the Arnoldi-Tikhonov regularizationmethod proposed in [11] by augmenting the Krylov subspaceK119898(119860 119860119887) by a 119896-dimensional subspaceW which contains
a rough approximation of the desired solution of (2) Thenthe subspace of projection we will exploit in the following isof the form
K119898(119860 119860119887) +W = span 119860119887 1198602119887 119860119898119887 +W (9)
Let 119882 be the 119899 by 119896 matrix whose columns form anorthonormal basis of the space W For the purpose ofaugmentation by W we apply the modified Arnoldi process[16] to construct the modified Arnoldi decomposition
119860 [119882119881119898] = [119880 119881
119898+1]119867 (10)
where 119880 isin R119899times119896 119881119898+1
isin R119899times(119898+1) [119880 119881119898+1] has orthonor-
mal columns and119867 isin R(119898+119896+1)times(119898+119896) is an upperHessenbergmatrix We point out that the leading principal 119896 by 119896submatrix of 119867 is the upper triangular matrix in the QRfactorization [5] of 119860119882 that is119867 is of the form
Mathematical Problems in Engineering 3
Input 119860 isin R119899times119899 119887 isin R119899119882 isin R119899times119896119898Output 119880 isin R119899times119896 119881
119898+1isin R119899times(119898+1)119867 isin R(119898+119896+1)times(119898+119896)
(1) Compute QR-factorization 119860119882 = 119880119867(2) Compute V
1= 119860119887 minus 119880(119880
119879
119860119887) V1= V11003817100381710038171003817V11003817100381710038171003817 and set 119881
1= [V1]
(3) For 119895 = 1 2 119898 DoV119895+1= 119860V
119895
For 119894 = 1 2 119896 Doℎ119894119895+119896
= 119880( 119894)119879
V119895+1
V119895+1= V119895+1minus ℎ119894119895+119896119880( 119894)
End ForFor 119894 = 1 2 119895 Do
ℎ119894+119896119895+119896
= V119879119894V119895+1
V119895+1= V119895+1minus ℎ119894+119896119895+119896
V119894
End Forℎ119895+119896+1119895+119896
= V119895+110038171003817100381710038171003817V119895+1
10038171003817100381710038171003817
V119895+1= V119895+1ℎ119895+119896+1119895+119896
119881119895+1= [119881119895 V119895+1]
End For
Algorithm 1 Modified Arnoldi process
119867 =
[[[[[[[[[[[[[[[
[
ℎ11ℎ12
sdot sdot sdot ℎ1119896
ℎ1119896+1
ℎ1119896+2
sdot sdot sdot ℎ1119896+119898
d d
sdot sdot sdot
ℎ119896minus1119896minus1
ℎ119896minus1119896
ℎ119896minus1119896+1
ℎ119896minus1119896+2
sdot sdot sdot ℎ119896minus1119896+119898
ℎ119896119896
ℎ119896119896+1
ℎ119896119896+2
sdot sdot sdot ℎ119896119896+119898
ℎ119896+1119896+1
ℎ119896+1119896+2
sdot sdot sdot ℎ119896+1119896+119898
ℎ119896+2119896+1
ℎ119896+2119896+2
sdot sdot sdot ℎ119896+2119896+119898
d d
ℎ119896+119898119896+119898minus1
ℎ119896+119898119896+119898
ℎ119896+119898+1119896+119898
]]]]]]]]]]]]]]]
]
(11)
ThemodifiedArnoldi relation (10) is shared by othermethodssuch as GMRES-E [19] and FGMRES [20] which are alsoaugmented type methods
The modified Arnoldi process is outlined in Algorithm 1We remark that the modified Arnoldi process in this paperis slightly different from the one used by Morgan [19]to augment the Krylov subspace with some approximateeigenvectors In his method the augmenting vectors are putafter the Krylov vectors while in Algorithm 1 the augment-ing subspace containing a rough approximation solution isincluded in the projection subspace from the beginningIn general this can give better results at the start of theregularization method proposed in this paper
We remark that a loss of orthogonality can occur whenthe algorithm progresses see [21] A remedy is the so-called reorthogonalization where the current vector has tobe orthogonalized against previously created vectors Onecan choose between a selective reorthogonalization or a full
reorthogonalization against all vectors in the current aug-mented subspace In this paper we only use the full reorthog-onalization The full reorthogonalization can be done as aclassical or modifed Gram-Schmidt orthogonalization see[21] for details
We now seek to determine an approximate solution 119909120583119898
of (3) in the augmented Krylov subspace K119898(119860 119860119887) +W
After computing theQR factorization [119882119881119898] = 119876119877with119876 isin
R119899times(119898+119896) having orthonormal columns and 119877 isin R(119898+119896)times(119898+119896)
being an upper triangularmatrix we substitute119909 = [119882119881119898]119910
119910 isin R119898+119896 into (3) It yields the reduced minimizationproblem
min119910isinR119898+119896
1003817100381710038171003817119860 [119882119881119898] 119910 minus 119887
1003817100381710038171003817
2
+1
120583
1003817100381710038171003817[119882119881119898] 1199101003817100381710038171003817
2
= min119910isinR119898+119896
1003817100381710038171003817[119880 119881119898+1]119867119910 minus 119887
1003817100381710038171003817
2
+1
120583
10038171003817100381710038171198761198771199101003817100381710038171003817
2
4 Mathematical Problems in Engineering
= min119910isinR119898+119896
10038171003817100381710038171003817119867119910 minus [119880119881
119898+1]119879
11988710038171003817100381710038171003817
2
+(119868 minus 119875)1198872
+1
120583
10038171003817100381710038171198771199101003817100381710038171003817
2
(12)
in which 119875 = [119880119881119898+1][119880 119881
119898+1]119879 is an orthogonal projector
onto span([119880 119881119898+1]) Obviously the reduced minimization
problem (12) is equivalent to
min119910isinR119898+119896
10038171003817100381710038171003817119867119910 minus [119880119881
119898+1]119879
11988710038171003817100381710038171003817
2
+1
120583
10038171003817100381710038171198771199101003817100381710038171003817
2
= min119910isinR119898+119896
100381710038171003817100381710038171003817100381710038171003817100381710038171003817
[
[
119867
1
radic120583119877]
]
119910 minus [[119880119881119898+1]119879
119887
0]
100381710038171003817100381710038171003817100381710038171003817100381710038171003817
2
(13)
The normal equations of the minimization problem (13) is
(119867119879
119867 +1
120583119877119879
119877)119910 = 119867119879
[119880 119881119898+1]119879
119887 (14)
We denote the solution of the minimization problem (13) by119910120583119898
Then from (14) it follows that
119910120583119898= (119867
119879
119867 +1
120583119877119879
119877)
minus1
119867119879
[119880 119881119898+1]119879
119887 (15)
The approximate solution of (3) is
119909120583119898= [119882119881
119898] 119910120583119898 (16)
Since the matrix119867119879119867+ (1120583)119877119879119877 has a larger conditionnumber than the matrix [119867119879 (1radic120583)119877119879]119879 we apply the QRfactorization of [119867119879 (1radic120583)119877119879]119879 to obtain the solution 119910
120583119898
of (13) insteadTheQR factorization of [119867119879 (1radic120583)119877119879]119879 canbe implemented by a sequence of Givens rotations
Define
120601119898(120583) =
10038171003817100381710038171003817119887 minus 119860119909
120583119898
10038171003817100381710038171003817
2
(17)
Substituting 119909120583119898
= [119882119881119898]119910120583119898
into (17) and using themodified Arnoldi decomposition (10) yield
120601119898(120583) =
10038171003817100381710038171003817119887 minus 119860 [119882119881
119898] 119910120583119898
10038171003817100381710038171003817
2
=10038171003817100381710038171003817119887 minus [119880119881
119898+1]119867119910120583119898
10038171003817100381710038171003817
2
=10038171003817100381710038171003817[119880 119881119898+1]119879
119887 minus 119867119910120583119898
10038171003817100381710038171003817
2
+ (119868 minus 119875)1198872
(18)
Substituting 119910120583119898
in (15) into (18) we obtain
120601119898(120583) =
100381710038171003817100381710038171003817100381710038171003817
[119868 minus 119867(119867119879
119867 +1
120583119877119879
119877)
minus1
119867119879
] [119880119881119898+1]119879
119887
100381710038171003817100381710038171003817100381710038171003817
2
+ (119868 minus 119875) 1198872
(19)
Note that
119868 minus 119867(119867119879
119867 +1
120583119877119879
119877)
minus1
119867119879
= [120583119867119877minus1
(119867119877minus1
)119879
+ 119868]
minus1
(20)
Therefore the function 120601119898(120583) can be expressed as
120601119898(120583) =
10038171003817100381710038171003817100381710038171003817
[120583119867119877minus1
(119867119877minus1
)119879
+ 119868]
minus1
[119880 119881119898+1]119879
119887
10038171003817100381710038171003817100381710038171003817
2
+ (119868 minus 119875) 1198872
(21)
Concerning the properties of 120601119898(120583) we have the follow-
ing results which are similar to those of Theorem 21 in [11]for the Arnoldi-Tikhonov regularization method
Theorem 1 The function 120601119898(120583) has the representation
120601119898(120583) = 119887
119879
[119880 119881119898+1] [120583119867119877
minus1
(119867119877minus1
)119879
+ 119868]
minus2
times [119880119881119898+1]119879
119887 + (119868 minus 119875) 1198872
(22)
Assume that 119860119887 = 0 and (119867119877minus1)119879[119880 119881119898+1]119879
119887 = 0 Then 120601119898is
strictly decreasing and convex for 120583 ge 0 with 120601119898(0) = 119887
2Moreover the equation
120601119898(120583) = 120591 (23)
has a unique solution 120583120591119898
such that 0 lt 120583120591119898lt infin for any 120591
with100381710038171003817100381710038171003817119875N((119867119877minus1)119879)[119880 119881119898+1]
119879
119887100381710038171003817100381710038171003817
2
+ (119868 minus 119875) 1198872
lt 120591 lt 1198872
(24)
where 119875N((119867119877minus1)119879) denotes the orthogonal projector ontoN((119867119877minus1)
119879
)
Proof The proof follows the same argument of the proof ofTheorem 21 in [11] and therefore is omitted
We easily obtain the following theorem of which theproof is almost the same as that of Corollary 22 in [11]
Theorem 2 Assume that the modified Arnoldi process breaksdown at step 119895 Then the sequence 119904
119894119895
119894=0defined by
1199040= 1198872
119904119894=100381710038171003817100381710038171003817119875N((119867119877minus1)
119879
)[119880 119881119898+1]119879
119887100381710038171003817100381710038171003817
2
+ (119868 minus 119875)1198872
0 lt 119894 lt 119895
119904119895= 0
(25)
is decreasing
We apply the discrepancy principle to the discrepancy 119887minus119860119909120583119898
to determine an appropriate regularization parameter120583 so that it satisfies (5) To make the equation
120601119898(120583) = 120578
2
1205762 (26)
Mathematical Problems in Engineering 5
Input 119860 isin R119899times119899 119887 isin R119899119882 isin R119899times119896 120576 1205781198980
Output 119909120583119898
120583119898119898 = 119898min + 1198980
(1) Compute the modified Arnoldi decomposition 119860[119882119881119898] = [119880 119881
119898+1]119867 with119898 = 119898min + 1198980
where119898min is the smallest number such that (119868 minus 119875)1198872 lt 12057821205762(2) Compute the solution 120583
119898of the equation 120601
119898(120583) = 120578
2
1205762 by Newtonrsquos method
(3) Compute the solution 119910120583119898
of the least-squares problem (13) and obtain the approximatesolution 119909
120583119898= [119882119881
119898]119910120583119898
Algorithm 2 Augmented Arnoldi-Tikhonov regularization method
have a solution it follows from Theorem 1 that the inputparameter 119898 of the modified Arnoldi should be chosen sothat 119904
119898lt 1205782
1205762 To simplify the computations we ignore
the first term in 119904119898 Then the smallest iterative step number
denoted by 119898min of the modified Arnoldi process is chosenso that
(119868 minus 119875) 1198872
lt 1205782
1205762
(27)
We can improve the quality of the computed solution bychoosing the practical iterative step number 119898 somewhatlarger than119898min
After choosing the number 119898 of the modified Arnoldiiterative steps the regularization parameter 120583
119898is determined
by solving the nonlinear equation 120601119898(120583) = 120578
2
1205762 Many
numerical methods have been proposed for the solutionof a nonlinear equation including Newtonrsquos method [22]super-Newtonrsquos [23] method and Halleyrsquos method [24] Forthe specific nonlinear equation 120601
119898(120583) = 120578
2
1205762 Reichel and
Shyshkov proposed a new zero-finder method in their newpaper [25] In this paper we still make use of Newtonrsquosmethod to obtain the regularization parameter 120583
119898
In Algorithm 2 we outline the augmented Arnoldi-Tikhonov regularizationmethod which is used to solve large-scale linear ill-posed systems (1)
Newtonrsquos method requires to evaluate 120601119898(120583) and its first
derivative with respect to 120583 for computing approximations120583(119895)
119898of 120583119898for 119895 = 0 1 2 Let
119911120583119898= [120583119867119877
minus1
(119867119877minus1
)119879
+ 119868]
minus1
[119880 119881119898+1]119879
119887 (28)
that is 119911120583119898
satisfies the system of linear equations
[120583119867119877minus1
(119867119877minus1
)119879
+ 119868] 119911 = [119880119881119898+1]119879
119887 (29)
Note that the above linear system is the normal equations ofthe least-squares problem
min119911isinR119898+119896+1
1003817100381710038171003817100381710038171003817100381710038171003817
[12058312
(119867119877minus1
)119879
119868] 119911 minus [
0
(U 119881119898+1)119879
119887]
1003817100381710038171003817100381710038171003817100381710038171003817
(30)
For numerical stability we compute the vector 119911120583119898
by solvingthe least-squares problem Then the 120601
119898(120583) is evaluated by
computing
120601119898(120583) = 119911
119879
120583119898119911120583119898+ (119868 minus 119875) 119887
2
(31)
It is easy to show that the first derivative of 1206011015840119898(120583) can be
written as
1206011015840
119898(120583) = minus2119911
119879
120583119898119908120583119898 (32)
where
119908120583119898= [120583119867119877
minus1
(119867119877minus1
)119879
+ 119868]
minus1
119867119877minus1
(119867119877minus1
)119879
119911120583119898 (33)
Hence we may compute 119908120583119898
by solving a least-squaresproblem analogous to the above with the vector [119880 119881
119898+1]119879
119887
replaced by119867119877minus1(119867119877minus1)119879119911120583119898
The algorithm for implementing the Newton iteration for
solving the nonlinear equation 120601119898(120583) = 120578
2
1205762 is presented in
Algorithm 3Let 119909119898be the 119898th iterate produced by the augmented
RRGMRES in [16] Then 119909119898satisfies
1003817100381710038171003817119860119909119898 minus 1198871003817100381710038171003817
2
= min119909isinK119898(119860119860119887)+W
119860119909 minus 1198872
= min119910isinR119898+119896
1003817100381710038171003817119860 [119882119881119898] 119910 minus 1198871003817100381710038171003817
2
= min119910isinR119898+119896
1003817100381710038171003817[119880 119881119898+1]119867119910 minus 1198871003817100381710038171003817
2
= min119910isinR119898+119896
10038171003817100381710038171003817119867119910 minus [119880119881
119898+1]119879
11988710038171003817100381710038171003817
2
+ (119868 minus 119875)1198872
(34)
As 120583 rarr infin the reduced minimization problem (12) is thesame as the above minimization problem which shows that
119909119898= lim120583rarrinfin
119909120583119898 (35)
Therefore we obtain the following result
Theorem 3 Let 119909119898
be the 119898th iterate determined by aug-mented RRGMRES applied to (1) with initial iterate 119909
0= 0
Then1003817100381710038171003817119860119909119898 minus 119887
1003817100381710038171003817
2
= 120601119898(infin) (36)
3 Numerical Experiments
In this section we present some numerical examplesto illustrate the performance of the augmented Arnoldi-Tikhonov regularization method for the solution of large-scale linear ill-posed systems We compare the augmented
6 Mathematical Problems in Engineering
(1) Set 1205830= 0 and 119894 = 0
(2) Solve the least-squares problem
min119911isinR119898+119896+1
10038171003817100381710038171003817100381710038171003817
[12058312
119894(119867119877minus1
)119879
119868] 119911 minus [
0
(119880119881119898+1)119879
119887]
10038171003817100381710038171003817100381710038171003817
to obtain 119911120583119894119898
(3) Compute 120601119898(120583119894) = 119911119879
120583119894119898119911120583119894119898+ (119868 minus 119875)119887
2(4) Solve the least-squares problem
min119911isinR119898+119896+1
100381710038171003817100381710038171003817100381710038171003817
[12058312
119894(119867119877minus1
)119879
119868] 119911 minus [
0
119867119877minus1
(119867119877minus1
)119879
119911120583119894119898
]
100381710038171003817100381710038171003817100381710038171003817to obtain 119908
120583119894119898
(5) Compute 1206011015840119898(120583119894) = minus2119911
119879
120583119894119898119908120583119894119898
(6) Compute the new approximation
120583119894+1= 120583119894minus120601119898(120583119894) minus 1205782
1205762
1206011015840119898(120583119894)
(7) If |120583119894+1minus 120583119894| lt 10
minus6 stop else 120583119894= 120583119894+1 119894 = 119894 + 1 and go to 2
Algorithm 3 Newtonrsquos method for 120601119898(120583) = 120578
2
1205762
Arnoldi-Tikhonov regularization method implemented byAlgorithm 2 to the Arnoldi-Tikhonov regularization methodproposed in [11] The Arnoldi-Tikhonov regularizationmethod is denoted by ATRM while the augmented Arnoldi-Tikhonov regularization method is denoted by AATRMIn all the following tables we denote by MV the numberof matrix-vector products and by RERR the relative error119909120583119898
minus 119909119909 where 119909 is the exact solution of the linearerror-free system of (2) Note that the number of matrix-vector products in Algorithm 2 is 119896 + 119898min + 1198980 In all theexamples 120578 = 101
All the numerical experiments are performed in Matlabon a PC with the usual double precision where the floatingpoint relative accuracy is 222 sdot 10minus16
Example 4 The first example considered is the Fredholmintegral equation of the first kind which takes the genericform
int
1
0
119896 (119904 119905) 119909 (119905) 119889119905 = 119887 (119904) 0 le 119904 le 1 (37)
Here both the kernel 119896(119904 119905) and the right-hand side 119887(119904) areknown functions while 119909(119905) is the unknown function Fortest the kernel 119896(119904 119905) and the right-hand side 119887(119904) are chosenas
119896 (119904 119905) = 119904 (119905 minus 1) 119904 lt 119905
119905 (119904 minus 1) 119904 ge 119905119887 (119904) =
1199043
minus 119904
6 (38)
With this choice the exact solution of (37) is 119905 We use theMatlab program deriv2 from the regularization package [26]to discretize the integral equation (37) and to generate asystem of linear equations (2) with the coefficient matrix 119860 isinR200times200 and the solution 119909 isin R200 The condition number of119860 is 49 sdot 104 The right-hand side 119887 is given by 119887 = 119860119909 + 119890where the elements of the error vector 119890 are generated fromnormal distribution with mean zero and the norm of 119890 is10minus2
sdot 119860119909 The augmentation subspace is one-dimensional
Table 1 Computational results of Example 4
Method 1198980
MV RERRAATRM 0 2 195 sdot 10
minus2
AATRM 1 3 117 sdot 10minus2
AATRM 2 4 625 sdot 10minus2
AATRM 3 5 119 sdot 10minus1
ATRM 0 4 314 sdot 10minus1
ATRM 1 5 294 sdot 10minus1
ATRM 2 6 287 sdot 10minus1
ATRM 3 7 289 sdot 10minus1
and is spanned by the vector 119908 = [1 2 200]119879 Numericalresults for the example are reported in Table 1 for severalchoice of the number119898
0of additional iterations
FromTable 1 we can see that for 0 le 1198980le 3 AATRMhas
smaller relative errors than ATRM and the smallest relativeerror is given by AATRM with 119898
0= 1 The exact solution
119909 and the approximate solutions generated by AATRM andARTM with119898
0= 1 are depicted in Figure 1
Example 5 This example comes again from the regulariza-tion package [26] and is the inversion of Laplace transform
int
infin
0
119890minus119904119905
119909 (119904) 119889119904 = 119887 (119905) 119905 ge 1 (39)
where the right-hand side 119887(119905) and the exact solution 119909(119905) aregiven by
119887 (119905) =1
119905 + 12 119909 (119905) = 119890
minus1199052
(40)
The system of linear equations (2) with the coefficient matrix119860 isin R250times250 and the solution 119909 isin R250 is obtained byusing the Matlab program ilaplace from the regularizationpackage [26] In the same way as Example 4 we construct
Mathematical Problems in Engineering 7
0 50 100 150 2000
001
002
003
004
005
006
007
008
Exact solutionAATRMATRM
Figure 1 Example 4 Exact solution119909 approximation solutionswith1198980= 1
Table 2 Computational results of Example 5
Method 1198980
MV RERRAATRM 1 5 33 sdot 10
minus1
AATRM 2 6 91 sdot 10minus2
AATRM 3 7 27 sdot 10minus1
ATRM 1 6 57 sdot 10minus1
ATRM 2 7 31 sdot 10minus1
ATRM 3 8 45 sdot 10minus1
Table 3 Computational results of Example 6
Method 1198980
MV RERRAATRM 1 10 22 sdot 10
minus1
AATRM 2 11 11 sdot 10minus1
AATRM 3 12 21 sdot 10minus1
ATRM 1 10 49 sdot 10minus1
ATRM 2 11 12 sdot 10minus1
ATRM 3 12 23 sdot 10minus1
the right-hand side 119887 of the system of linear equations (1)The augmentation subspace is also one-dimensional and isspanned by the vector 119908 = [1 12 1250]
119879 Numericalresults for the example are reported in Table 2 for119898
0= 1 2 3
Table 2 shows that AATRM with 1198980= 2 has the smallest
relative error and AARTM works slightly better than ATRMfor this problem
Example 6 This example considered here is the same asExample 5 except that the right-hand side 119887(119905) and the exactsolution 119909(119905) are given by
119887 (119905) =2
(119905 + 12)3 119909 (119905) = 119905
2
119890minus1199052
(41)
By using the sameMatlab program as Example 5 we generatea system with 119860 isin R1000times1000 The augmentation subspace
is one-dimensional and is spanned by the vector 119908 =
[1 1 1]119879 In Table 3 we report numerical results for119898
0=
1 2 3
We observe from Table 3 that for this example AATRMhas almost the same relative errors as ATRM and the approx-imate solution can be slightly improved by the augmentationspace spanned by 119908 = [1 1 1]119879
4 Conclusions
In this paper we propose an iterative method for solvinglarge-scale linear ill-posed systems The method is based onthe Tikhonov regularization technique and the augmentedArnoldi technique The augmentation subspace is a user-supplied low-dimensional subspace which should containa rough approximation of the desired solution Numericalexperiments show that the augmented method is effective forsome practical problems
Acknowledgments
Yiqin Lin is supported by the National Natural ScienceFoundation of China under Grant 10801048 the NaturalScience Foundation ofHunanProvince underGrant 11JJ4009the Scientific Research Foundation of Education Bureau ofHunan Province for Outstanding Young Scholars in Univer-sity under Grant 10B038 the Science and Technology Plan-ning Project of Hunan Province under Grant 2010JT4042and the Chinese Postdoctoral Science Foundation underGrant 2012M511386 Liang Bao is supported by the NationalNatural Science Foundation of China under Grants 10926150and 11101149 and the Fundamental Research Funds for theCentral Universities
References
[1] H W Engl M Hanke and A Neubauer Regularization ofInverse Problems vol 375 of Mathematics and Its ApplicationsKluwer Academic Dordrecht The Netherlands 1996
[2] A N Tikhonov and V Y Arsenin Solutions of Ill-PosedProblems John Wiley amp Sons New York NY USA 1977
[3] A N Tikhonov A V Goncharsky V V Stepanov and AG Yagola Numerical Methods for the Solution of Ill-PosedProblems vol 328 of Mathematics and Its Applications KluwerAcademic Dordrecht The Netherlands 1995
[4] P C Hansen Rank-Deficient and Discrete Ill-Posed ProblemsSIAM Monographs on Mathematical Modeling and Computa-tion SIAM Philadelphia Pa USA 1998
[5] G H Golub and C F van Loan Matrix Computations JohnsHopkins Studies in the Mathematical Sciences Johns HopkinsUniversity Press Baltimore Md USA 3rd edition 1996
[6] A Bjorck Numerical Methods for Least Squares ProblemsSIAM Philadelphia Pa USA 1996
[7] A Bjorck ldquoA bidiagonalization algorithm for solving large andsparse ill-posed systems of linear equationsrdquo BIT NumericalMathematics vol 28 no 3 pp 659ndash670 1988
[8] D Calvetti S Morigi L Reichel and F Sgallari ldquoTikhonovregularization and the 119871-curve for large discrete ill-posed
8 Mathematical Problems in Engineering
problemsrdquo Journal of Computational and Applied Mathematicsvol 123 no 1-2 pp 423ndash446 2000
[9] D Calvetti and L Reichel ldquoTikhonov regularization of largelinear problemsrdquo BIT NumericalMathematics vol 43 no 2 pp263ndash283 2003
[10] D P OrsquoLeary and J A Simmons ldquoA bidiagonalization-regularization procedure for large scale discretizations of ill-posed problemsrdquo SIAM Journal on Scientific and StatisticalComputing vol 2 no 4 pp 474ndash489 1981
[11] B Lewis and L Reichel ldquoArnoldi-Tikhonov regularizationmethodsrdquo Journal of Computational and Applied Mathematicsvol 226 no 1 pp 92ndash102 2009
[12] T F Chan and K R Jackson ldquoNonlinearly preconditionedKrylov subspace methods for discrete Newton algorithmsrdquoSIAM Journal on Scientific and Statistical Computing vol 5 no3 pp 533ndash542 1984
[13] L Reichel F Sgallari andQ Ye ldquoTikhonov regularization basedon generalized Krylov subspace methodsrdquo Applied NumericalMathematics vol 62 no 9 pp 1215ndash1228 2012
[14] D Calvetti B Lewis and L Reichel ldquoOn the choice of subspacefor iterative methods for linear discrete ill-posed problemsrdquoInternational Journal of Applied Mathematics and ComputerScience vol 11 no 5 pp 1069ndash1092 2001 Numerical analysisand systems theory (Perpignan 2000)
[15] L Reichel and Q Ye ldquoBreakdown-free GMRES for singularsystemsrdquo SIAM Journal onMatrixAnalysis andApplications vol26 no 4 pp 1001ndash1021 2005
[16] J Baglama and L Reichel ldquoAugmented GMRES-type methodsrdquoNumerical Linear Algebra with Applications vol 14 no 4 pp337ndash350 2007
[17] K Morikuni L Reichel and K Hayami ldquoFGMRES for lineardiscrete problemsrdquo Tech Rep NII 2012
[18] D Calvetti B Lewis and L Reichel ldquoOn the regularizingproperties of the GMRES methodrdquo Numerische Mathematikvol 91 no 4 pp 605ndash625 2002
[19] R B Morgan ldquoA restarted GMRES method augmented witheigenvectorsrdquo SIAM Journal on Matrix Analysis and Applica-tions vol 16 no 4 pp 1154ndash1171 1995
[20] Y Saad ldquoA flexible inner-outer preconditioned GMRES algo-rithmrdquo SIAM Journal on Scientific Computing vol 14 no 2 pp461ndash469 1993
[21] B N Parlett The Symmetric Eigenvalue Problem vol 20 ofClassics in Applied Mathematics SIAM Philadelphia Pa USA1998
[22] C T Kelley Solving Nonlinear Equations with Newtonrsquos Methodvol 1 of Fundamentals of Algorithms SIAM Philadelphia PaUSA 2003
[23] J A Ezquerro andM A Hernandez ldquoOn a convex accelerationof Newtonrsquos methodrdquo Journal of Optimization Theory andApplications vol 100 no 2 pp 311ndash326 1999
[24] W Gander ldquoOn Halleyrsquos iteration methodrdquo The AmericanMathematical Monthly vol 92 no 2 pp 131ndash134 1985
[25] L Reichel and A Shyshkov ldquoA new zero-finder for Tikhonovregularizationrdquo BIT Numerical Mathematics vol 48 no 3 pp627ndash643 2008
[26] P C Hansen ldquoRegularization tools a Matlab package foranalysis and solution of discrete ill-posed problemsrdquoNumericalAlgorithms vol 6 no 1-2 pp 1ndash35 1994
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Mathematical Problems in Engineering 3
Input 119860 isin R119899times119899 119887 isin R119899119882 isin R119899times119896119898Output 119880 isin R119899times119896 119881
119898+1isin R119899times(119898+1)119867 isin R(119898+119896+1)times(119898+119896)
(1) Compute QR-factorization 119860119882 = 119880119867(2) Compute V
1= 119860119887 minus 119880(119880
119879
119860119887) V1= V11003817100381710038171003817V11003817100381710038171003817 and set 119881
1= [V1]
(3) For 119895 = 1 2 119898 DoV119895+1= 119860V
119895
For 119894 = 1 2 119896 Doℎ119894119895+119896
= 119880( 119894)119879
V119895+1
V119895+1= V119895+1minus ℎ119894119895+119896119880( 119894)
End ForFor 119894 = 1 2 119895 Do
ℎ119894+119896119895+119896
= V119879119894V119895+1
V119895+1= V119895+1minus ℎ119894+119896119895+119896
V119894
End Forℎ119895+119896+1119895+119896
= V119895+110038171003817100381710038171003817V119895+1
10038171003817100381710038171003817
V119895+1= V119895+1ℎ119895+119896+1119895+119896
119881119895+1= [119881119895 V119895+1]
End For
Algorithm 1 Modified Arnoldi process
119867 =
[[[[[[[[[[[[[[[
[
ℎ11ℎ12
sdot sdot sdot ℎ1119896
ℎ1119896+1
ℎ1119896+2
sdot sdot sdot ℎ1119896+119898
d d
sdot sdot sdot
ℎ119896minus1119896minus1
ℎ119896minus1119896
ℎ119896minus1119896+1
ℎ119896minus1119896+2
sdot sdot sdot ℎ119896minus1119896+119898
ℎ119896119896
ℎ119896119896+1
ℎ119896119896+2
sdot sdot sdot ℎ119896119896+119898
ℎ119896+1119896+1
ℎ119896+1119896+2
sdot sdot sdot ℎ119896+1119896+119898
ℎ119896+2119896+1
ℎ119896+2119896+2
sdot sdot sdot ℎ119896+2119896+119898
d d
ℎ119896+119898119896+119898minus1
ℎ119896+119898119896+119898
ℎ119896+119898+1119896+119898
]]]]]]]]]]]]]]]
]
(11)
ThemodifiedArnoldi relation (10) is shared by othermethodssuch as GMRES-E [19] and FGMRES [20] which are alsoaugmented type methods
The modified Arnoldi process is outlined in Algorithm 1We remark that the modified Arnoldi process in this paperis slightly different from the one used by Morgan [19]to augment the Krylov subspace with some approximateeigenvectors In his method the augmenting vectors are putafter the Krylov vectors while in Algorithm 1 the augment-ing subspace containing a rough approximation solution isincluded in the projection subspace from the beginningIn general this can give better results at the start of theregularization method proposed in this paper
We remark that a loss of orthogonality can occur whenthe algorithm progresses see [21] A remedy is the so-called reorthogonalization where the current vector has tobe orthogonalized against previously created vectors Onecan choose between a selective reorthogonalization or a full
reorthogonalization against all vectors in the current aug-mented subspace In this paper we only use the full reorthog-onalization The full reorthogonalization can be done as aclassical or modifed Gram-Schmidt orthogonalization see[21] for details
We now seek to determine an approximate solution 119909120583119898
of (3) in the augmented Krylov subspace K119898(119860 119860119887) +W
After computing theQR factorization [119882119881119898] = 119876119877with119876 isin
R119899times(119898+119896) having orthonormal columns and 119877 isin R(119898+119896)times(119898+119896)
being an upper triangularmatrix we substitute119909 = [119882119881119898]119910
119910 isin R119898+119896 into (3) It yields the reduced minimizationproblem
min119910isinR119898+119896
1003817100381710038171003817119860 [119882119881119898] 119910 minus 119887
1003817100381710038171003817
2
+1
120583
1003817100381710038171003817[119882119881119898] 1199101003817100381710038171003817
2
= min119910isinR119898+119896
1003817100381710038171003817[119880 119881119898+1]119867119910 minus 119887
1003817100381710038171003817
2
+1
120583
10038171003817100381710038171198761198771199101003817100381710038171003817
2
4 Mathematical Problems in Engineering
= min119910isinR119898+119896
10038171003817100381710038171003817119867119910 minus [119880119881
119898+1]119879
11988710038171003817100381710038171003817
2
+(119868 minus 119875)1198872
+1
120583
10038171003817100381710038171198771199101003817100381710038171003817
2
(12)
in which 119875 = [119880119881119898+1][119880 119881
119898+1]119879 is an orthogonal projector
onto span([119880 119881119898+1]) Obviously the reduced minimization
problem (12) is equivalent to
min119910isinR119898+119896
10038171003817100381710038171003817119867119910 minus [119880119881
119898+1]119879
11988710038171003817100381710038171003817
2
+1
120583
10038171003817100381710038171198771199101003817100381710038171003817
2
= min119910isinR119898+119896
100381710038171003817100381710038171003817100381710038171003817100381710038171003817
[
[
119867
1
radic120583119877]
]
119910 minus [[119880119881119898+1]119879
119887
0]
100381710038171003817100381710038171003817100381710038171003817100381710038171003817
2
(13)
The normal equations of the minimization problem (13) is
(119867119879
119867 +1
120583119877119879
119877)119910 = 119867119879
[119880 119881119898+1]119879
119887 (14)
We denote the solution of the minimization problem (13) by119910120583119898
Then from (14) it follows that
119910120583119898= (119867
119879
119867 +1
120583119877119879
119877)
minus1
119867119879
[119880 119881119898+1]119879
119887 (15)
The approximate solution of (3) is
119909120583119898= [119882119881
119898] 119910120583119898 (16)
Since the matrix119867119879119867+ (1120583)119877119879119877 has a larger conditionnumber than the matrix [119867119879 (1radic120583)119877119879]119879 we apply the QRfactorization of [119867119879 (1radic120583)119877119879]119879 to obtain the solution 119910
120583119898
of (13) insteadTheQR factorization of [119867119879 (1radic120583)119877119879]119879 canbe implemented by a sequence of Givens rotations
Define
120601119898(120583) =
10038171003817100381710038171003817119887 minus 119860119909
120583119898
10038171003817100381710038171003817
2
(17)
Substituting 119909120583119898
= [119882119881119898]119910120583119898
into (17) and using themodified Arnoldi decomposition (10) yield
120601119898(120583) =
10038171003817100381710038171003817119887 minus 119860 [119882119881
119898] 119910120583119898
10038171003817100381710038171003817
2
=10038171003817100381710038171003817119887 minus [119880119881
119898+1]119867119910120583119898
10038171003817100381710038171003817
2
=10038171003817100381710038171003817[119880 119881119898+1]119879
119887 minus 119867119910120583119898
10038171003817100381710038171003817
2
+ (119868 minus 119875)1198872
(18)
Substituting 119910120583119898
in (15) into (18) we obtain
120601119898(120583) =
100381710038171003817100381710038171003817100381710038171003817
[119868 minus 119867(119867119879
119867 +1
120583119877119879
119877)
minus1
119867119879
] [119880119881119898+1]119879
119887
100381710038171003817100381710038171003817100381710038171003817
2
+ (119868 minus 119875) 1198872
(19)
Note that
119868 minus 119867(119867119879
119867 +1
120583119877119879
119877)
minus1
119867119879
= [120583119867119877minus1
(119867119877minus1
)119879
+ 119868]
minus1
(20)
Therefore the function 120601119898(120583) can be expressed as
120601119898(120583) =
10038171003817100381710038171003817100381710038171003817
[120583119867119877minus1
(119867119877minus1
)119879
+ 119868]
minus1
[119880 119881119898+1]119879
119887
10038171003817100381710038171003817100381710038171003817
2
+ (119868 minus 119875) 1198872
(21)
Concerning the properties of 120601119898(120583) we have the follow-
ing results which are similar to those of Theorem 21 in [11]for the Arnoldi-Tikhonov regularization method
Theorem 1 The function 120601119898(120583) has the representation
120601119898(120583) = 119887
119879
[119880 119881119898+1] [120583119867119877
minus1
(119867119877minus1
)119879
+ 119868]
minus2
times [119880119881119898+1]119879
119887 + (119868 minus 119875) 1198872
(22)
Assume that 119860119887 = 0 and (119867119877minus1)119879[119880 119881119898+1]119879
119887 = 0 Then 120601119898is
strictly decreasing and convex for 120583 ge 0 with 120601119898(0) = 119887
2Moreover the equation
120601119898(120583) = 120591 (23)
has a unique solution 120583120591119898
such that 0 lt 120583120591119898lt infin for any 120591
with100381710038171003817100381710038171003817119875N((119867119877minus1)119879)[119880 119881119898+1]
119879
119887100381710038171003817100381710038171003817
2
+ (119868 minus 119875) 1198872
lt 120591 lt 1198872
(24)
where 119875N((119867119877minus1)119879) denotes the orthogonal projector ontoN((119867119877minus1)
119879
)
Proof The proof follows the same argument of the proof ofTheorem 21 in [11] and therefore is omitted
We easily obtain the following theorem of which theproof is almost the same as that of Corollary 22 in [11]
Theorem 2 Assume that the modified Arnoldi process breaksdown at step 119895 Then the sequence 119904
119894119895
119894=0defined by
1199040= 1198872
119904119894=100381710038171003817100381710038171003817119875N((119867119877minus1)
119879
)[119880 119881119898+1]119879
119887100381710038171003817100381710038171003817
2
+ (119868 minus 119875)1198872
0 lt 119894 lt 119895
119904119895= 0
(25)
is decreasing
We apply the discrepancy principle to the discrepancy 119887minus119860119909120583119898
to determine an appropriate regularization parameter120583 so that it satisfies (5) To make the equation
120601119898(120583) = 120578
2
1205762 (26)
Mathematical Problems in Engineering 5
Input 119860 isin R119899times119899 119887 isin R119899119882 isin R119899times119896 120576 1205781198980
Output 119909120583119898
120583119898119898 = 119898min + 1198980
(1) Compute the modified Arnoldi decomposition 119860[119882119881119898] = [119880 119881
119898+1]119867 with119898 = 119898min + 1198980
where119898min is the smallest number such that (119868 minus 119875)1198872 lt 12057821205762(2) Compute the solution 120583
119898of the equation 120601
119898(120583) = 120578
2
1205762 by Newtonrsquos method
(3) Compute the solution 119910120583119898
of the least-squares problem (13) and obtain the approximatesolution 119909
120583119898= [119882119881
119898]119910120583119898
Algorithm 2 Augmented Arnoldi-Tikhonov regularization method
have a solution it follows from Theorem 1 that the inputparameter 119898 of the modified Arnoldi should be chosen sothat 119904
119898lt 1205782
1205762 To simplify the computations we ignore
the first term in 119904119898 Then the smallest iterative step number
denoted by 119898min of the modified Arnoldi process is chosenso that
(119868 minus 119875) 1198872
lt 1205782
1205762
(27)
We can improve the quality of the computed solution bychoosing the practical iterative step number 119898 somewhatlarger than119898min
After choosing the number 119898 of the modified Arnoldiiterative steps the regularization parameter 120583
119898is determined
by solving the nonlinear equation 120601119898(120583) = 120578
2
1205762 Many
numerical methods have been proposed for the solutionof a nonlinear equation including Newtonrsquos method [22]super-Newtonrsquos [23] method and Halleyrsquos method [24] Forthe specific nonlinear equation 120601
119898(120583) = 120578
2
1205762 Reichel and
Shyshkov proposed a new zero-finder method in their newpaper [25] In this paper we still make use of Newtonrsquosmethod to obtain the regularization parameter 120583
119898
In Algorithm 2 we outline the augmented Arnoldi-Tikhonov regularizationmethod which is used to solve large-scale linear ill-posed systems (1)
Newtonrsquos method requires to evaluate 120601119898(120583) and its first
derivative with respect to 120583 for computing approximations120583(119895)
119898of 120583119898for 119895 = 0 1 2 Let
119911120583119898= [120583119867119877
minus1
(119867119877minus1
)119879
+ 119868]
minus1
[119880 119881119898+1]119879
119887 (28)
that is 119911120583119898
satisfies the system of linear equations
[120583119867119877minus1
(119867119877minus1
)119879
+ 119868] 119911 = [119880119881119898+1]119879
119887 (29)
Note that the above linear system is the normal equations ofthe least-squares problem
min119911isinR119898+119896+1
1003817100381710038171003817100381710038171003817100381710038171003817
[12058312
(119867119877minus1
)119879
119868] 119911 minus [
0
(U 119881119898+1)119879
119887]
1003817100381710038171003817100381710038171003817100381710038171003817
(30)
For numerical stability we compute the vector 119911120583119898
by solvingthe least-squares problem Then the 120601
119898(120583) is evaluated by
computing
120601119898(120583) = 119911
119879
120583119898119911120583119898+ (119868 minus 119875) 119887
2
(31)
It is easy to show that the first derivative of 1206011015840119898(120583) can be
written as
1206011015840
119898(120583) = minus2119911
119879
120583119898119908120583119898 (32)
where
119908120583119898= [120583119867119877
minus1
(119867119877minus1
)119879
+ 119868]
minus1
119867119877minus1
(119867119877minus1
)119879
119911120583119898 (33)
Hence we may compute 119908120583119898
by solving a least-squaresproblem analogous to the above with the vector [119880 119881
119898+1]119879
119887
replaced by119867119877minus1(119867119877minus1)119879119911120583119898
The algorithm for implementing the Newton iteration for
solving the nonlinear equation 120601119898(120583) = 120578
2
1205762 is presented in
Algorithm 3Let 119909119898be the 119898th iterate produced by the augmented
RRGMRES in [16] Then 119909119898satisfies
1003817100381710038171003817119860119909119898 minus 1198871003817100381710038171003817
2
= min119909isinK119898(119860119860119887)+W
119860119909 minus 1198872
= min119910isinR119898+119896
1003817100381710038171003817119860 [119882119881119898] 119910 minus 1198871003817100381710038171003817
2
= min119910isinR119898+119896
1003817100381710038171003817[119880 119881119898+1]119867119910 minus 1198871003817100381710038171003817
2
= min119910isinR119898+119896
10038171003817100381710038171003817119867119910 minus [119880119881
119898+1]119879
11988710038171003817100381710038171003817
2
+ (119868 minus 119875)1198872
(34)
As 120583 rarr infin the reduced minimization problem (12) is thesame as the above minimization problem which shows that
119909119898= lim120583rarrinfin
119909120583119898 (35)
Therefore we obtain the following result
Theorem 3 Let 119909119898
be the 119898th iterate determined by aug-mented RRGMRES applied to (1) with initial iterate 119909
0= 0
Then1003817100381710038171003817119860119909119898 minus 119887
1003817100381710038171003817
2
= 120601119898(infin) (36)
3 Numerical Experiments
In this section we present some numerical examplesto illustrate the performance of the augmented Arnoldi-Tikhonov regularization method for the solution of large-scale linear ill-posed systems We compare the augmented
6 Mathematical Problems in Engineering
(1) Set 1205830= 0 and 119894 = 0
(2) Solve the least-squares problem
min119911isinR119898+119896+1
10038171003817100381710038171003817100381710038171003817
[12058312
119894(119867119877minus1
)119879
119868] 119911 minus [
0
(119880119881119898+1)119879
119887]
10038171003817100381710038171003817100381710038171003817
to obtain 119911120583119894119898
(3) Compute 120601119898(120583119894) = 119911119879
120583119894119898119911120583119894119898+ (119868 minus 119875)119887
2(4) Solve the least-squares problem
min119911isinR119898+119896+1
100381710038171003817100381710038171003817100381710038171003817
[12058312
119894(119867119877minus1
)119879
119868] 119911 minus [
0
119867119877minus1
(119867119877minus1
)119879
119911120583119894119898
]
100381710038171003817100381710038171003817100381710038171003817to obtain 119908
120583119894119898
(5) Compute 1206011015840119898(120583119894) = minus2119911
119879
120583119894119898119908120583119894119898
(6) Compute the new approximation
120583119894+1= 120583119894minus120601119898(120583119894) minus 1205782
1205762
1206011015840119898(120583119894)
(7) If |120583119894+1minus 120583119894| lt 10
minus6 stop else 120583119894= 120583119894+1 119894 = 119894 + 1 and go to 2
Algorithm 3 Newtonrsquos method for 120601119898(120583) = 120578
2
1205762
Arnoldi-Tikhonov regularization method implemented byAlgorithm 2 to the Arnoldi-Tikhonov regularization methodproposed in [11] The Arnoldi-Tikhonov regularizationmethod is denoted by ATRM while the augmented Arnoldi-Tikhonov regularization method is denoted by AATRMIn all the following tables we denote by MV the numberof matrix-vector products and by RERR the relative error119909120583119898
minus 119909119909 where 119909 is the exact solution of the linearerror-free system of (2) Note that the number of matrix-vector products in Algorithm 2 is 119896 + 119898min + 1198980 In all theexamples 120578 = 101
All the numerical experiments are performed in Matlabon a PC with the usual double precision where the floatingpoint relative accuracy is 222 sdot 10minus16
Example 4 The first example considered is the Fredholmintegral equation of the first kind which takes the genericform
int
1
0
119896 (119904 119905) 119909 (119905) 119889119905 = 119887 (119904) 0 le 119904 le 1 (37)
Here both the kernel 119896(119904 119905) and the right-hand side 119887(119904) areknown functions while 119909(119905) is the unknown function Fortest the kernel 119896(119904 119905) and the right-hand side 119887(119904) are chosenas
119896 (119904 119905) = 119904 (119905 minus 1) 119904 lt 119905
119905 (119904 minus 1) 119904 ge 119905119887 (119904) =
1199043
minus 119904
6 (38)
With this choice the exact solution of (37) is 119905 We use theMatlab program deriv2 from the regularization package [26]to discretize the integral equation (37) and to generate asystem of linear equations (2) with the coefficient matrix 119860 isinR200times200 and the solution 119909 isin R200 The condition number of119860 is 49 sdot 104 The right-hand side 119887 is given by 119887 = 119860119909 + 119890where the elements of the error vector 119890 are generated fromnormal distribution with mean zero and the norm of 119890 is10minus2
sdot 119860119909 The augmentation subspace is one-dimensional
Table 1 Computational results of Example 4
Method 1198980
MV RERRAATRM 0 2 195 sdot 10
minus2
AATRM 1 3 117 sdot 10minus2
AATRM 2 4 625 sdot 10minus2
AATRM 3 5 119 sdot 10minus1
ATRM 0 4 314 sdot 10minus1
ATRM 1 5 294 sdot 10minus1
ATRM 2 6 287 sdot 10minus1
ATRM 3 7 289 sdot 10minus1
and is spanned by the vector 119908 = [1 2 200]119879 Numericalresults for the example are reported in Table 1 for severalchoice of the number119898
0of additional iterations
FromTable 1 we can see that for 0 le 1198980le 3 AATRMhas
smaller relative errors than ATRM and the smallest relativeerror is given by AATRM with 119898
0= 1 The exact solution
119909 and the approximate solutions generated by AATRM andARTM with119898
0= 1 are depicted in Figure 1
Example 5 This example comes again from the regulariza-tion package [26] and is the inversion of Laplace transform
int
infin
0
119890minus119904119905
119909 (119904) 119889119904 = 119887 (119905) 119905 ge 1 (39)
where the right-hand side 119887(119905) and the exact solution 119909(119905) aregiven by
119887 (119905) =1
119905 + 12 119909 (119905) = 119890
minus1199052
(40)
The system of linear equations (2) with the coefficient matrix119860 isin R250times250 and the solution 119909 isin R250 is obtained byusing the Matlab program ilaplace from the regularizationpackage [26] In the same way as Example 4 we construct
Mathematical Problems in Engineering 7
0 50 100 150 2000
001
002
003
004
005
006
007
008
Exact solutionAATRMATRM
Figure 1 Example 4 Exact solution119909 approximation solutionswith1198980= 1
Table 2 Computational results of Example 5
Method 1198980
MV RERRAATRM 1 5 33 sdot 10
minus1
AATRM 2 6 91 sdot 10minus2
AATRM 3 7 27 sdot 10minus1
ATRM 1 6 57 sdot 10minus1
ATRM 2 7 31 sdot 10minus1
ATRM 3 8 45 sdot 10minus1
Table 3 Computational results of Example 6
Method 1198980
MV RERRAATRM 1 10 22 sdot 10
minus1
AATRM 2 11 11 sdot 10minus1
AATRM 3 12 21 sdot 10minus1
ATRM 1 10 49 sdot 10minus1
ATRM 2 11 12 sdot 10minus1
ATRM 3 12 23 sdot 10minus1
the right-hand side 119887 of the system of linear equations (1)The augmentation subspace is also one-dimensional and isspanned by the vector 119908 = [1 12 1250]
119879 Numericalresults for the example are reported in Table 2 for119898
0= 1 2 3
Table 2 shows that AATRM with 1198980= 2 has the smallest
relative error and AARTM works slightly better than ATRMfor this problem
Example 6 This example considered here is the same asExample 5 except that the right-hand side 119887(119905) and the exactsolution 119909(119905) are given by
119887 (119905) =2
(119905 + 12)3 119909 (119905) = 119905
2
119890minus1199052
(41)
By using the sameMatlab program as Example 5 we generatea system with 119860 isin R1000times1000 The augmentation subspace
is one-dimensional and is spanned by the vector 119908 =
[1 1 1]119879 In Table 3 we report numerical results for119898
0=
1 2 3
We observe from Table 3 that for this example AATRMhas almost the same relative errors as ATRM and the approx-imate solution can be slightly improved by the augmentationspace spanned by 119908 = [1 1 1]119879
4 Conclusions
In this paper we propose an iterative method for solvinglarge-scale linear ill-posed systems The method is based onthe Tikhonov regularization technique and the augmentedArnoldi technique The augmentation subspace is a user-supplied low-dimensional subspace which should containa rough approximation of the desired solution Numericalexperiments show that the augmented method is effective forsome practical problems
Acknowledgments
Yiqin Lin is supported by the National Natural ScienceFoundation of China under Grant 10801048 the NaturalScience Foundation ofHunanProvince underGrant 11JJ4009the Scientific Research Foundation of Education Bureau ofHunan Province for Outstanding Young Scholars in Univer-sity under Grant 10B038 the Science and Technology Plan-ning Project of Hunan Province under Grant 2010JT4042and the Chinese Postdoctoral Science Foundation underGrant 2012M511386 Liang Bao is supported by the NationalNatural Science Foundation of China under Grants 10926150and 11101149 and the Fundamental Research Funds for theCentral Universities
References
[1] H W Engl M Hanke and A Neubauer Regularization ofInverse Problems vol 375 of Mathematics and Its ApplicationsKluwer Academic Dordrecht The Netherlands 1996
[2] A N Tikhonov and V Y Arsenin Solutions of Ill-PosedProblems John Wiley amp Sons New York NY USA 1977
[3] A N Tikhonov A V Goncharsky V V Stepanov and AG Yagola Numerical Methods for the Solution of Ill-PosedProblems vol 328 of Mathematics and Its Applications KluwerAcademic Dordrecht The Netherlands 1995
[4] P C Hansen Rank-Deficient and Discrete Ill-Posed ProblemsSIAM Monographs on Mathematical Modeling and Computa-tion SIAM Philadelphia Pa USA 1998
[5] G H Golub and C F van Loan Matrix Computations JohnsHopkins Studies in the Mathematical Sciences Johns HopkinsUniversity Press Baltimore Md USA 3rd edition 1996
[6] A Bjorck Numerical Methods for Least Squares ProblemsSIAM Philadelphia Pa USA 1996
[7] A Bjorck ldquoA bidiagonalization algorithm for solving large andsparse ill-posed systems of linear equationsrdquo BIT NumericalMathematics vol 28 no 3 pp 659ndash670 1988
[8] D Calvetti S Morigi L Reichel and F Sgallari ldquoTikhonovregularization and the 119871-curve for large discrete ill-posed
8 Mathematical Problems in Engineering
problemsrdquo Journal of Computational and Applied Mathematicsvol 123 no 1-2 pp 423ndash446 2000
[9] D Calvetti and L Reichel ldquoTikhonov regularization of largelinear problemsrdquo BIT NumericalMathematics vol 43 no 2 pp263ndash283 2003
[10] D P OrsquoLeary and J A Simmons ldquoA bidiagonalization-regularization procedure for large scale discretizations of ill-posed problemsrdquo SIAM Journal on Scientific and StatisticalComputing vol 2 no 4 pp 474ndash489 1981
[11] B Lewis and L Reichel ldquoArnoldi-Tikhonov regularizationmethodsrdquo Journal of Computational and Applied Mathematicsvol 226 no 1 pp 92ndash102 2009
[12] T F Chan and K R Jackson ldquoNonlinearly preconditionedKrylov subspace methods for discrete Newton algorithmsrdquoSIAM Journal on Scientific and Statistical Computing vol 5 no3 pp 533ndash542 1984
[13] L Reichel F Sgallari andQ Ye ldquoTikhonov regularization basedon generalized Krylov subspace methodsrdquo Applied NumericalMathematics vol 62 no 9 pp 1215ndash1228 2012
[14] D Calvetti B Lewis and L Reichel ldquoOn the choice of subspacefor iterative methods for linear discrete ill-posed problemsrdquoInternational Journal of Applied Mathematics and ComputerScience vol 11 no 5 pp 1069ndash1092 2001 Numerical analysisand systems theory (Perpignan 2000)
[15] L Reichel and Q Ye ldquoBreakdown-free GMRES for singularsystemsrdquo SIAM Journal onMatrixAnalysis andApplications vol26 no 4 pp 1001ndash1021 2005
[16] J Baglama and L Reichel ldquoAugmented GMRES-type methodsrdquoNumerical Linear Algebra with Applications vol 14 no 4 pp337ndash350 2007
[17] K Morikuni L Reichel and K Hayami ldquoFGMRES for lineardiscrete problemsrdquo Tech Rep NII 2012
[18] D Calvetti B Lewis and L Reichel ldquoOn the regularizingproperties of the GMRES methodrdquo Numerische Mathematikvol 91 no 4 pp 605ndash625 2002
[19] R B Morgan ldquoA restarted GMRES method augmented witheigenvectorsrdquo SIAM Journal on Matrix Analysis and Applica-tions vol 16 no 4 pp 1154ndash1171 1995
[20] Y Saad ldquoA flexible inner-outer preconditioned GMRES algo-rithmrdquo SIAM Journal on Scientific Computing vol 14 no 2 pp461ndash469 1993
[21] B N Parlett The Symmetric Eigenvalue Problem vol 20 ofClassics in Applied Mathematics SIAM Philadelphia Pa USA1998
[22] C T Kelley Solving Nonlinear Equations with Newtonrsquos Methodvol 1 of Fundamentals of Algorithms SIAM Philadelphia PaUSA 2003
[23] J A Ezquerro andM A Hernandez ldquoOn a convex accelerationof Newtonrsquos methodrdquo Journal of Optimization Theory andApplications vol 100 no 2 pp 311ndash326 1999
[24] W Gander ldquoOn Halleyrsquos iteration methodrdquo The AmericanMathematical Monthly vol 92 no 2 pp 131ndash134 1985
[25] L Reichel and A Shyshkov ldquoA new zero-finder for Tikhonovregularizationrdquo BIT Numerical Mathematics vol 48 no 3 pp627ndash643 2008
[26] P C Hansen ldquoRegularization tools a Matlab package foranalysis and solution of discrete ill-posed problemsrdquoNumericalAlgorithms vol 6 no 1-2 pp 1ndash35 1994
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
4 Mathematical Problems in Engineering
= min119910isinR119898+119896
10038171003817100381710038171003817119867119910 minus [119880119881
119898+1]119879
11988710038171003817100381710038171003817
2
+(119868 minus 119875)1198872
+1
120583
10038171003817100381710038171198771199101003817100381710038171003817
2
(12)
in which 119875 = [119880119881119898+1][119880 119881
119898+1]119879 is an orthogonal projector
onto span([119880 119881119898+1]) Obviously the reduced minimization
problem (12) is equivalent to
min119910isinR119898+119896
10038171003817100381710038171003817119867119910 minus [119880119881
119898+1]119879
11988710038171003817100381710038171003817
2
+1
120583
10038171003817100381710038171198771199101003817100381710038171003817
2
= min119910isinR119898+119896
100381710038171003817100381710038171003817100381710038171003817100381710038171003817
[
[
119867
1
radic120583119877]
]
119910 minus [[119880119881119898+1]119879
119887
0]
100381710038171003817100381710038171003817100381710038171003817100381710038171003817
2
(13)
The normal equations of the minimization problem (13) is
(119867119879
119867 +1
120583119877119879
119877)119910 = 119867119879
[119880 119881119898+1]119879
119887 (14)
We denote the solution of the minimization problem (13) by119910120583119898
Then from (14) it follows that
119910120583119898= (119867
119879
119867 +1
120583119877119879
119877)
minus1
119867119879
[119880 119881119898+1]119879
119887 (15)
The approximate solution of (3) is
119909120583119898= [119882119881
119898] 119910120583119898 (16)
Since the matrix119867119879119867+ (1120583)119877119879119877 has a larger conditionnumber than the matrix [119867119879 (1radic120583)119877119879]119879 we apply the QRfactorization of [119867119879 (1radic120583)119877119879]119879 to obtain the solution 119910
120583119898
of (13) insteadTheQR factorization of [119867119879 (1radic120583)119877119879]119879 canbe implemented by a sequence of Givens rotations
Define
120601119898(120583) =
10038171003817100381710038171003817119887 minus 119860119909
120583119898
10038171003817100381710038171003817
2
(17)
Substituting 119909120583119898
= [119882119881119898]119910120583119898
into (17) and using themodified Arnoldi decomposition (10) yield
120601119898(120583) =
10038171003817100381710038171003817119887 minus 119860 [119882119881
119898] 119910120583119898
10038171003817100381710038171003817
2
=10038171003817100381710038171003817119887 minus [119880119881
119898+1]119867119910120583119898
10038171003817100381710038171003817
2
=10038171003817100381710038171003817[119880 119881119898+1]119879
119887 minus 119867119910120583119898
10038171003817100381710038171003817
2
+ (119868 minus 119875)1198872
(18)
Substituting 119910120583119898
in (15) into (18) we obtain
120601119898(120583) =
100381710038171003817100381710038171003817100381710038171003817
[119868 minus 119867(119867119879
119867 +1
120583119877119879
119877)
minus1
119867119879
] [119880119881119898+1]119879
119887
100381710038171003817100381710038171003817100381710038171003817
2
+ (119868 minus 119875) 1198872
(19)
Note that
119868 minus 119867(119867119879
119867 +1
120583119877119879
119877)
minus1
119867119879
= [120583119867119877minus1
(119867119877minus1
)119879
+ 119868]
minus1
(20)
Therefore the function 120601119898(120583) can be expressed as
120601119898(120583) =
10038171003817100381710038171003817100381710038171003817
[120583119867119877minus1
(119867119877minus1
)119879
+ 119868]
minus1
[119880 119881119898+1]119879
119887
10038171003817100381710038171003817100381710038171003817
2
+ (119868 minus 119875) 1198872
(21)
Concerning the properties of 120601119898(120583) we have the follow-
ing results which are similar to those of Theorem 21 in [11]for the Arnoldi-Tikhonov regularization method
Theorem 1 The function 120601119898(120583) has the representation
120601119898(120583) = 119887
119879
[119880 119881119898+1] [120583119867119877
minus1
(119867119877minus1
)119879
+ 119868]
minus2
times [119880119881119898+1]119879
119887 + (119868 minus 119875) 1198872
(22)
Assume that 119860119887 = 0 and (119867119877minus1)119879[119880 119881119898+1]119879
119887 = 0 Then 120601119898is
strictly decreasing and convex for 120583 ge 0 with 120601119898(0) = 119887
2Moreover the equation
120601119898(120583) = 120591 (23)
has a unique solution 120583120591119898
such that 0 lt 120583120591119898lt infin for any 120591
with100381710038171003817100381710038171003817119875N((119867119877minus1)119879)[119880 119881119898+1]
119879
119887100381710038171003817100381710038171003817
2
+ (119868 minus 119875) 1198872
lt 120591 lt 1198872
(24)
where 119875N((119867119877minus1)119879) denotes the orthogonal projector ontoN((119867119877minus1)
119879
)
Proof The proof follows the same argument of the proof ofTheorem 21 in [11] and therefore is omitted
We easily obtain the following theorem of which theproof is almost the same as that of Corollary 22 in [11]
Theorem 2 Assume that the modified Arnoldi process breaksdown at step 119895 Then the sequence 119904
119894119895
119894=0defined by
1199040= 1198872
119904119894=100381710038171003817100381710038171003817119875N((119867119877minus1)
119879
)[119880 119881119898+1]119879
119887100381710038171003817100381710038171003817
2
+ (119868 minus 119875)1198872
0 lt 119894 lt 119895
119904119895= 0
(25)
is decreasing
We apply the discrepancy principle to the discrepancy 119887minus119860119909120583119898
to determine an appropriate regularization parameter120583 so that it satisfies (5) To make the equation
120601119898(120583) = 120578
2
1205762 (26)
Mathematical Problems in Engineering 5
Input 119860 isin R119899times119899 119887 isin R119899119882 isin R119899times119896 120576 1205781198980
Output 119909120583119898
120583119898119898 = 119898min + 1198980
(1) Compute the modified Arnoldi decomposition 119860[119882119881119898] = [119880 119881
119898+1]119867 with119898 = 119898min + 1198980
where119898min is the smallest number such that (119868 minus 119875)1198872 lt 12057821205762(2) Compute the solution 120583
119898of the equation 120601
119898(120583) = 120578
2
1205762 by Newtonrsquos method
(3) Compute the solution 119910120583119898
of the least-squares problem (13) and obtain the approximatesolution 119909
120583119898= [119882119881
119898]119910120583119898
Algorithm 2 Augmented Arnoldi-Tikhonov regularization method
have a solution it follows from Theorem 1 that the inputparameter 119898 of the modified Arnoldi should be chosen sothat 119904
119898lt 1205782
1205762 To simplify the computations we ignore
the first term in 119904119898 Then the smallest iterative step number
denoted by 119898min of the modified Arnoldi process is chosenso that
(119868 minus 119875) 1198872
lt 1205782
1205762
(27)
We can improve the quality of the computed solution bychoosing the practical iterative step number 119898 somewhatlarger than119898min
After choosing the number 119898 of the modified Arnoldiiterative steps the regularization parameter 120583
119898is determined
by solving the nonlinear equation 120601119898(120583) = 120578
2
1205762 Many
numerical methods have been proposed for the solutionof a nonlinear equation including Newtonrsquos method [22]super-Newtonrsquos [23] method and Halleyrsquos method [24] Forthe specific nonlinear equation 120601
119898(120583) = 120578
2
1205762 Reichel and
Shyshkov proposed a new zero-finder method in their newpaper [25] In this paper we still make use of Newtonrsquosmethod to obtain the regularization parameter 120583
119898
In Algorithm 2 we outline the augmented Arnoldi-Tikhonov regularizationmethod which is used to solve large-scale linear ill-posed systems (1)
Newtonrsquos method requires to evaluate 120601119898(120583) and its first
derivative with respect to 120583 for computing approximations120583(119895)
119898of 120583119898for 119895 = 0 1 2 Let
119911120583119898= [120583119867119877
minus1
(119867119877minus1
)119879
+ 119868]
minus1
[119880 119881119898+1]119879
119887 (28)
that is 119911120583119898
satisfies the system of linear equations
[120583119867119877minus1
(119867119877minus1
)119879
+ 119868] 119911 = [119880119881119898+1]119879
119887 (29)
Note that the above linear system is the normal equations ofthe least-squares problem
min119911isinR119898+119896+1
1003817100381710038171003817100381710038171003817100381710038171003817
[12058312
(119867119877minus1
)119879
119868] 119911 minus [
0
(U 119881119898+1)119879
119887]
1003817100381710038171003817100381710038171003817100381710038171003817
(30)
For numerical stability we compute the vector 119911120583119898
by solvingthe least-squares problem Then the 120601
119898(120583) is evaluated by
computing
120601119898(120583) = 119911
119879
120583119898119911120583119898+ (119868 minus 119875) 119887
2
(31)
It is easy to show that the first derivative of 1206011015840119898(120583) can be
written as
1206011015840
119898(120583) = minus2119911
119879
120583119898119908120583119898 (32)
where
119908120583119898= [120583119867119877
minus1
(119867119877minus1
)119879
+ 119868]
minus1
119867119877minus1
(119867119877minus1
)119879
119911120583119898 (33)
Hence we may compute 119908120583119898
by solving a least-squaresproblem analogous to the above with the vector [119880 119881
119898+1]119879
119887
replaced by119867119877minus1(119867119877minus1)119879119911120583119898
The algorithm for implementing the Newton iteration for
solving the nonlinear equation 120601119898(120583) = 120578
2
1205762 is presented in
Algorithm 3Let 119909119898be the 119898th iterate produced by the augmented
RRGMRES in [16] Then 119909119898satisfies
1003817100381710038171003817119860119909119898 minus 1198871003817100381710038171003817
2
= min119909isinK119898(119860119860119887)+W
119860119909 minus 1198872
= min119910isinR119898+119896
1003817100381710038171003817119860 [119882119881119898] 119910 minus 1198871003817100381710038171003817
2
= min119910isinR119898+119896
1003817100381710038171003817[119880 119881119898+1]119867119910 minus 1198871003817100381710038171003817
2
= min119910isinR119898+119896
10038171003817100381710038171003817119867119910 minus [119880119881
119898+1]119879
11988710038171003817100381710038171003817
2
+ (119868 minus 119875)1198872
(34)
As 120583 rarr infin the reduced minimization problem (12) is thesame as the above minimization problem which shows that
119909119898= lim120583rarrinfin
119909120583119898 (35)
Therefore we obtain the following result
Theorem 3 Let 119909119898
be the 119898th iterate determined by aug-mented RRGMRES applied to (1) with initial iterate 119909
0= 0
Then1003817100381710038171003817119860119909119898 minus 119887
1003817100381710038171003817
2
= 120601119898(infin) (36)
3 Numerical Experiments
In this section we present some numerical examplesto illustrate the performance of the augmented Arnoldi-Tikhonov regularization method for the solution of large-scale linear ill-posed systems We compare the augmented
6 Mathematical Problems in Engineering
(1) Set 1205830= 0 and 119894 = 0
(2) Solve the least-squares problem
min119911isinR119898+119896+1
10038171003817100381710038171003817100381710038171003817
[12058312
119894(119867119877minus1
)119879
119868] 119911 minus [
0
(119880119881119898+1)119879
119887]
10038171003817100381710038171003817100381710038171003817
to obtain 119911120583119894119898
(3) Compute 120601119898(120583119894) = 119911119879
120583119894119898119911120583119894119898+ (119868 minus 119875)119887
2(4) Solve the least-squares problem
min119911isinR119898+119896+1
100381710038171003817100381710038171003817100381710038171003817
[12058312
119894(119867119877minus1
)119879
119868] 119911 minus [
0
119867119877minus1
(119867119877minus1
)119879
119911120583119894119898
]
100381710038171003817100381710038171003817100381710038171003817to obtain 119908
120583119894119898
(5) Compute 1206011015840119898(120583119894) = minus2119911
119879
120583119894119898119908120583119894119898
(6) Compute the new approximation
120583119894+1= 120583119894minus120601119898(120583119894) minus 1205782
1205762
1206011015840119898(120583119894)
(7) If |120583119894+1minus 120583119894| lt 10
minus6 stop else 120583119894= 120583119894+1 119894 = 119894 + 1 and go to 2
Algorithm 3 Newtonrsquos method for 120601119898(120583) = 120578
2
1205762
Arnoldi-Tikhonov regularization method implemented byAlgorithm 2 to the Arnoldi-Tikhonov regularization methodproposed in [11] The Arnoldi-Tikhonov regularizationmethod is denoted by ATRM while the augmented Arnoldi-Tikhonov regularization method is denoted by AATRMIn all the following tables we denote by MV the numberof matrix-vector products and by RERR the relative error119909120583119898
minus 119909119909 where 119909 is the exact solution of the linearerror-free system of (2) Note that the number of matrix-vector products in Algorithm 2 is 119896 + 119898min + 1198980 In all theexamples 120578 = 101
All the numerical experiments are performed in Matlabon a PC with the usual double precision where the floatingpoint relative accuracy is 222 sdot 10minus16
Example 4 The first example considered is the Fredholmintegral equation of the first kind which takes the genericform
int
1
0
119896 (119904 119905) 119909 (119905) 119889119905 = 119887 (119904) 0 le 119904 le 1 (37)
Here both the kernel 119896(119904 119905) and the right-hand side 119887(119904) areknown functions while 119909(119905) is the unknown function Fortest the kernel 119896(119904 119905) and the right-hand side 119887(119904) are chosenas
119896 (119904 119905) = 119904 (119905 minus 1) 119904 lt 119905
119905 (119904 minus 1) 119904 ge 119905119887 (119904) =
1199043
minus 119904
6 (38)
With this choice the exact solution of (37) is 119905 We use theMatlab program deriv2 from the regularization package [26]to discretize the integral equation (37) and to generate asystem of linear equations (2) with the coefficient matrix 119860 isinR200times200 and the solution 119909 isin R200 The condition number of119860 is 49 sdot 104 The right-hand side 119887 is given by 119887 = 119860119909 + 119890where the elements of the error vector 119890 are generated fromnormal distribution with mean zero and the norm of 119890 is10minus2
sdot 119860119909 The augmentation subspace is one-dimensional
Table 1 Computational results of Example 4
Method 1198980
MV RERRAATRM 0 2 195 sdot 10
minus2
AATRM 1 3 117 sdot 10minus2
AATRM 2 4 625 sdot 10minus2
AATRM 3 5 119 sdot 10minus1
ATRM 0 4 314 sdot 10minus1
ATRM 1 5 294 sdot 10minus1
ATRM 2 6 287 sdot 10minus1
ATRM 3 7 289 sdot 10minus1
and is spanned by the vector 119908 = [1 2 200]119879 Numericalresults for the example are reported in Table 1 for severalchoice of the number119898
0of additional iterations
FromTable 1 we can see that for 0 le 1198980le 3 AATRMhas
smaller relative errors than ATRM and the smallest relativeerror is given by AATRM with 119898
0= 1 The exact solution
119909 and the approximate solutions generated by AATRM andARTM with119898
0= 1 are depicted in Figure 1
Example 5 This example comes again from the regulariza-tion package [26] and is the inversion of Laplace transform
int
infin
0
119890minus119904119905
119909 (119904) 119889119904 = 119887 (119905) 119905 ge 1 (39)
where the right-hand side 119887(119905) and the exact solution 119909(119905) aregiven by
119887 (119905) =1
119905 + 12 119909 (119905) = 119890
minus1199052
(40)
The system of linear equations (2) with the coefficient matrix119860 isin R250times250 and the solution 119909 isin R250 is obtained byusing the Matlab program ilaplace from the regularizationpackage [26] In the same way as Example 4 we construct
Mathematical Problems in Engineering 7
0 50 100 150 2000
001
002
003
004
005
006
007
008
Exact solutionAATRMATRM
Figure 1 Example 4 Exact solution119909 approximation solutionswith1198980= 1
Table 2 Computational results of Example 5
Method 1198980
MV RERRAATRM 1 5 33 sdot 10
minus1
AATRM 2 6 91 sdot 10minus2
AATRM 3 7 27 sdot 10minus1
ATRM 1 6 57 sdot 10minus1
ATRM 2 7 31 sdot 10minus1
ATRM 3 8 45 sdot 10minus1
Table 3 Computational results of Example 6
Method 1198980
MV RERRAATRM 1 10 22 sdot 10
minus1
AATRM 2 11 11 sdot 10minus1
AATRM 3 12 21 sdot 10minus1
ATRM 1 10 49 sdot 10minus1
ATRM 2 11 12 sdot 10minus1
ATRM 3 12 23 sdot 10minus1
the right-hand side 119887 of the system of linear equations (1)The augmentation subspace is also one-dimensional and isspanned by the vector 119908 = [1 12 1250]
119879 Numericalresults for the example are reported in Table 2 for119898
0= 1 2 3
Table 2 shows that AATRM with 1198980= 2 has the smallest
relative error and AARTM works slightly better than ATRMfor this problem
Example 6 This example considered here is the same asExample 5 except that the right-hand side 119887(119905) and the exactsolution 119909(119905) are given by
119887 (119905) =2
(119905 + 12)3 119909 (119905) = 119905
2
119890minus1199052
(41)
By using the sameMatlab program as Example 5 we generatea system with 119860 isin R1000times1000 The augmentation subspace
is one-dimensional and is spanned by the vector 119908 =
[1 1 1]119879 In Table 3 we report numerical results for119898
0=
1 2 3
We observe from Table 3 that for this example AATRMhas almost the same relative errors as ATRM and the approx-imate solution can be slightly improved by the augmentationspace spanned by 119908 = [1 1 1]119879
4 Conclusions
In this paper we propose an iterative method for solvinglarge-scale linear ill-posed systems The method is based onthe Tikhonov regularization technique and the augmentedArnoldi technique The augmentation subspace is a user-supplied low-dimensional subspace which should containa rough approximation of the desired solution Numericalexperiments show that the augmented method is effective forsome practical problems
Acknowledgments
Yiqin Lin is supported by the National Natural ScienceFoundation of China under Grant 10801048 the NaturalScience Foundation ofHunanProvince underGrant 11JJ4009the Scientific Research Foundation of Education Bureau ofHunan Province for Outstanding Young Scholars in Univer-sity under Grant 10B038 the Science and Technology Plan-ning Project of Hunan Province under Grant 2010JT4042and the Chinese Postdoctoral Science Foundation underGrant 2012M511386 Liang Bao is supported by the NationalNatural Science Foundation of China under Grants 10926150and 11101149 and the Fundamental Research Funds for theCentral Universities
References
[1] H W Engl M Hanke and A Neubauer Regularization ofInverse Problems vol 375 of Mathematics and Its ApplicationsKluwer Academic Dordrecht The Netherlands 1996
[2] A N Tikhonov and V Y Arsenin Solutions of Ill-PosedProblems John Wiley amp Sons New York NY USA 1977
[3] A N Tikhonov A V Goncharsky V V Stepanov and AG Yagola Numerical Methods for the Solution of Ill-PosedProblems vol 328 of Mathematics and Its Applications KluwerAcademic Dordrecht The Netherlands 1995
[4] P C Hansen Rank-Deficient and Discrete Ill-Posed ProblemsSIAM Monographs on Mathematical Modeling and Computa-tion SIAM Philadelphia Pa USA 1998
[5] G H Golub and C F van Loan Matrix Computations JohnsHopkins Studies in the Mathematical Sciences Johns HopkinsUniversity Press Baltimore Md USA 3rd edition 1996
[6] A Bjorck Numerical Methods for Least Squares ProblemsSIAM Philadelphia Pa USA 1996
[7] A Bjorck ldquoA bidiagonalization algorithm for solving large andsparse ill-posed systems of linear equationsrdquo BIT NumericalMathematics vol 28 no 3 pp 659ndash670 1988
[8] D Calvetti S Morigi L Reichel and F Sgallari ldquoTikhonovregularization and the 119871-curve for large discrete ill-posed
8 Mathematical Problems in Engineering
problemsrdquo Journal of Computational and Applied Mathematicsvol 123 no 1-2 pp 423ndash446 2000
[9] D Calvetti and L Reichel ldquoTikhonov regularization of largelinear problemsrdquo BIT NumericalMathematics vol 43 no 2 pp263ndash283 2003
[10] D P OrsquoLeary and J A Simmons ldquoA bidiagonalization-regularization procedure for large scale discretizations of ill-posed problemsrdquo SIAM Journal on Scientific and StatisticalComputing vol 2 no 4 pp 474ndash489 1981
[11] B Lewis and L Reichel ldquoArnoldi-Tikhonov regularizationmethodsrdquo Journal of Computational and Applied Mathematicsvol 226 no 1 pp 92ndash102 2009
[12] T F Chan and K R Jackson ldquoNonlinearly preconditionedKrylov subspace methods for discrete Newton algorithmsrdquoSIAM Journal on Scientific and Statistical Computing vol 5 no3 pp 533ndash542 1984
[13] L Reichel F Sgallari andQ Ye ldquoTikhonov regularization basedon generalized Krylov subspace methodsrdquo Applied NumericalMathematics vol 62 no 9 pp 1215ndash1228 2012
[14] D Calvetti B Lewis and L Reichel ldquoOn the choice of subspacefor iterative methods for linear discrete ill-posed problemsrdquoInternational Journal of Applied Mathematics and ComputerScience vol 11 no 5 pp 1069ndash1092 2001 Numerical analysisand systems theory (Perpignan 2000)
[15] L Reichel and Q Ye ldquoBreakdown-free GMRES for singularsystemsrdquo SIAM Journal onMatrixAnalysis andApplications vol26 no 4 pp 1001ndash1021 2005
[16] J Baglama and L Reichel ldquoAugmented GMRES-type methodsrdquoNumerical Linear Algebra with Applications vol 14 no 4 pp337ndash350 2007
[17] K Morikuni L Reichel and K Hayami ldquoFGMRES for lineardiscrete problemsrdquo Tech Rep NII 2012
[18] D Calvetti B Lewis and L Reichel ldquoOn the regularizingproperties of the GMRES methodrdquo Numerische Mathematikvol 91 no 4 pp 605ndash625 2002
[19] R B Morgan ldquoA restarted GMRES method augmented witheigenvectorsrdquo SIAM Journal on Matrix Analysis and Applica-tions vol 16 no 4 pp 1154ndash1171 1995
[20] Y Saad ldquoA flexible inner-outer preconditioned GMRES algo-rithmrdquo SIAM Journal on Scientific Computing vol 14 no 2 pp461ndash469 1993
[21] B N Parlett The Symmetric Eigenvalue Problem vol 20 ofClassics in Applied Mathematics SIAM Philadelphia Pa USA1998
[22] C T Kelley Solving Nonlinear Equations with Newtonrsquos Methodvol 1 of Fundamentals of Algorithms SIAM Philadelphia PaUSA 2003
[23] J A Ezquerro andM A Hernandez ldquoOn a convex accelerationof Newtonrsquos methodrdquo Journal of Optimization Theory andApplications vol 100 no 2 pp 311ndash326 1999
[24] W Gander ldquoOn Halleyrsquos iteration methodrdquo The AmericanMathematical Monthly vol 92 no 2 pp 131ndash134 1985
[25] L Reichel and A Shyshkov ldquoA new zero-finder for Tikhonovregularizationrdquo BIT Numerical Mathematics vol 48 no 3 pp627ndash643 2008
[26] P C Hansen ldquoRegularization tools a Matlab package foranalysis and solution of discrete ill-posed problemsrdquoNumericalAlgorithms vol 6 no 1-2 pp 1ndash35 1994
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Mathematical Problems in Engineering 5
Input 119860 isin R119899times119899 119887 isin R119899119882 isin R119899times119896 120576 1205781198980
Output 119909120583119898
120583119898119898 = 119898min + 1198980
(1) Compute the modified Arnoldi decomposition 119860[119882119881119898] = [119880 119881
119898+1]119867 with119898 = 119898min + 1198980
where119898min is the smallest number such that (119868 minus 119875)1198872 lt 12057821205762(2) Compute the solution 120583
119898of the equation 120601
119898(120583) = 120578
2
1205762 by Newtonrsquos method
(3) Compute the solution 119910120583119898
of the least-squares problem (13) and obtain the approximatesolution 119909
120583119898= [119882119881
119898]119910120583119898
Algorithm 2 Augmented Arnoldi-Tikhonov regularization method
have a solution it follows from Theorem 1 that the inputparameter 119898 of the modified Arnoldi should be chosen sothat 119904
119898lt 1205782
1205762 To simplify the computations we ignore
the first term in 119904119898 Then the smallest iterative step number
denoted by 119898min of the modified Arnoldi process is chosenso that
(119868 minus 119875) 1198872
lt 1205782
1205762
(27)
We can improve the quality of the computed solution bychoosing the practical iterative step number 119898 somewhatlarger than119898min
After choosing the number 119898 of the modified Arnoldiiterative steps the regularization parameter 120583
119898is determined
by solving the nonlinear equation 120601119898(120583) = 120578
2
1205762 Many
numerical methods have been proposed for the solutionof a nonlinear equation including Newtonrsquos method [22]super-Newtonrsquos [23] method and Halleyrsquos method [24] Forthe specific nonlinear equation 120601
119898(120583) = 120578
2
1205762 Reichel and
Shyshkov proposed a new zero-finder method in their newpaper [25] In this paper we still make use of Newtonrsquosmethod to obtain the regularization parameter 120583
119898
In Algorithm 2 we outline the augmented Arnoldi-Tikhonov regularizationmethod which is used to solve large-scale linear ill-posed systems (1)
Newtonrsquos method requires to evaluate 120601119898(120583) and its first
derivative with respect to 120583 for computing approximations120583(119895)
119898of 120583119898for 119895 = 0 1 2 Let
119911120583119898= [120583119867119877
minus1
(119867119877minus1
)119879
+ 119868]
minus1
[119880 119881119898+1]119879
119887 (28)
that is 119911120583119898
satisfies the system of linear equations
[120583119867119877minus1
(119867119877minus1
)119879
+ 119868] 119911 = [119880119881119898+1]119879
119887 (29)
Note that the above linear system is the normal equations ofthe least-squares problem
min119911isinR119898+119896+1
1003817100381710038171003817100381710038171003817100381710038171003817
[12058312
(119867119877minus1
)119879
119868] 119911 minus [
0
(U 119881119898+1)119879
119887]
1003817100381710038171003817100381710038171003817100381710038171003817
(30)
For numerical stability we compute the vector 119911120583119898
by solvingthe least-squares problem Then the 120601
119898(120583) is evaluated by
computing
120601119898(120583) = 119911
119879
120583119898119911120583119898+ (119868 minus 119875) 119887
2
(31)
It is easy to show that the first derivative of 1206011015840119898(120583) can be
written as
1206011015840
119898(120583) = minus2119911
119879
120583119898119908120583119898 (32)
where
119908120583119898= [120583119867119877
minus1
(119867119877minus1
)119879
+ 119868]
minus1
119867119877minus1
(119867119877minus1
)119879
119911120583119898 (33)
Hence we may compute 119908120583119898
by solving a least-squaresproblem analogous to the above with the vector [119880 119881
119898+1]119879
119887
replaced by119867119877minus1(119867119877minus1)119879119911120583119898
The algorithm for implementing the Newton iteration for
solving the nonlinear equation 120601119898(120583) = 120578
2
1205762 is presented in
Algorithm 3Let 119909119898be the 119898th iterate produced by the augmented
RRGMRES in [16] Then 119909119898satisfies
1003817100381710038171003817119860119909119898 minus 1198871003817100381710038171003817
2
= min119909isinK119898(119860119860119887)+W
119860119909 minus 1198872
= min119910isinR119898+119896
1003817100381710038171003817119860 [119882119881119898] 119910 minus 1198871003817100381710038171003817
2
= min119910isinR119898+119896
1003817100381710038171003817[119880 119881119898+1]119867119910 minus 1198871003817100381710038171003817
2
= min119910isinR119898+119896
10038171003817100381710038171003817119867119910 minus [119880119881
119898+1]119879
11988710038171003817100381710038171003817
2
+ (119868 minus 119875)1198872
(34)
As 120583 rarr infin the reduced minimization problem (12) is thesame as the above minimization problem which shows that
119909119898= lim120583rarrinfin
119909120583119898 (35)
Therefore we obtain the following result
Theorem 3 Let 119909119898
be the 119898th iterate determined by aug-mented RRGMRES applied to (1) with initial iterate 119909
0= 0
Then1003817100381710038171003817119860119909119898 minus 119887
1003817100381710038171003817
2
= 120601119898(infin) (36)
3 Numerical Experiments
In this section we present some numerical examplesto illustrate the performance of the augmented Arnoldi-Tikhonov regularization method for the solution of large-scale linear ill-posed systems We compare the augmented
6 Mathematical Problems in Engineering
(1) Set 1205830= 0 and 119894 = 0
(2) Solve the least-squares problem
min119911isinR119898+119896+1
10038171003817100381710038171003817100381710038171003817
[12058312
119894(119867119877minus1
)119879
119868] 119911 minus [
0
(119880119881119898+1)119879
119887]
10038171003817100381710038171003817100381710038171003817
to obtain 119911120583119894119898
(3) Compute 120601119898(120583119894) = 119911119879
120583119894119898119911120583119894119898+ (119868 minus 119875)119887
2(4) Solve the least-squares problem
min119911isinR119898+119896+1
100381710038171003817100381710038171003817100381710038171003817
[12058312
119894(119867119877minus1
)119879
119868] 119911 minus [
0
119867119877minus1
(119867119877minus1
)119879
119911120583119894119898
]
100381710038171003817100381710038171003817100381710038171003817to obtain 119908
120583119894119898
(5) Compute 1206011015840119898(120583119894) = minus2119911
119879
120583119894119898119908120583119894119898
(6) Compute the new approximation
120583119894+1= 120583119894minus120601119898(120583119894) minus 1205782
1205762
1206011015840119898(120583119894)
(7) If |120583119894+1minus 120583119894| lt 10
minus6 stop else 120583119894= 120583119894+1 119894 = 119894 + 1 and go to 2
Algorithm 3 Newtonrsquos method for 120601119898(120583) = 120578
2
1205762
Arnoldi-Tikhonov regularization method implemented byAlgorithm 2 to the Arnoldi-Tikhonov regularization methodproposed in [11] The Arnoldi-Tikhonov regularizationmethod is denoted by ATRM while the augmented Arnoldi-Tikhonov regularization method is denoted by AATRMIn all the following tables we denote by MV the numberof matrix-vector products and by RERR the relative error119909120583119898
minus 119909119909 where 119909 is the exact solution of the linearerror-free system of (2) Note that the number of matrix-vector products in Algorithm 2 is 119896 + 119898min + 1198980 In all theexamples 120578 = 101
All the numerical experiments are performed in Matlabon a PC with the usual double precision where the floatingpoint relative accuracy is 222 sdot 10minus16
Example 4 The first example considered is the Fredholmintegral equation of the first kind which takes the genericform
int
1
0
119896 (119904 119905) 119909 (119905) 119889119905 = 119887 (119904) 0 le 119904 le 1 (37)
Here both the kernel 119896(119904 119905) and the right-hand side 119887(119904) areknown functions while 119909(119905) is the unknown function Fortest the kernel 119896(119904 119905) and the right-hand side 119887(119904) are chosenas
119896 (119904 119905) = 119904 (119905 minus 1) 119904 lt 119905
119905 (119904 minus 1) 119904 ge 119905119887 (119904) =
1199043
minus 119904
6 (38)
With this choice the exact solution of (37) is 119905 We use theMatlab program deriv2 from the regularization package [26]to discretize the integral equation (37) and to generate asystem of linear equations (2) with the coefficient matrix 119860 isinR200times200 and the solution 119909 isin R200 The condition number of119860 is 49 sdot 104 The right-hand side 119887 is given by 119887 = 119860119909 + 119890where the elements of the error vector 119890 are generated fromnormal distribution with mean zero and the norm of 119890 is10minus2
sdot 119860119909 The augmentation subspace is one-dimensional
Table 1 Computational results of Example 4
Method 1198980
MV RERRAATRM 0 2 195 sdot 10
minus2
AATRM 1 3 117 sdot 10minus2
AATRM 2 4 625 sdot 10minus2
AATRM 3 5 119 sdot 10minus1
ATRM 0 4 314 sdot 10minus1
ATRM 1 5 294 sdot 10minus1
ATRM 2 6 287 sdot 10minus1
ATRM 3 7 289 sdot 10minus1
and is spanned by the vector 119908 = [1 2 200]119879 Numericalresults for the example are reported in Table 1 for severalchoice of the number119898
0of additional iterations
FromTable 1 we can see that for 0 le 1198980le 3 AATRMhas
smaller relative errors than ATRM and the smallest relativeerror is given by AATRM with 119898
0= 1 The exact solution
119909 and the approximate solutions generated by AATRM andARTM with119898
0= 1 are depicted in Figure 1
Example 5 This example comes again from the regulariza-tion package [26] and is the inversion of Laplace transform
int
infin
0
119890minus119904119905
119909 (119904) 119889119904 = 119887 (119905) 119905 ge 1 (39)
where the right-hand side 119887(119905) and the exact solution 119909(119905) aregiven by
119887 (119905) =1
119905 + 12 119909 (119905) = 119890
minus1199052
(40)
The system of linear equations (2) with the coefficient matrix119860 isin R250times250 and the solution 119909 isin R250 is obtained byusing the Matlab program ilaplace from the regularizationpackage [26] In the same way as Example 4 we construct
Mathematical Problems in Engineering 7
0 50 100 150 2000
001
002
003
004
005
006
007
008
Exact solutionAATRMATRM
Figure 1 Example 4 Exact solution119909 approximation solutionswith1198980= 1
Table 2 Computational results of Example 5
Method 1198980
MV RERRAATRM 1 5 33 sdot 10
minus1
AATRM 2 6 91 sdot 10minus2
AATRM 3 7 27 sdot 10minus1
ATRM 1 6 57 sdot 10minus1
ATRM 2 7 31 sdot 10minus1
ATRM 3 8 45 sdot 10minus1
Table 3 Computational results of Example 6
Method 1198980
MV RERRAATRM 1 10 22 sdot 10
minus1
AATRM 2 11 11 sdot 10minus1
AATRM 3 12 21 sdot 10minus1
ATRM 1 10 49 sdot 10minus1
ATRM 2 11 12 sdot 10minus1
ATRM 3 12 23 sdot 10minus1
the right-hand side 119887 of the system of linear equations (1)The augmentation subspace is also one-dimensional and isspanned by the vector 119908 = [1 12 1250]
119879 Numericalresults for the example are reported in Table 2 for119898
0= 1 2 3
Table 2 shows that AATRM with 1198980= 2 has the smallest
relative error and AARTM works slightly better than ATRMfor this problem
Example 6 This example considered here is the same asExample 5 except that the right-hand side 119887(119905) and the exactsolution 119909(119905) are given by
119887 (119905) =2
(119905 + 12)3 119909 (119905) = 119905
2
119890minus1199052
(41)
By using the sameMatlab program as Example 5 we generatea system with 119860 isin R1000times1000 The augmentation subspace
is one-dimensional and is spanned by the vector 119908 =
[1 1 1]119879 In Table 3 we report numerical results for119898
0=
1 2 3
We observe from Table 3 that for this example AATRMhas almost the same relative errors as ATRM and the approx-imate solution can be slightly improved by the augmentationspace spanned by 119908 = [1 1 1]119879
4 Conclusions
In this paper we propose an iterative method for solvinglarge-scale linear ill-posed systems The method is based onthe Tikhonov regularization technique and the augmentedArnoldi technique The augmentation subspace is a user-supplied low-dimensional subspace which should containa rough approximation of the desired solution Numericalexperiments show that the augmented method is effective forsome practical problems
Acknowledgments
Yiqin Lin is supported by the National Natural ScienceFoundation of China under Grant 10801048 the NaturalScience Foundation ofHunanProvince underGrant 11JJ4009the Scientific Research Foundation of Education Bureau ofHunan Province for Outstanding Young Scholars in Univer-sity under Grant 10B038 the Science and Technology Plan-ning Project of Hunan Province under Grant 2010JT4042and the Chinese Postdoctoral Science Foundation underGrant 2012M511386 Liang Bao is supported by the NationalNatural Science Foundation of China under Grants 10926150and 11101149 and the Fundamental Research Funds for theCentral Universities
References
[1] H W Engl M Hanke and A Neubauer Regularization ofInverse Problems vol 375 of Mathematics and Its ApplicationsKluwer Academic Dordrecht The Netherlands 1996
[2] A N Tikhonov and V Y Arsenin Solutions of Ill-PosedProblems John Wiley amp Sons New York NY USA 1977
[3] A N Tikhonov A V Goncharsky V V Stepanov and AG Yagola Numerical Methods for the Solution of Ill-PosedProblems vol 328 of Mathematics and Its Applications KluwerAcademic Dordrecht The Netherlands 1995
[4] P C Hansen Rank-Deficient and Discrete Ill-Posed ProblemsSIAM Monographs on Mathematical Modeling and Computa-tion SIAM Philadelphia Pa USA 1998
[5] G H Golub and C F van Loan Matrix Computations JohnsHopkins Studies in the Mathematical Sciences Johns HopkinsUniversity Press Baltimore Md USA 3rd edition 1996
[6] A Bjorck Numerical Methods for Least Squares ProblemsSIAM Philadelphia Pa USA 1996
[7] A Bjorck ldquoA bidiagonalization algorithm for solving large andsparse ill-posed systems of linear equationsrdquo BIT NumericalMathematics vol 28 no 3 pp 659ndash670 1988
[8] D Calvetti S Morigi L Reichel and F Sgallari ldquoTikhonovregularization and the 119871-curve for large discrete ill-posed
8 Mathematical Problems in Engineering
problemsrdquo Journal of Computational and Applied Mathematicsvol 123 no 1-2 pp 423ndash446 2000
[9] D Calvetti and L Reichel ldquoTikhonov regularization of largelinear problemsrdquo BIT NumericalMathematics vol 43 no 2 pp263ndash283 2003
[10] D P OrsquoLeary and J A Simmons ldquoA bidiagonalization-regularization procedure for large scale discretizations of ill-posed problemsrdquo SIAM Journal on Scientific and StatisticalComputing vol 2 no 4 pp 474ndash489 1981
[11] B Lewis and L Reichel ldquoArnoldi-Tikhonov regularizationmethodsrdquo Journal of Computational and Applied Mathematicsvol 226 no 1 pp 92ndash102 2009
[12] T F Chan and K R Jackson ldquoNonlinearly preconditionedKrylov subspace methods for discrete Newton algorithmsrdquoSIAM Journal on Scientific and Statistical Computing vol 5 no3 pp 533ndash542 1984
[13] L Reichel F Sgallari andQ Ye ldquoTikhonov regularization basedon generalized Krylov subspace methodsrdquo Applied NumericalMathematics vol 62 no 9 pp 1215ndash1228 2012
[14] D Calvetti B Lewis and L Reichel ldquoOn the choice of subspacefor iterative methods for linear discrete ill-posed problemsrdquoInternational Journal of Applied Mathematics and ComputerScience vol 11 no 5 pp 1069ndash1092 2001 Numerical analysisand systems theory (Perpignan 2000)
[15] L Reichel and Q Ye ldquoBreakdown-free GMRES for singularsystemsrdquo SIAM Journal onMatrixAnalysis andApplications vol26 no 4 pp 1001ndash1021 2005
[16] J Baglama and L Reichel ldquoAugmented GMRES-type methodsrdquoNumerical Linear Algebra with Applications vol 14 no 4 pp337ndash350 2007
[17] K Morikuni L Reichel and K Hayami ldquoFGMRES for lineardiscrete problemsrdquo Tech Rep NII 2012
[18] D Calvetti B Lewis and L Reichel ldquoOn the regularizingproperties of the GMRES methodrdquo Numerische Mathematikvol 91 no 4 pp 605ndash625 2002
[19] R B Morgan ldquoA restarted GMRES method augmented witheigenvectorsrdquo SIAM Journal on Matrix Analysis and Applica-tions vol 16 no 4 pp 1154ndash1171 1995
[20] Y Saad ldquoA flexible inner-outer preconditioned GMRES algo-rithmrdquo SIAM Journal on Scientific Computing vol 14 no 2 pp461ndash469 1993
[21] B N Parlett The Symmetric Eigenvalue Problem vol 20 ofClassics in Applied Mathematics SIAM Philadelphia Pa USA1998
[22] C T Kelley Solving Nonlinear Equations with Newtonrsquos Methodvol 1 of Fundamentals of Algorithms SIAM Philadelphia PaUSA 2003
[23] J A Ezquerro andM A Hernandez ldquoOn a convex accelerationof Newtonrsquos methodrdquo Journal of Optimization Theory andApplications vol 100 no 2 pp 311ndash326 1999
[24] W Gander ldquoOn Halleyrsquos iteration methodrdquo The AmericanMathematical Monthly vol 92 no 2 pp 131ndash134 1985
[25] L Reichel and A Shyshkov ldquoA new zero-finder for Tikhonovregularizationrdquo BIT Numerical Mathematics vol 48 no 3 pp627ndash643 2008
[26] P C Hansen ldquoRegularization tools a Matlab package foranalysis and solution of discrete ill-posed problemsrdquoNumericalAlgorithms vol 6 no 1-2 pp 1ndash35 1994
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
6 Mathematical Problems in Engineering
(1) Set 1205830= 0 and 119894 = 0
(2) Solve the least-squares problem
min119911isinR119898+119896+1
10038171003817100381710038171003817100381710038171003817
[12058312
119894(119867119877minus1
)119879
119868] 119911 minus [
0
(119880119881119898+1)119879
119887]
10038171003817100381710038171003817100381710038171003817
to obtain 119911120583119894119898
(3) Compute 120601119898(120583119894) = 119911119879
120583119894119898119911120583119894119898+ (119868 minus 119875)119887
2(4) Solve the least-squares problem
min119911isinR119898+119896+1
100381710038171003817100381710038171003817100381710038171003817
[12058312
119894(119867119877minus1
)119879
119868] 119911 minus [
0
119867119877minus1
(119867119877minus1
)119879
119911120583119894119898
]
100381710038171003817100381710038171003817100381710038171003817to obtain 119908
120583119894119898
(5) Compute 1206011015840119898(120583119894) = minus2119911
119879
120583119894119898119908120583119894119898
(6) Compute the new approximation
120583119894+1= 120583119894minus120601119898(120583119894) minus 1205782
1205762
1206011015840119898(120583119894)
(7) If |120583119894+1minus 120583119894| lt 10
minus6 stop else 120583119894= 120583119894+1 119894 = 119894 + 1 and go to 2
Algorithm 3 Newtonrsquos method for 120601119898(120583) = 120578
2
1205762
Arnoldi-Tikhonov regularization method implemented byAlgorithm 2 to the Arnoldi-Tikhonov regularization methodproposed in [11] The Arnoldi-Tikhonov regularizationmethod is denoted by ATRM while the augmented Arnoldi-Tikhonov regularization method is denoted by AATRMIn all the following tables we denote by MV the numberof matrix-vector products and by RERR the relative error119909120583119898
minus 119909119909 where 119909 is the exact solution of the linearerror-free system of (2) Note that the number of matrix-vector products in Algorithm 2 is 119896 + 119898min + 1198980 In all theexamples 120578 = 101
All the numerical experiments are performed in Matlabon a PC with the usual double precision where the floatingpoint relative accuracy is 222 sdot 10minus16
Example 4 The first example considered is the Fredholmintegral equation of the first kind which takes the genericform
int
1
0
119896 (119904 119905) 119909 (119905) 119889119905 = 119887 (119904) 0 le 119904 le 1 (37)
Here both the kernel 119896(119904 119905) and the right-hand side 119887(119904) areknown functions while 119909(119905) is the unknown function Fortest the kernel 119896(119904 119905) and the right-hand side 119887(119904) are chosenas
119896 (119904 119905) = 119904 (119905 minus 1) 119904 lt 119905
119905 (119904 minus 1) 119904 ge 119905119887 (119904) =
1199043
minus 119904
6 (38)
With this choice the exact solution of (37) is 119905 We use theMatlab program deriv2 from the regularization package [26]to discretize the integral equation (37) and to generate asystem of linear equations (2) with the coefficient matrix 119860 isinR200times200 and the solution 119909 isin R200 The condition number of119860 is 49 sdot 104 The right-hand side 119887 is given by 119887 = 119860119909 + 119890where the elements of the error vector 119890 are generated fromnormal distribution with mean zero and the norm of 119890 is10minus2
sdot 119860119909 The augmentation subspace is one-dimensional
Table 1 Computational results of Example 4
Method 1198980
MV RERRAATRM 0 2 195 sdot 10
minus2
AATRM 1 3 117 sdot 10minus2
AATRM 2 4 625 sdot 10minus2
AATRM 3 5 119 sdot 10minus1
ATRM 0 4 314 sdot 10minus1
ATRM 1 5 294 sdot 10minus1
ATRM 2 6 287 sdot 10minus1
ATRM 3 7 289 sdot 10minus1
and is spanned by the vector 119908 = [1 2 200]119879 Numericalresults for the example are reported in Table 1 for severalchoice of the number119898
0of additional iterations
FromTable 1 we can see that for 0 le 1198980le 3 AATRMhas
smaller relative errors than ATRM and the smallest relativeerror is given by AATRM with 119898
0= 1 The exact solution
119909 and the approximate solutions generated by AATRM andARTM with119898
0= 1 are depicted in Figure 1
Example 5 This example comes again from the regulariza-tion package [26] and is the inversion of Laplace transform
int
infin
0
119890minus119904119905
119909 (119904) 119889119904 = 119887 (119905) 119905 ge 1 (39)
where the right-hand side 119887(119905) and the exact solution 119909(119905) aregiven by
119887 (119905) =1
119905 + 12 119909 (119905) = 119890
minus1199052
(40)
The system of linear equations (2) with the coefficient matrix119860 isin R250times250 and the solution 119909 isin R250 is obtained byusing the Matlab program ilaplace from the regularizationpackage [26] In the same way as Example 4 we construct
Mathematical Problems in Engineering 7
0 50 100 150 2000
001
002
003
004
005
006
007
008
Exact solutionAATRMATRM
Figure 1 Example 4 Exact solution119909 approximation solutionswith1198980= 1
Table 2 Computational results of Example 5
Method 1198980
MV RERRAATRM 1 5 33 sdot 10
minus1
AATRM 2 6 91 sdot 10minus2
AATRM 3 7 27 sdot 10minus1
ATRM 1 6 57 sdot 10minus1
ATRM 2 7 31 sdot 10minus1
ATRM 3 8 45 sdot 10minus1
Table 3 Computational results of Example 6
Method 1198980
MV RERRAATRM 1 10 22 sdot 10
minus1
AATRM 2 11 11 sdot 10minus1
AATRM 3 12 21 sdot 10minus1
ATRM 1 10 49 sdot 10minus1
ATRM 2 11 12 sdot 10minus1
ATRM 3 12 23 sdot 10minus1
the right-hand side 119887 of the system of linear equations (1)The augmentation subspace is also one-dimensional and isspanned by the vector 119908 = [1 12 1250]
119879 Numericalresults for the example are reported in Table 2 for119898
0= 1 2 3
Table 2 shows that AATRM with 1198980= 2 has the smallest
relative error and AARTM works slightly better than ATRMfor this problem
Example 6 This example considered here is the same asExample 5 except that the right-hand side 119887(119905) and the exactsolution 119909(119905) are given by
119887 (119905) =2
(119905 + 12)3 119909 (119905) = 119905
2
119890minus1199052
(41)
By using the sameMatlab program as Example 5 we generatea system with 119860 isin R1000times1000 The augmentation subspace
is one-dimensional and is spanned by the vector 119908 =
[1 1 1]119879 In Table 3 we report numerical results for119898
0=
1 2 3
We observe from Table 3 that for this example AATRMhas almost the same relative errors as ATRM and the approx-imate solution can be slightly improved by the augmentationspace spanned by 119908 = [1 1 1]119879
4 Conclusions
In this paper we propose an iterative method for solvinglarge-scale linear ill-posed systems The method is based onthe Tikhonov regularization technique and the augmentedArnoldi technique The augmentation subspace is a user-supplied low-dimensional subspace which should containa rough approximation of the desired solution Numericalexperiments show that the augmented method is effective forsome practical problems
Acknowledgments
Yiqin Lin is supported by the National Natural ScienceFoundation of China under Grant 10801048 the NaturalScience Foundation ofHunanProvince underGrant 11JJ4009the Scientific Research Foundation of Education Bureau ofHunan Province for Outstanding Young Scholars in Univer-sity under Grant 10B038 the Science and Technology Plan-ning Project of Hunan Province under Grant 2010JT4042and the Chinese Postdoctoral Science Foundation underGrant 2012M511386 Liang Bao is supported by the NationalNatural Science Foundation of China under Grants 10926150and 11101149 and the Fundamental Research Funds for theCentral Universities
References
[1] H W Engl M Hanke and A Neubauer Regularization ofInverse Problems vol 375 of Mathematics and Its ApplicationsKluwer Academic Dordrecht The Netherlands 1996
[2] A N Tikhonov and V Y Arsenin Solutions of Ill-PosedProblems John Wiley amp Sons New York NY USA 1977
[3] A N Tikhonov A V Goncharsky V V Stepanov and AG Yagola Numerical Methods for the Solution of Ill-PosedProblems vol 328 of Mathematics and Its Applications KluwerAcademic Dordrecht The Netherlands 1995
[4] P C Hansen Rank-Deficient and Discrete Ill-Posed ProblemsSIAM Monographs on Mathematical Modeling and Computa-tion SIAM Philadelphia Pa USA 1998
[5] G H Golub and C F van Loan Matrix Computations JohnsHopkins Studies in the Mathematical Sciences Johns HopkinsUniversity Press Baltimore Md USA 3rd edition 1996
[6] A Bjorck Numerical Methods for Least Squares ProblemsSIAM Philadelphia Pa USA 1996
[7] A Bjorck ldquoA bidiagonalization algorithm for solving large andsparse ill-posed systems of linear equationsrdquo BIT NumericalMathematics vol 28 no 3 pp 659ndash670 1988
[8] D Calvetti S Morigi L Reichel and F Sgallari ldquoTikhonovregularization and the 119871-curve for large discrete ill-posed
8 Mathematical Problems in Engineering
problemsrdquo Journal of Computational and Applied Mathematicsvol 123 no 1-2 pp 423ndash446 2000
[9] D Calvetti and L Reichel ldquoTikhonov regularization of largelinear problemsrdquo BIT NumericalMathematics vol 43 no 2 pp263ndash283 2003
[10] D P OrsquoLeary and J A Simmons ldquoA bidiagonalization-regularization procedure for large scale discretizations of ill-posed problemsrdquo SIAM Journal on Scientific and StatisticalComputing vol 2 no 4 pp 474ndash489 1981
[11] B Lewis and L Reichel ldquoArnoldi-Tikhonov regularizationmethodsrdquo Journal of Computational and Applied Mathematicsvol 226 no 1 pp 92ndash102 2009
[12] T F Chan and K R Jackson ldquoNonlinearly preconditionedKrylov subspace methods for discrete Newton algorithmsrdquoSIAM Journal on Scientific and Statistical Computing vol 5 no3 pp 533ndash542 1984
[13] L Reichel F Sgallari andQ Ye ldquoTikhonov regularization basedon generalized Krylov subspace methodsrdquo Applied NumericalMathematics vol 62 no 9 pp 1215ndash1228 2012
[14] D Calvetti B Lewis and L Reichel ldquoOn the choice of subspacefor iterative methods for linear discrete ill-posed problemsrdquoInternational Journal of Applied Mathematics and ComputerScience vol 11 no 5 pp 1069ndash1092 2001 Numerical analysisand systems theory (Perpignan 2000)
[15] L Reichel and Q Ye ldquoBreakdown-free GMRES for singularsystemsrdquo SIAM Journal onMatrixAnalysis andApplications vol26 no 4 pp 1001ndash1021 2005
[16] J Baglama and L Reichel ldquoAugmented GMRES-type methodsrdquoNumerical Linear Algebra with Applications vol 14 no 4 pp337ndash350 2007
[17] K Morikuni L Reichel and K Hayami ldquoFGMRES for lineardiscrete problemsrdquo Tech Rep NII 2012
[18] D Calvetti B Lewis and L Reichel ldquoOn the regularizingproperties of the GMRES methodrdquo Numerische Mathematikvol 91 no 4 pp 605ndash625 2002
[19] R B Morgan ldquoA restarted GMRES method augmented witheigenvectorsrdquo SIAM Journal on Matrix Analysis and Applica-tions vol 16 no 4 pp 1154ndash1171 1995
[20] Y Saad ldquoA flexible inner-outer preconditioned GMRES algo-rithmrdquo SIAM Journal on Scientific Computing vol 14 no 2 pp461ndash469 1993
[21] B N Parlett The Symmetric Eigenvalue Problem vol 20 ofClassics in Applied Mathematics SIAM Philadelphia Pa USA1998
[22] C T Kelley Solving Nonlinear Equations with Newtonrsquos Methodvol 1 of Fundamentals of Algorithms SIAM Philadelphia PaUSA 2003
[23] J A Ezquerro andM A Hernandez ldquoOn a convex accelerationof Newtonrsquos methodrdquo Journal of Optimization Theory andApplications vol 100 no 2 pp 311ndash326 1999
[24] W Gander ldquoOn Halleyrsquos iteration methodrdquo The AmericanMathematical Monthly vol 92 no 2 pp 131ndash134 1985
[25] L Reichel and A Shyshkov ldquoA new zero-finder for Tikhonovregularizationrdquo BIT Numerical Mathematics vol 48 no 3 pp627ndash643 2008
[26] P C Hansen ldquoRegularization tools a Matlab package foranalysis and solution of discrete ill-posed problemsrdquoNumericalAlgorithms vol 6 no 1-2 pp 1ndash35 1994
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Mathematical Problems in Engineering 7
0 50 100 150 2000
001
002
003
004
005
006
007
008
Exact solutionAATRMATRM
Figure 1 Example 4 Exact solution119909 approximation solutionswith1198980= 1
Table 2 Computational results of Example 5
Method 1198980
MV RERRAATRM 1 5 33 sdot 10
minus1
AATRM 2 6 91 sdot 10minus2
AATRM 3 7 27 sdot 10minus1
ATRM 1 6 57 sdot 10minus1
ATRM 2 7 31 sdot 10minus1
ATRM 3 8 45 sdot 10minus1
Table 3 Computational results of Example 6
Method 1198980
MV RERRAATRM 1 10 22 sdot 10
minus1
AATRM 2 11 11 sdot 10minus1
AATRM 3 12 21 sdot 10minus1
ATRM 1 10 49 sdot 10minus1
ATRM 2 11 12 sdot 10minus1
ATRM 3 12 23 sdot 10minus1
the right-hand side 119887 of the system of linear equations (1)The augmentation subspace is also one-dimensional and isspanned by the vector 119908 = [1 12 1250]
119879 Numericalresults for the example are reported in Table 2 for119898
0= 1 2 3
Table 2 shows that AATRM with 1198980= 2 has the smallest
relative error and AARTM works slightly better than ATRMfor this problem
Example 6 This example considered here is the same asExample 5 except that the right-hand side 119887(119905) and the exactsolution 119909(119905) are given by
119887 (119905) =2
(119905 + 12)3 119909 (119905) = 119905
2
119890minus1199052
(41)
By using the sameMatlab program as Example 5 we generatea system with 119860 isin R1000times1000 The augmentation subspace
is one-dimensional and is spanned by the vector 119908 =
[1 1 1]119879 In Table 3 we report numerical results for119898
0=
1 2 3
We observe from Table 3 that for this example AATRMhas almost the same relative errors as ATRM and the approx-imate solution can be slightly improved by the augmentationspace spanned by 119908 = [1 1 1]119879
4 Conclusions
In this paper we propose an iterative method for solvinglarge-scale linear ill-posed systems The method is based onthe Tikhonov regularization technique and the augmentedArnoldi technique The augmentation subspace is a user-supplied low-dimensional subspace which should containa rough approximation of the desired solution Numericalexperiments show that the augmented method is effective forsome practical problems
Acknowledgments
Yiqin Lin is supported by the National Natural ScienceFoundation of China under Grant 10801048 the NaturalScience Foundation ofHunanProvince underGrant 11JJ4009the Scientific Research Foundation of Education Bureau ofHunan Province for Outstanding Young Scholars in Univer-sity under Grant 10B038 the Science and Technology Plan-ning Project of Hunan Province under Grant 2010JT4042and the Chinese Postdoctoral Science Foundation underGrant 2012M511386 Liang Bao is supported by the NationalNatural Science Foundation of China under Grants 10926150and 11101149 and the Fundamental Research Funds for theCentral Universities
References
[1] H W Engl M Hanke and A Neubauer Regularization ofInverse Problems vol 375 of Mathematics and Its ApplicationsKluwer Academic Dordrecht The Netherlands 1996
[2] A N Tikhonov and V Y Arsenin Solutions of Ill-PosedProblems John Wiley amp Sons New York NY USA 1977
[3] A N Tikhonov A V Goncharsky V V Stepanov and AG Yagola Numerical Methods for the Solution of Ill-PosedProblems vol 328 of Mathematics and Its Applications KluwerAcademic Dordrecht The Netherlands 1995
[4] P C Hansen Rank-Deficient and Discrete Ill-Posed ProblemsSIAM Monographs on Mathematical Modeling and Computa-tion SIAM Philadelphia Pa USA 1998
[5] G H Golub and C F van Loan Matrix Computations JohnsHopkins Studies in the Mathematical Sciences Johns HopkinsUniversity Press Baltimore Md USA 3rd edition 1996
[6] A Bjorck Numerical Methods for Least Squares ProblemsSIAM Philadelphia Pa USA 1996
[7] A Bjorck ldquoA bidiagonalization algorithm for solving large andsparse ill-posed systems of linear equationsrdquo BIT NumericalMathematics vol 28 no 3 pp 659ndash670 1988
[8] D Calvetti S Morigi L Reichel and F Sgallari ldquoTikhonovregularization and the 119871-curve for large discrete ill-posed
8 Mathematical Problems in Engineering
problemsrdquo Journal of Computational and Applied Mathematicsvol 123 no 1-2 pp 423ndash446 2000
[9] D Calvetti and L Reichel ldquoTikhonov regularization of largelinear problemsrdquo BIT NumericalMathematics vol 43 no 2 pp263ndash283 2003
[10] D P OrsquoLeary and J A Simmons ldquoA bidiagonalization-regularization procedure for large scale discretizations of ill-posed problemsrdquo SIAM Journal on Scientific and StatisticalComputing vol 2 no 4 pp 474ndash489 1981
[11] B Lewis and L Reichel ldquoArnoldi-Tikhonov regularizationmethodsrdquo Journal of Computational and Applied Mathematicsvol 226 no 1 pp 92ndash102 2009
[12] T F Chan and K R Jackson ldquoNonlinearly preconditionedKrylov subspace methods for discrete Newton algorithmsrdquoSIAM Journal on Scientific and Statistical Computing vol 5 no3 pp 533ndash542 1984
[13] L Reichel F Sgallari andQ Ye ldquoTikhonov regularization basedon generalized Krylov subspace methodsrdquo Applied NumericalMathematics vol 62 no 9 pp 1215ndash1228 2012
[14] D Calvetti B Lewis and L Reichel ldquoOn the choice of subspacefor iterative methods for linear discrete ill-posed problemsrdquoInternational Journal of Applied Mathematics and ComputerScience vol 11 no 5 pp 1069ndash1092 2001 Numerical analysisand systems theory (Perpignan 2000)
[15] L Reichel and Q Ye ldquoBreakdown-free GMRES for singularsystemsrdquo SIAM Journal onMatrixAnalysis andApplications vol26 no 4 pp 1001ndash1021 2005
[16] J Baglama and L Reichel ldquoAugmented GMRES-type methodsrdquoNumerical Linear Algebra with Applications vol 14 no 4 pp337ndash350 2007
[17] K Morikuni L Reichel and K Hayami ldquoFGMRES for lineardiscrete problemsrdquo Tech Rep NII 2012
[18] D Calvetti B Lewis and L Reichel ldquoOn the regularizingproperties of the GMRES methodrdquo Numerische Mathematikvol 91 no 4 pp 605ndash625 2002
[19] R B Morgan ldquoA restarted GMRES method augmented witheigenvectorsrdquo SIAM Journal on Matrix Analysis and Applica-tions vol 16 no 4 pp 1154ndash1171 1995
[20] Y Saad ldquoA flexible inner-outer preconditioned GMRES algo-rithmrdquo SIAM Journal on Scientific Computing vol 14 no 2 pp461ndash469 1993
[21] B N Parlett The Symmetric Eigenvalue Problem vol 20 ofClassics in Applied Mathematics SIAM Philadelphia Pa USA1998
[22] C T Kelley Solving Nonlinear Equations with Newtonrsquos Methodvol 1 of Fundamentals of Algorithms SIAM Philadelphia PaUSA 2003
[23] J A Ezquerro andM A Hernandez ldquoOn a convex accelerationof Newtonrsquos methodrdquo Journal of Optimization Theory andApplications vol 100 no 2 pp 311ndash326 1999
[24] W Gander ldquoOn Halleyrsquos iteration methodrdquo The AmericanMathematical Monthly vol 92 no 2 pp 131ndash134 1985
[25] L Reichel and A Shyshkov ldquoA new zero-finder for Tikhonovregularizationrdquo BIT Numerical Mathematics vol 48 no 3 pp627ndash643 2008
[26] P C Hansen ldquoRegularization tools a Matlab package foranalysis and solution of discrete ill-posed problemsrdquoNumericalAlgorithms vol 6 no 1-2 pp 1ndash35 1994
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
8 Mathematical Problems in Engineering
problemsrdquo Journal of Computational and Applied Mathematicsvol 123 no 1-2 pp 423ndash446 2000
[9] D Calvetti and L Reichel ldquoTikhonov regularization of largelinear problemsrdquo BIT NumericalMathematics vol 43 no 2 pp263ndash283 2003
[10] D P OrsquoLeary and J A Simmons ldquoA bidiagonalization-regularization procedure for large scale discretizations of ill-posed problemsrdquo SIAM Journal on Scientific and StatisticalComputing vol 2 no 4 pp 474ndash489 1981
[11] B Lewis and L Reichel ldquoArnoldi-Tikhonov regularizationmethodsrdquo Journal of Computational and Applied Mathematicsvol 226 no 1 pp 92ndash102 2009
[12] T F Chan and K R Jackson ldquoNonlinearly preconditionedKrylov subspace methods for discrete Newton algorithmsrdquoSIAM Journal on Scientific and Statistical Computing vol 5 no3 pp 533ndash542 1984
[13] L Reichel F Sgallari andQ Ye ldquoTikhonov regularization basedon generalized Krylov subspace methodsrdquo Applied NumericalMathematics vol 62 no 9 pp 1215ndash1228 2012
[14] D Calvetti B Lewis and L Reichel ldquoOn the choice of subspacefor iterative methods for linear discrete ill-posed problemsrdquoInternational Journal of Applied Mathematics and ComputerScience vol 11 no 5 pp 1069ndash1092 2001 Numerical analysisand systems theory (Perpignan 2000)
[15] L Reichel and Q Ye ldquoBreakdown-free GMRES for singularsystemsrdquo SIAM Journal onMatrixAnalysis andApplications vol26 no 4 pp 1001ndash1021 2005
[16] J Baglama and L Reichel ldquoAugmented GMRES-type methodsrdquoNumerical Linear Algebra with Applications vol 14 no 4 pp337ndash350 2007
[17] K Morikuni L Reichel and K Hayami ldquoFGMRES for lineardiscrete problemsrdquo Tech Rep NII 2012
[18] D Calvetti B Lewis and L Reichel ldquoOn the regularizingproperties of the GMRES methodrdquo Numerische Mathematikvol 91 no 4 pp 605ndash625 2002
[19] R B Morgan ldquoA restarted GMRES method augmented witheigenvectorsrdquo SIAM Journal on Matrix Analysis and Applica-tions vol 16 no 4 pp 1154ndash1171 1995
[20] Y Saad ldquoA flexible inner-outer preconditioned GMRES algo-rithmrdquo SIAM Journal on Scientific Computing vol 14 no 2 pp461ndash469 1993
[21] B N Parlett The Symmetric Eigenvalue Problem vol 20 ofClassics in Applied Mathematics SIAM Philadelphia Pa USA1998
[22] C T Kelley Solving Nonlinear Equations with Newtonrsquos Methodvol 1 of Fundamentals of Algorithms SIAM Philadelphia PaUSA 2003
[23] J A Ezquerro andM A Hernandez ldquoOn a convex accelerationof Newtonrsquos methodrdquo Journal of Optimization Theory andApplications vol 100 no 2 pp 311ndash326 1999
[24] W Gander ldquoOn Halleyrsquos iteration methodrdquo The AmericanMathematical Monthly vol 92 no 2 pp 131ndash134 1985
[25] L Reichel and A Shyshkov ldquoA new zero-finder for Tikhonovregularizationrdquo BIT Numerical Mathematics vol 48 no 3 pp627ndash643 2008
[26] P C Hansen ldquoRegularization tools a Matlab package foranalysis and solution of discrete ill-posed problemsrdquoNumericalAlgorithms vol 6 no 1-2 pp 1ndash35 1994
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of