1

Harnessing the Cloud for Securely OutsourcingLarge-scale Systems of Linear Equations

Cong Wang, Student Member, IEEE, Kui Ren, Member, IEEE, Jia Wang, Member, IEEE, andKarthik Mahendra Raje Urs

Abstract—Cloud computing economically enables customers with limited computational resources to outsource large-scale computa-tions to the cloud. However, how to protect customers’ confidential data involved in the computations then becomes a major securityconcern. In this paper, we present a secure outsourcing mechanism for solving large-scale systems of linear equations (LE) in cloud.Because applying traditional approaches like Gaussian elimination or LU decomposition (aka. direct method) to such large-scaleLE problems would be prohibitively expensive, we build the secure LE outsourcing mechanism via a completely different approach— iterative method, which is much easier to implement in practice and only demands relatively simpler matrix-vector operations.Specifically, our mechanism enables a customer to securely harness the cloud for iteratively finding successive approximations to theLE solution, while keeping both the sensitive input and output of the computation private. For robust cheating detection, we furtherexplore the algebraic property of matrix-vector operations and propose an efficient result verification mechanism, which allows thecustomer to verify all answers received from previous iterative approximations in one batch with high probability. Thorough securityanalysis and prototype experiments on Amazon EC2 demonstrate the validity and practicality of our proposed design.

Index Terms—Confidential data, computation outsourcing, system of linear equations, cloud computing.

F

1 INTRODUCTION

Cloud Computing provides on-demand network accessto a shared pool of configurable computing resourcesthat can be rapidly deployed with great efficiency andminimal management overhead [2]. Under such un-precedented computing model, customers with compu-tationally weak devices are no longer limited by theslow processing speed, memory, and other hardwareconstraints, but can enjoy the literally unlimited com-puting resources in the cloud through the convenient yetflexible pay-per-use manners [3].

Despite the tremendous benefits, the fact that cus-tomers and cloud are not necessarily in the sametrusted domain brings many security concerns and chal-lenges towards this promising computation outsourcingmodel [4]. Firstly, customer’s data that are processed andgenerated during the computation in cloud are oftensensitive in nature, such as business financial records,proprietary research data, and personally identifiablehealth information etc [5]. While applying ordinary en-cryption techniques to these sensitive information beforeoutsourcing could be one way to combat the securityconcern, it also makes the task of computation over en-crypted data in general a very difficult problem [6]. Sec-ondly, since the operational details inside the cloud arenot transparent enough to customers [5], no guarantee is

• Cong Wang, Kui Ren, Jia Wang, and Karthik Mahendra Raje Urs areall with the Department of Electrical and Computer Engineering, IllinoisInstitute of Technology, Chicago, IL 60616. E-mail: {cong, kren, jwang,karthik}@ece.iit.edu.

A preliminary version [1] of this paper is to be presented at the 31thInternational Conference on Distributed Computing Systems (ICDCS 2011).

provided on the quality of the computed results from thecloud. For example, for computations demanding a largeamount of resources, there are huge financial incentivesfor the cloud server to be “lazy” if the customer cannottell the correctness of the answer. Besides, possible soft-ware/hardware malfunctions and/or outsider attacksmight also affect the quality of the computed results.Thus, we argue that the cloud is intrinsically not securefrom the viewpoint of customers. Without providing amechanism for secure computation outsourcing, i.e., toprotect the sensitive input and output information ofthe outsourced computing needs and to validate theintegrity of the computation result, it would be hardto expect cloud customers to turn over control of theircomputing needs from local machines to cloud solelybased on its economic savings and resource flexibility.

Focusing on the engineering and scientific computingproblems, this paper investigates secure outsourcing forwidely applicable large-scale systems of linear equations(LE), which are among the most popular algorithmicand computational tools in various engineering disci-plines that analyze and optimize real-world systems.By “large”, we mean the storage requirements of thesystem coefficient matrix may easily exceed the availablememory of the customer’s computing device [7], likea modern portable laptop. In practice, there are manyreal world problems that would lead to very large-scale and even dense systems of linear equations withup to hundreds of thousands [8], [9] or a few millionunknowns [10]. For example, a typical double-precision50, 000×50, 000 system matrix resulted from electromag-netic application would easily occupy up to 20 GBytesstorage space, seriously challenging the computational

2

power of these low-end computing devices. Because theexecution time of a computer program depends not onlyon the number of operations it must execute, but alsoon the location of the data in the memory hierarchy ofthe computer [7], solving such large-scale problems oncustomer’s weak computing devices can be practicallyimpossible, due to the inevitably involved huge IO cost.Therefore, resorting to cloud for such computation inten-sive tasks can be arguably the only choice for customerswith weak computational power, especially when thesolution is demanded in a timely fashion.

It is worth noting that in the literature, several cryp-tographic protocols for solving various core problemsin linear algebra, including the systems of linear equa-tions [11]–[16] have already been proposed from thesecure multiparty computation (SMC) community. How-ever, these approaches are in general ill-suited in thecontext of computation outsourcing model with largeproblem size. First and foremost, all these work devel-oped under SMC model do not address the asymmetryamong the computational power possessed by cloud andthe customer, i.e., they all impose each involved partycomparable computation burdens, which in this paperour design specifically intends to avoid (otherwise, thereis no point for the customer seeking help from cloud).Secondly, the framework of SMC usually assumes eachinvolved party knows a portion of the problem inputinformation (e.g., each party knows only a share of thecoefficient matrix). This assumption is not true any morein our model, where the information leakage on both theinput and output of the LE problem to the cloud shouldbe strictly forbidden. Last but not the least, almost allthese solutions are focusing on the traditional directmethod for jointly solving the LE, like the joint Gaussianelimination method in [12], [15], or the secure matrixinversion method in [13]. While working well for smallsize problems, these approaches in general do not derivepractically acceptable solution time for large-scale LE,due to the expensive cubic-time computational burdenfor matrix-matrix operations and the huge IO cost oncustomer’s weak devices (detailed discussions in Section8).

The analysis from existing approaches and the com-putational practicality motivates us to design securemechanism of outsourcing LE via a completely differentapproach — iterative method, where the solution isextracted via finding successive approximations to thesolution until the required accuracy is obtained (to beintroduced later in Section 2.3.1). Compared to directmethod, iterative method only demands relatively sim-pler matrix-vector operations (with O(n2) computationalcost), which is much easier to implement in practiceand widely adopted for large-scale LE [8], [10], [17].To the best of our knowledge, no existing work hasever successfully tackled secure protocols for iterativemethods on solving large-scale systems of LE in the com-putation outsourcing model, and we give the first studyin this paper. Specifically, our mechanism utilizes the ad-

ditive homomorphic property of public key encryptionscheme, like the one proposed by Paillier [18], and allowscustomers with weak computational devices, startingfrom an initial guess, to securely harness the cloud forfinding successive approximations to the solution in aprivacy-preserving and cheating-resilient manner. For alinear system with n×n coefficient matrix, each iterativealgorithm execution of the proposed mechanism onlyincurs O(n) local computational burden on customer’sweak device and asymptotically eliminates the expensiveIO cost, i.e., no unrealistic memory demands, whichon the other hand may significantly downgrade theperformance if the problem is solved by the customeralone. To detect any attempted result corruption bycloud server, we also propose a very efficient cheatingdetection mechanism to effectively verify in one batchof all the computation results by the cloud server fromprevious algorithm iterations with high probability. Ourcontributions can be summarized as follows:

1) For the first time, we formulate the problem in thecomputation outsourcing model for securely solv-ing large-scale systems of LE via iterative methods,and provide the secure mechanism design whichfulfills input/output privacy, cheating resilience,and efficiency.

2) Our mechanism brings computational savings as itonly incurs O(n) local computation burden for thecustomer within each algorithm iteration and de-mands no unrealistic IO cost, while solving large-scale LE locally usually demands more than O(n2)computation cost in terms of both time and mem-ory requirements [10].

3) We explore the algebraic property of matrix-vectormultiplication to design a batch result verificationmechanism, which allows customers to verify allanswers computed by cloud from previous itera-tions in one batch, and further ensures both theefficiency advantage and the robustness of the de-sign.

4) The experiment on Amazon EC2 [19] shows ourmechanism can help customers achieve up to2.43 × savings when the sizes of the LE problemsare relatively small (n ≤ 50, 000). Better efficiencygain can be easily anticipated when n goes to largersize. In particular, when n increases to 500, 000,the anticipated computational savings for customercan be up to 26.09 ×.

The rest of the paper is organized as follows. Section2 introduces the system and threat model, and ourdesign goals. Then we provide the detailed mechanismdescription in Section 4 and Section 5, both with securityanalysis. Section 6 gives the discussions on the practicalimplementation issues. Section 7 gives the performanceevaluation, followed by Section 8 which overviews therelated work. Finally, Section 9 gives the concludingremark of the whole paper.

3

Customer

LE problem Φ

Cloud

Servers

Encrypt LE problem ΦK

Answer to Φ Verify &

Decrypt

Answer to ΦK

Securely Harnessing the

Computation Power

Fig. 1: Architecture of secure outsourcing problems oflarge-scale systems of linear equations in Cloud Com-puting

2 PROBLEM STATEMENT

2.1 System and Threat Model

We consider a computation outsourcing architecture in-volving two different entities, as illustrated in Fig. 1: thecloud customer, who wants to solve some computation-ally expensive LE problems with his low-end devices;the cloud server (CS), which has significant computationresources to provide utility computing services.

The customer has a large-scale system of LE problemAx = b, denoted as Φ = (A,b), to be solved. However,due to the lack of computing resources, like processingpower, memory, and storage etc., he cannot carry outsuch expensive (O(nρ)(2 < ρ ≤ 3)) computation locally.Thus, the customer resorts to cloud server (CS) forsolving the LE computation and leverages its computa-tion power in a pay-per-use manner. Instead of directlysending original problem Φ, the customer first uses asecret key K to map Φ into some encrypted versionΦK . Then, based on ΦK , the customer starts the com-putation outsourcing protocol with CS, and harnessesthe powerful resources of cloud in a privacy-preservingmanner. The CS is expected to help the customer findingthe answer of ΦK , but supposed to learn as little aspossible on the sensitive information contained in Φ.After receiving the solution of encrypted problem ΦK ,the customer should be able to first verify the answer. Ifit’s correct, he then uses the secret K to map the outputinto the desired answer for the original problem Φ.

As later we shall see in the proposed model, thecustomer still needs to perform a one-time setup phaseof encrypting the coefficient matrix with relatively costlyO(n2) computation.1 But it is important to stress that thisprocess can be performed under a trusted environmentwhere the weak customer with no sufficient computa-tional power outsources it to a trusted party. (Similartreatments have been utilized in [20]). The motivating ex-ample can be a military application where the customerhas this one-time encryption process executed insidethe military base by a trusted server, and then goes offinto the field with access only to untrusted CS. Another

1. Since the encryption on each element of the matrix coefficient areindependent, the whole one-time operation can be easily parallelizedby using multithreading technique on the modern multi-core systems.For example, enabling double threading on a six core system couldeasily speedup the operation efficiency with a factor of 12.

example can be the customer has the system modelingcoefficient matrix A encrypted on his company’s work-station, and then uses his portable device outside whilestill hoping to make timely decisions (derive solutionsxi) based on different observation bi in the field, fori = 1, 2, . . . , s. (Further discussion on amortization ofsuch one-time cost is given in Section 6.2.) Thus, to makethe rest of the paper easier to catch, we assume that CS isalready in possession of the encrypted coefficient matrix,and the customer who knows the decryption key hopesto securely harness the cloud for on-demand computingoutsourcing needs, i.e., solving LE problems {Ax = bi}.

The security threats faced by the computation modelprimarily come from the malicious behavior of CS,which may behave beyond “honest-but-curious” modelas assumed by some previous works on cloud datasecurity (e.g., [21]–[23]). It is either because CS intendsto do so or because it can be attacked/compromised. Inaddition to be persistently interested in analyzing theencrypted input sent by the customer and the encryptedoutput produced by the computation to learn the sen-sitive information, CS can also behave unfaithfully orintentionally sabotage the computation, e.g. to lie aboutthe result to save the computing resources, while hopingnot to be caught at the same time.

Finally we assume the communication channels be-tween cloud server and the customer is authenticatedand reliable, which can be achieved in practice with littleoverhead.

2.2 Design Goals

To enable secure and practical outsourcing of LE un-der the aforementioned model, our mechanism designshould achieve the following security and performanceguarantees.• Input/output Privacy: No sensitive information

from the customer’s private data can be derived bythe cloud server during performing the LE compu-tation.

• Robust Cheating Detection: Output from faithfulcloud server must be decrypted and verified suc-cessfully by the customer. No output from cheatingcloud server can pass the verification by the cus-tomer with non-negligible probability.

• Efficiency: The local computation done by the cus-tomer should be substantially less than solving theoriginal LE on his own. Here the computation bur-den is measured in terms of both time cost andmemory requirements.

2.3 Preliminaries and Notations

2.3.1 Iterative MethodIn many engineering computing and industrial applica-tions, iterative method has been widely used in practicefor solving large-scale systems of LE [8], and sometimesis the mandatory choice [17] over direct method due to

4

its ease of implementation and relatively less compu-tational power consumption, including the memory andstorage IO requirement [10]. We now review some basicson the general form of stationary iterative methods forsolving LE problems. A system of linear equations canbe written as

Ax = b, (1)

where x is the n× 1 vector of unknowns, A is an n× n(non-singular) coefficient matrix and b is an n× 1 right-hand side vector (so called constant terms). Most itera-tive methods involve passing from one iteration to thenext by modifying a few components of some approxi-mate vector solution at a time until the required accuracyis obtained. Without loss of generality, we focus on Jacobiiteration [17] here and throughout the paper presentationfor its simplicity. Though extensions to other stationaryiterative methods can be straightforward, we don’t studythem in the current work.

We begin with the decomposition: A = D + R, whereD is the diagonal component, and R is the re-maining matrix. Then the Eq. (1) can be writtenas Ax = (D + R)x = b, and finally reorganized as:x = −D−1 ·R · x + D−1 · b. According to the Jacobimethod, we can use an iterative technique to solvethe left hand side of this expression for x(k+1), usingprevious value for x(k) on the right hand side. If we de-note T = −D−1 ·R and c = D−1 · b, the above iterativeequations can be simply represented as

x(k+1) = T · x(k) + c. (2)

The convergence is not always guaranteed for all matri-ces, but it is the case for a large body of LE problemsderived from many real world applications [17].

2.3.2 Homomorphic Encryption

Our construction utilizes an efficient semantically-secureencryption scheme with additive homomorphic prop-erty. Given two integers x1 and x2, we have Enc(x1 +x2) = Enc(x1)∗Enc(x2), and also Enc(x1∗x2) = Enc(x1)x2 .In our implementation we adopt the one presented byPaillier in [18]. The Paillier cryptosystem is a publickey cryptosystem. Similar to RSA, the public key is N ,which is the product of two large primes p and q. Theplaintext space is ZN , while ciphertexts are representedas elements of group Z∗N2 . The encryption of messagex ∈ ZN is done by randomly choosing r ∈ Z∗N2 andcomputing Enc(x) = gx · rN mod N2, where g is anelement from Z∗N2 with order multiple of N .

For a vector x = (x1, x2, . . . , xn)T ∈ (ZN )n, we useEnc(x) to denote the coordinate-wise encryption of x:Enc(x) = (Enc(x1),Enc(x2), . . . ,Enc(xn))T . Similarly, forsome n × n matrix T, where each of the componentT[i, j] in T is from ZN , we denote the component-wiseencryption of T as Enc(T), and we have Enc(T)[i, j] =Enc(T[i, j]).

3 THE FRAMEWORK AND BASIC SOLUTIONS

In the Introduction we motivated the need for securelyharnessing the cloud for solving large-scale LE prob-lems. In this section, we start from the framework forour proposed mechanism design, and provide a basicoutsourcing solution based on direct method. The basicsolution fulfills the input/output privacy defined inSection 2.2, but does not meet the efficiency requirement.The analysis of this basic solution gives insights andmotivations on our main mechanism design based oniterative methods.

3.1 The General FrameworksTo make the following presentation easier to follow, wefirst give the general high level description of the frame-work of the proposed mechanism, which consists ofthree phases: (ProbTransform, ProbSolve, ResultVerify).• ProbTransform. In this phase, cloud customer would

initialize a randomized key generation algorithm andprepare the LE problem into some encrypted form ΦKvia key K for phase ProbSolve. Transformation and/orencryption operations will be needed when necessary.

• ProbSolve. In this phase, cloud customer would usethe encrypted form ΦK of LE to start the computationoutsourcing process. In case of using iterative methods,the protocol ends when the solution within the requiredaccuracy is found.

• ResultVerify. In this phase, the cloud customer wouldverify the encrypted result produced from cloud server,using the randomized secret key K. A correct output xto the problem is produced by decrypting the encryptedoutput. When the validation fails, the customer outputs⊥, indicating the cloud server was cheating.

Note that the proposed framework suits for secureoutsourcing mechanisms based on both direct methodand iterative method. In the following, we first give abasic direct method based mechanism design.

3.2 Basic Direct Method based MechanismsWe study in this subsection a straight-forward approachfor encrypting the problem for direct solvers, and showthat the local computation cost based on these techniquesalong may result in an unsatisfactory mechanism fromthe efficiency gain perspective.

Specifically, in the ProbTransform phase, the customerpicks a random vector r ∈ Rn as his secret keying mate-rial. Then he rewrites Eq. (1) as A(x + r) = b + Ar. Lety = x + r and b′ = b + Ar, we have Ay = b′. To hidethe coefficient matrix A, the customer would select arandom invertible matrix Q that has the same dimensionas A. Left-multiplying Q to both sides of Ay = b′ wouldgive us

A′y = b′′, (3)

where A′ = QA and b′′ = Q(b + Ar). Clearly, as Q andr are chosen randomly and kept as secret, cloud has noway to know (A,b,x), except the dimension of x.

5

The customer can then start the ProbSolve phase byoutsourcing ΦK = (A′,b′′) to the cloud, who solves ΦKand sends back answer y. Once the correctness of y isverified, the customer can derive x via x = y − r for theoriginal problem Φ = (A,b).Remark. While faithfully achieving the input/outputprivacy, the above approach is not attractive for the fol-lowing two reasons: 1) The local problem transformationcost for matrix multiplication QA is O(n3), which iscomparable to the cost of solving Ax = b [25]. Consid-ering the extra cost of ResultVerify, the discussion ofwhich we intentionally defer to a later Section 5, thereis no guaranteed computational saving for the customerin this basic mechanism. 2) The local cubic time cost canbecome prohibitively expensive when n goes large to theorders of hundreds of thousand. Besides, it violates ourassumption in Section 2 that the customer cannot carryout expensive O(nρ)(2 < ρ ≤ 3) computation locally.

In the recent literature, Atallah et al. has proposedwork for secure outsourcing matrix multiplication usingonly O(n2) local complexity. However, in practice thosework can hardly be applied to our case of calculatingQ ·A for Eq. (3), especially for large n. The reasonis that in their work, either non-collusion servers arerequired [26] or scalar operations are expanded to poly-nomials and thus incur huge communication and com-putation overhead [27]. Both assumptions are difficultto be met in practice. (See detailed discussion on this atSection 8).

4 THE PROPOSED SOLUTION

The above observation and discussion shows that directmethod based approach might not be a good optionfor resource-limited customers for secure outsourcinglarge-scale LE with computational savings in mind. Thismotivates us to design secure outsourcing mechanismusing iterative method. To better facilitate the iterativemethod based mechanism design to be explored later, wefirst make some general but non-stringent assumptionsabout the system as follows: 1) we assume the systemcoefficient matrix A is a strictly diagonally dominantmatrix2. It helps ensure the non-singularity of A and theconvergence of the iterative method that is to be detailedlater. Note that this is not a stringent requirement, asmany real-world formulated LE problems satisfy thisassumption, such as the statistical calculations [24], orthe radar cross-section calculations [8] etc. 2) Althoughproper preconditioning techniques (e.g. see [7]–[10], [17]for details) on the coefficient matrix A can significantlyimprove the performance of any iterative method, wedo not study the cost of these techniques in this paper.As we focus on the security design only, we assumethe coefficient matrix A already ensures fast enoughconvergence behavior, i.e., the number of iterations L�n. 3) We assume the matrix A is first transformed to

2. We focus on general dense matrices in this paper.

Algorithm 1: Problem Transformation PhaseData: original problem Φ = (A,b)Result: transformed problem as shown in Eq. (5)begin

1 pick random r ∈ Rn;2 compute b′ = b + Ar, and c′ = D−1 · b′;3 replace tuple (x, c) in Eq. (2) with (y = x + r, c′);

return transformed problem as Eq. (5);

T = D−1 ·R, where A = D + R as in Eq. (2), andthen stored in cloud in its encrypted form Enc(T) viahomomorphic encryption introduced in Section 2.3.2. Asstated in our system model, this one-time setup phase isdone before ProbTransform phase by some trusted work-station under different application scenarios. Hereinafter,we may interchangeably use the two forms of coefficientmatrix A or T without further notice.

For ease of presentation, we consider a semi-honestcloud server here, and defer the mechanisms of cheatingdetection to a later section.

4.1 Problem TransformationThe customer who has coefficient vector b and seekssolution x satisfying Ax = b cannot directly starts theProbSolve with cloud, since such interaction may exposethe private information on final result x. Thus, we stillneed a transformation technique to allow customer toproperly hide such information first. Similar to the basicmechanism in Section 3.2, in the ProbTransform phase,the customer picks a random vector r ∈ Rn as his secretkeying material, and rewrites Eq. (1) as the new LEproblem

Ay = b′, (4)

where y = x + r and b′ = b + Ar. Clearly, as long asthe relationship x = y − r holds, then for any solutionx satisfying Eq. (1), we can find a solution y satisfyingthe Eq. (4), and vice versa. Thus, the solution x to Eq. (1)can be found by solving a transformed LE problem inEq. (4). At this point, both the output x and input tupleb have been perfectly hidden via random vector r. Sinceour goal is to start the iterative process with the cloudfor solving LE, we can reformulate Eq. (4) into the moreconvenient iterative form as Eq. (2), which we rewrite asfollows:

y(k+1) = T · y(k) + c′, (5)

where T = −D−1 ·R, c′ = D−1 · b′, and A = D + R.As a result, the problem input Φ = (A,b) that needs

to be protected is changed to tuple ΦK = (T, c′), whereT has already been encrypted and stored as Enc(T)at cloud, and c′ is just a randomly masked version ofb via random n × 1 vector r. The problem output xis also masked to y = x + r. Now we are ready forthe next phase of ProbSolve. The whole procedure ofProbTransform is summarized in Algorithm 1.

6

Remark. This problem transformation on the local cus-tomer side only requires two matrix-vector multiplica-tions: one for b′ = b + Ar and one for c′ = D−1 · b′.When n goes large, expensive IO cost at customer devicecould downgrade the performance of such operations.However, as soon we shall see, such one-time problemtransformation cost can be easily amortized throughoutthe overall efficient iterative algorithm executions, espe-cially when we deal with honest but curious adversaries.It is also worth noting that this transformation does notneed to modify the matrix of A (or T), which gives usadvantage of reusing. Specifically, the customers withdifferent constant terms bi can utilize this transformationtechnique by choosing a different r and then harness thecloud for solving different LE problems {Ax = bi}, asseen in Section 6.2.

4.2 The Iterative Problem SolvingAfter the problem transformation step, now we are readyto describe the phase of ProbSolve. The purpose ofour protocol is to let the customer securely harnessthe cloud for the most expensive computation, i.e., thematrix-vector multiplication T · y(k) in Eq. (5) for eachalgorithm iteration, k = 1, 2, . . . , L. We show how it canbe realized with the help of the additive homomorphicencryption introduced in Section 2.3.2. Since it is aniterative computing process, we only describe the veryfirst round of the process as follows. We leave theconvergence analysis and other input/output privacyguarantee in later subsections. For ease of presentation,in what follows we assume without loss of generalitythat our main protocol of solving LE works over integers.All arithmetic is modular with respect to the modulusN of the homomorphic encryption, and the modulus islarge enough to contain the answer. The reason whythis is a reasonable assumption and how our designshandle noninteger real numbers is given in more detailin Section 6 of practical considerations.1) For the very first iteration, the customer starts theinitial guess on the vector y(0) = (y

(0)1 , y

(0)2 , . . . , y

(0)n )T ,

and then sends it to the cloud.2) The cloud server, in possession of the encryptedmatrix Enc(T), computes the value Enc(T · y(0)) by usingthe homomorphic property of the encryption:

Enc(T · y(0))[i] = Enc(n∑j=1

T[i, j] · y(0)j )

=

n∏j=1

Enc(T[i, j])y(0)j , (6)

for i = 1, . . . , n, and sends Enc(T · y(0)) to customer.3) After receiving Enc(T · y(0)), the customer decryptsand gets T · y(0) using his private key. He then updatesthe next approximation y(1) = T · y(0) + c′ via Eq. (5).

For the k-th iteration, it follows similarly that thecustomer provides the k-th approximation of y(k) to the

Algorithm 2: Iterative Problem Solving PhaseData: transformed problem with input c′ and

Enc(T)Result: solution x to the original problem

Φ = (A,b)% L: maximum number of iterations to beperformed;% ε: measurement of convergence point;begin

1 Customer picks y(0) ∈ (ZN )n;for (k ← 0 to L) do

2 Customer sends y(k) to cloud;3 Cloud computes Enc(Ty(k)) via Eq. (6);4 Customer decrypts Ty(k) via his private key;

if ||y(k) − y(k+1)|| ≤ ε then5 break with convergence point y(k+1);

6 return x = y(k+1) − r;

cloud, k = 1, 2, . . . , L. The cloud computes Enc(T · y(k))for the customer for the next update of y(k+1). Theprotocol execution continues until the result converges,as shown in Algorithm 2.Remark. For the k-th iteration, the dominant customer’slocal computation overhead is only to decrypt the vectorof Enc(T · y(k)), which takes O(n) complexity, and ingeneral does not require expensive IO cost, as demon-strated in our experiment in Section 7. This is less thanthe O(n2) cost demanded by the matrix-vector multi-plication T · y(k) of Eq. (5) in terms of both time andmemory requirements. Note that the decryption com-putation is generally quite expensive compared to thefloating point arithmetic operation. Thus, the theoreticalcomputation efficiency gain (in terms of both time andmemory requirements) can only be exhibited when theproblem size n goes very large. However, this is nota problem in our case, since we are specifically usingiterative methods to securely solve large-scale LE. Laterin Section 7, we will show some performance results anddiscuss the possible selections of deciding the propersize n for the proposed problem. Also note that thecommunication overhead between the customer and thecloud is only two vectors of size n for each iteration,which is reasonably efficient.

4.3 Convergence AnalysisWhen dealing with iterative methods, it is a must todetermine whether and when the iteration will converge.Here we utilize the general convergence result from [17]:For LE problem Ax = b, if A is a strictly diagonallydominant or an irreducibly diagonally dominant matrix,then the associated Jacobi iterations converge for anyinitial guess of x0.

Since in our model the coefficient matrix A is readilydiagonally dominant, to determine whether to terminate

7

the ProbSolve phase, the customer only needs to test if

||y(k) − y(k+1)|| ≤ ε, (7)

for some small enough ε > 0. And the termination pointy(k+1) will help get the final result x via x = y(k+1) − r.The local computation cost for each iteration is still O(n).

4.4 Input/output Privacy Analysis4.4.1 Output Privacy AnalysisFrom above protocol instantiation, we can see thatthroughout the whole process, the cloud server onlysees the plaintext of y(k), the encrypted version of ma-trix Enc(T), and encrypted vectors Enc(T · y(k)), k =1, 2, . . . , L. Since y is a blinded version of original so-lution x, it is safe to send y to the cloud in plaintext. Noinformation of x would be leaked as long as r is keptsecret by the customer.

4.4.2 Input Privacy AnalysisWhile the output is protected perfectly, it is worth notingthat some knowledge about the input tuple Φk = (T, c′)could be implicitly leaked through the protocol exe-cution itself. The reason is as follows: for each twoconsecutive iterations of the protocol, namely, the k-thand the (k + 1)-th, the cloud server sees actually theplaintext of both y(k) and y(k+1). Thus, a “clever” cloudserver could initiate a system of linear equations via Eq.(5) and attempts to learn the unknown components ofT and c′. More specifically, for the total L iterations,the cloud server could establish a series of (L − 1) × nequations from y(k), k = 0, 1, . . . , L−1,3 while hoping tosolve n2 + n unknowns of T and c′.

However, as we have assumed in Section 3.1 that var-ious preconditioning techniques can ensure fast enoughconvergence behavior, we have the number of iterationsL� n. (In fact, if L is close or even larger than n, therewould be no advantage of using iterative method overdirect method at all.) As a result, from the (L − 1) × nequations, the n2 + n components of T and c is largelyunderdetermined and cannot be exactly determined byany means. Thus, as long as the cloud server has noprevious knowledge of the coefficient matrix A, we statethat such bounded information leakage from (L− 1)×nequations can be negligible, especially when the size ofproblem n goes very large.

In fact, we can further enhance the guarantee of inputprivacy by introducing a random scaling factor ak ∈ ZNfor each iteration to break the linkability of two con-secutive iterations of the protocol. Specifically, insteadof sending y(k) directly to the cloud server for the k-thiteration, the customer sends ak · y(k) for each iterationof the ProbSolve. When the cloud server sends back theencrypted value Enc(ak ·Ty(k)), the customer just simplydecrypts the vector of ak ·Ty(k+1), divides each compo-nent with ak, and then updates the next approximation

3. Note that y(L) as the final answer is not transmitted to the cloudserver.

y(k+1) via Eq. (5). Similarly, for the next iteration anotherrandom scaling factor ak+1 is multiplied to y(k+1) beforesent to the cloud server.Remark. With the random scaling factor ak, it is notpossible to derive the original value y(k) via aky

(k).Thus, the cloud server can no longer directly establishlinear equations from received aky

(k) and ak+1y(k+1),

but a series of non-linear equations with extra randomunknowns a1, a2, . . . , aL. We should note that while thismethod further enhances the guarantee of input privacyby bringing extra randomness and nonlinearity of thesystem equations, it does not incurs any expensive op-eration, making the customer’s local computation coststill O(n) within each iteration.

5 CHEATING DETECTION

Till now, the proposed protocol works only under theassumption of honest but curious cloud server. However,such semi-trusted model is not strong enough to capturethe adversary behaviors in the real world. In many cases,an unfaithful cloud server in reality could sabotage theprotocol execution by either being lazy or intentionallycorrupting the computation result, while hoping not tobe detected. In the following, we propose to designresult verification methods to handle these two maliciousbehaviors. Our goal is to verify the correctness of thesolution by using as few as possible expensive matrix-vector multiplication operations.

For easy notation purposes, we denote z(k) = T · y(k)

as the expected correct responses from cloud server, andz(k) = T · y(k) as the actual received value from cloudserver, where k = 1, 2, . . . ,L. Here we also assume L ≤L, meaning the ResultVerify phase is initiated within atmost L iterations.

5.1 Dealing with Lazy AdversaryWe first consider detecting the laziness of cloud server.Since computing the addition and multiplication overencrypted domain could cost a lot of computationalpower, the cloud server might not be willing to com-mit service-level-agreed computing resources in orderto save cost. More severely, for the k-th iteration, theadversary could simply reply the result z(k−1) of theprevious (k − 1)-th iteration without computation.

As a result, the customer who uses z(k−1) to updatefor the next y(k+1) will get the result y(k+1) = y(k).Consequently, he may be incorrectly led to believe thesolution of equation Ay = b′ is found. Thus, for themalicious adversary, only checking the Eq. (7) is notsufficient to convince the customer that the solution hasconverged. According to Eq. (4) one further step has tobe executed as

||Ay(k+1) − b′|| ≤ ε. (8)

Remark This checking equation incurs the local costof O(n2) for customer. While potentially expensive for

8

large size of n, we should note that it does not haveto be executed within every iteration. It only needs tobe tested after the test on y(k) and y(k+1) via Eq. (7) ispassed. If Eq. (7) is not passed, it means y(k+1) is not theconvergence point yet. On the other hand, if Eq. (7) issuccessfully passed, we can then initiate the test of Eq.(8). If Eq. (8) holds, we say the final solution is found,which is x = y(k+1) − r. If it doesn’t, we can tell thatthe cloud server is cheating (being lazy). In either case,this matrix-vector multiplication of Eq. (8) only needsto be executed at most once throughout the protocolexecution.

5.2 Dealing with Truly Malicious Adversary

While a lazy adversary only sends previous result as thecurrent one, a truly malicious adversary can sabotage thewhole protocol execution by returning arbitrary answers.For example, the malicious cloud server could computeEq. (6) via arbitrary vectors y(k) other than customer’sy(k). In the worst case, it would make the protocolnever converge, wasting the resources of the customer.Thus, we must design an efficient and effective methodto detect such malicious behavior, so as to ensure theresult quality. The straightforward way would be to redothe matrix-vector multiplication T · y(k) and check if itequals to the received z(k) for each iteration k. Thisis not appealing since it consumes equivalent amountof resources in comparison to that of computing theresults directly. Below we utilize the algebraic propertyof matrix-vector multiplication and design a method totest the correctness of all received answers z(k) = T·y(k),k = 1, 2, . . . ,L in only one batch, i.e., using only onematrix-vector multiplication.

Suppose after L iterations, the solution still does notconverge. The customer can initiate a ResultVerify phaseby randomly selecting L numbers, α1, α2, . . . , αL fromB ⊂ ZN , where each αk is of l-bit length and l < logN .He then computes the linear combination θ over they(k)’s, which he has provided in the previous k itera-tions, k = 1, 2, . . . ,L: θ =

∑Lk=1 αk · y(k). Next, to test

the correctness of all the intermediate results, {z(k) =T · y(k)}, k = 1, 2, . . . ,L, received from cloud server, thecustomer simply checks if the following equation holds:

T · θ ?=

L∑k=1

αk · z(k). (9)

The above equation can be elaborated as follows:

T · θ = T ·L∑k=1

αk · y(k)

=

L∑k=1

αk ·T · y(k) =

L∑k=1

αk · z(k).

Since each αk is chosen randomly from B = {0, 1}l ⊂ZN , we have the following theorem capturing the

correctness and soundness of the cheating detectionmethod:

Theorem 1: The result verification Eq. (9) holds if andonly if z(k) = T · y(k) for all k = 1, 2, . . . ,L, with errorprobability at most 2−l.

Proof: It is straightforward that the Eq. (9) alwaysholds when the received values of each iteration arecomputed correctly. We now prove the soundness. Letz(1), z(2), . . . , z(L) denote the actual received results fromcloud server, among which there exist some unfaithfullycomputed results. And let z(1), z(2), . . . , z(L) denote thecorrect result of the matrix-vector multiplication of exe-cuted iteration.

Since the results are incorrect, there exist at least somek such that z(k) 6= z(k), for k = 1, 2, . . . ,L. This inequalityalso indicates that at least one of the n components ofvector z(k) and z(k) is not equal to each other. Withoutloss of generality, we assume z(k)[1] 6= z(k)[1].

Now suppose the test Eq. (9) accepts the incorrectresults on a particular choice of α1, α2, . . . , αL. From thecorrectness of Eq. (9) we must have

L∑k=1

αk · z(k) =

L∑k=1

αk · z(k) (10)

Consider the 1st row of this vector equation, we have

L∑k=1

αk · z(k)[1] =

L∑k=1

αk · z(k)[1] (11)

Since z(k)[1] 6= z(k)[1], we can rearrange the aboveequation as

α1 = −(z(k)[1]− z(k)[1])−1 · (L∑k=2

αk · (z(k)[1]− z(k)[1])) (12)

This means for any fixed choice of α2, α3, . . . , αL, thereis exactly one choice of α1 ∈ B = {0, 1}l ⊂ ZN(from Eq. (12)) such that Eq. (11) is true. Thus, if wefix α2, α3, . . . , αL and draw α1 randomly from B, theprobability that Eq. (11) holds is 2−l. The same is alsotrue if we draw α1, α2, . . . , αL independently at random.Note that we only consider the case where one of the ncomponents of vector z(k) and z(k) is not equal to eachother. In case there are λ > 1 components are different,the probability that eq. (11) holds would be 2−λ·l < 2−l.Therefore, the test eq. (9) holds for unfaithful results isat most 2−l.

Finally, we remark that some of our probability argu-ment from Eq. (12) is inspired by Bellare’s analysis oftheir fast batch verification for modular exponentiationand digital signature schemes [28].

Remark. It is easy to tell that the computation overheadof Eq. (9) is only bounded by one matrix-vector mul-tiplication of the left-hand-side of the equation (recallL ≤ L� n). The size of l is a tradeoff between efficiencyand security. While a conservative choice of high securityshould probably require l around 80 bit, for a fast check

9

a reasonable choice of 20 bits of l is also acceptable [29].Note that in practice this result verification does notneed to happen very frequently, because the propertyof batch verification ensures the quality of all previousreceived values {z(k) = T · y(k)}, k = 1, 2, . . . ,L. Thusthe customer can preset the threshold L as sufficientlylarge such that either he detects the unfaithful behaviorof cloud server or the program will converge soon afterL iterations. In the best case, Eq. (9) only needs to beinstantiated once. Combined with Eq. (8), we can seethat our method for cheating detection indeed achievesas few as possible expensive matrix-vector multiplicationoperation. As a result, the overall design asymptoticallyeliminates the expensive IO cost on customer throughoutthe successive approximation process for seeking thesolution as well as result verification.

6 PRACTICAL CONSIDERATIONSIn the previous section, we assumed that the proposedprotocol works well over integer values. Now we discussthese practical issues on how to adapt our proposedsolution to work over non-integer and even negativevalues, and justify that our solution is a reasonable one.In addition, we will also describe some specific appli-cation scenarios of our proposed mechanism where wecan amortize the one-time O(n2) setup cost by reusingthe coefficient matrix A. Finally, we show the utilizationof cost associativity of cloud computing can actuallyhelp us save much of protocol running time during theimplementation on cloud. These discussions also give usinsights of designing our experiments on the real cloudplatform in the next section.

6.1 Dealing with Non-Integer NumbersRemember in the Paillier cryptosystem, the messagespace is ZN : {0, 1, . . . , N − 1}. Thus, we have to dealwith issues of both negative numbers and real numbersin order to use the primitive correctly. We begin withthe handling of negative numbers. Since the homomor-phic property of Paillier cryptosystem is over moduloarithmetic, we can shift the negative numbers to theinterval of {(N − 1)/2 + 1, . . . , N − 1}, with −1 = N − 1mod N , while keeping non-negative numbers in theinterval of {0, 1, . . . , (N − 1)/2}. When the decryptedvalue yi > N/2, we shift yi back to the result yi − N .By doing so we limit the value of input to the intervalof (−N/2, (N −1)/2]. But since N is of 1024 bit length, itensures the interval is large enough for accommodatingall the computation to be performed in our scheme.

As for the real value y ∈ Rn, or yi ∈ R, we canintroduce some scaling factor (e.g., the multipliers of 10)to first scale up the value into integer value. Then afterthe protocol execution, we can scale the result back to theoriginal value by removing the scaling factor.4 More for-mally speaking, let 1/q be the selected multiplier, where

4. In the literature, using scaling factors to handle non-integer valuesis not new. We acknowledge that similar treatment can be found inmany previous work, e.g., [30]–[32], to list a few.

q is some small enough quantization factor. Then we canapproximate the value of yi ∈ R as yi = byi/qc ∈ ZN .This may results in some accuracy problem. But if weselect sufficiently thin quantization factor like q = 10−3,it may lose little accuracy. Obviously, the additive ho-momorphic property of Paillier encryption scheme stillholds, since

Dec(Enc(y1) · Enc(y2)) = y1 + y2 =y1 + y2

q. (13)

This allows cloud to perform an arbitrarily number ofsums among ciphertext without worrying the roundingerrors. Also, the property of plaintext multiplication alsoholds, since

Dec(Enc(y1)y2) = y1 · y2 =y1 · y2

q2. (14)

Here the presence of factor q2 from Eq. (14) indi-cates that the size of the encrypted value would growexponentially with the number of multiplications overplaintext. However, this is not an issue in our scheme,because the computation of Eq. (6) in the ProbSolvephase only needs one-time multiplication between eachscaled value pair T[i, j] and y

(k)j for j = 1, 2, . . . , n.

Consequently, after the customer decrypts the resultT · y(k) of Eq. (6), the compensation factor for the finalreal value of each component would be q2 (but not q).

6.2 Application over a Series of Equations{Ax = bi}

Lots of real world engineering computing problems canbe formulated directly or indirectly as systems of linearequations. Moreover, many of these problems would notonly demand the solution of a single system of linearequations Ax = b, but also require to solve a matrixequation multiple times for blocks of right-hand-sideconstant terms b1,b2, . . . ,bs. (See Chapter 1.4 from [10]and references therein for such example applications oflarge dense system of LE on electromagnetic problemsand others.) Our proposed mechanism can be appliedto these problem scenarios and explore the flexibility inreusing encrypted matrix Enc(T) in Eq. (5), which notonly amortizes the one-time cost on trusted workstationfor encrypting the coefficient matrix by a factor of s, butalso increases the usability of the proposed mechanism.

However, the encrypted matrix cannot be reused inunlimited times, since it would give cloud server ac-cumulated information to establish matrix equations tosolve for T. (see Section 4.4). Thus, for security pur-poses, the number s of reuse should never exceedsbn/Lc, where L denotes the expected maximum numberof iterations for solving one matrix equation Ax = bi,i = 1, 2, . . . , s. In this way, we can ensure the groupof equations on matrix T is always underdetermined(and thus have unlimited solutions) and guarantees theacceptable input privacy.

10

6.3 Utilizing the Cost Associativity of Cloud Com-puting

To better utilize the benefits of cloud, we propose toexplore the famous “cost associativity” of cloud com-puting, which was first suggested by [33], to speedupthe cloud sides computation, i.e., the Eq. (6) in ourscheme. Cost associativity states that using 1000 EC2instances [19] in the cloud for 1 hour costs the same asusing 1 instances for 1000 hours. This gives us advantageto parallelize the computation of Eq. (6), which can bedecomposed into subtasks running simultaneously onmultiple available EC2 instances.

Specifically, when we implement our scheme, we canfirst separate the encrypted matrix Enc(T) into t sub-matrices, each formed by n/t rows out of the n rowsof Enc(T). (we assume n can be divided exactly by there). Then instead of sending Enc(T) to a single EC2instance, we send each submatrix of Enc(T) to t differentEC2 instances in the cloud. When doing the computationof Eq. (6) for the k-th iteration, the customer can sendy(k) to this cluster of t EC2 instances and instruct eachof them to compute only partial result based on itspossessed submatrix, which would be a subvector withn/t components. After collecting all the subvectors, thecustomer simply decrypts and merges them to get theresult of T · y(k). In general, initiating t EC2 instancesin the cloud allows us to reduce the overall cloud sidecomputation time to 1/t, with no extra cost to customers.This will be validated in our experiment in the nextsection.

7 PERFORMANCE ANALYSIS

We implement our mechanisms using C language. Al-gorithms utilize the GNU Scientific Library, the GNUMultiple Precision Arithmetic Library, and the PaillierLibrary with modulus N of size 1024 bit. The customerside process is conducted on a laptop with Intel Core2 Duo processor running at 2.16 GHz, 1 GB RAM,and a 5400 RPM Western Digital 250 GB Serial ATAdrive with an 8MB buffer. The cloud side process isconducted on Amazon Elastic Computing Cloud (EC2)with High-Memory instance type [19]. The scaling fac-tor for real numbers is set to be 103. Our randomlygenerated diagonally-dominant test benchmark focuseson the large-scale problems only, where n ranges from5,000 to 50,000. The solutions are all converged within 50iterations. Since the computation dominates the runningtime as evidenced by our experiment, we ignore thecommunication latency between the customer and thecloud for this application. All results represent the meanof 10 trials. In order to handle large-scale matrix-vectoroperations, proper matrix splitting approaches are to beused, which also demonstrates how the IO cost couldsignificantly downgrade the performance if the wholecomputation is solely performed on the customer’s localmachine.

TABLE 1: Transformation cost for different size of prob-lems.

Benchmark ProbTransform cost# dimension storage size transformation time1 n = 5,000 200 MB 7.35 sec2 n = 8,000 512 MB 28.17 sec3 n = 1,0000 800 MB 47.61 sec4 n = 20,000 3.2 GB less than 4 mins5 n = 30,000 7.2 GB less than 7 mins6 n = 40,000 12.8 GB less than 13 mins7 n = 50,000 20 GB less than 23 mins

7.1 Problem Transformation Cost

We first summarize the cost for customer performingProbTransform. As shown in Section 4.1, the transfor-mation cost is dominated by the two matrix-vectormultiplication in Eq. (5). Note that when n goes large,the resulted matrix would be too large to be hold incustomer’s local machine memory. Thus, the matrix-vector multiplication cannot be performed in one step.Instead, the matrix has to be split into multiple subma-trices, and each time only a submatrix can be loadedin the memory for computing a portion of the finalresult. In our experiment with 1 GB RAM laptop, wesplit the matrix into submatrices with 200 MB each foreasy in-memory operation. The time results for differentproblem sizes are shown in Table 1. For the largestbenchmark size n = 50, 000, the problem transformationonly costs around 22 minutes on our laptop. Comparedto the baseline experiment where the customer solvesthe equation by himself (shown in the next section), suchcomputational burden should be considered practicallyacceptable and it can be easily amortized throughout theoverall iterative algorithm executions.

7.2 Local Computation Comparison for ProblemSolving

In our protocol by harnessing the computation power ofcloud, the dominant operation in each iteration for cus-tomer is only to perform n decryptions. If the customersolves the problem by himself, which is the baselineof our comparison, the dominant computation burdenwithin each iteration would be the matrix-vector mul-tiplication with the input size n2. We compare the twocomputation cost in table 2, where both timing resultsand estimated memory consumption for each singlealgorithm iteration are reported. To better present thetrend of the efficiency gain between the two experiments,the timing comparison results are also plotted in Fig. 2.

To have a fair comparison, again we have to considerthe memory requirements incurred by the two operation.In particular, when n goes large, the IO time has to betaken into consideration. Similar to the transformationcost test, in our baseline experiment, each matrix is splitinto submatrices with 200 MB each for easy in-memoryarithmetic operation. In this way, when performing the

11

0 1 2 3 4 5

x 104

0

100

200

300

400

500

600

700

Problem size n

Tim

e (s

econ

ds)

Our scheme with 1024 bit keyOur scheme with 768 bit keyThe comparison baseline

Fig. 2: Comparison of customer local computation costamong baseline and our scheme with different choicesof key length.

matrix-vector multiplication for a 50, 000×50, 000 matrixwith 20 GB space, at least 100 times expensive IO oper-ations for a 200 MB submatrix have to be performed,which significantly increases the total time cost in ourbaseline experiment, as shown in Fig. 2. On the otherhand, our proposed scheme only demands local n de-cryption operations, which does not have such highmemory demands. For 1024 bit key, holding the 50, 000×1 encrypted vector only needs 50, 000× 256Bytes < 13.0MB memory, which can be easily satisfied by modernportable computing devices. Thus, the total local com-putation cost simply goes linearly with the problem sizen. For completeness, we also conduct the experimentwith reduced key length of the Paillier cryptosystem.This is motivated by applications where only short termguarantees of secrecy (the coefficient matrix of A) maybe required. (See short key experiments in [32], [34] forexample.) Thus, if the customer decides a smaller keylength is acceptable, the total timing will be reduced.

We can see that the crossover point occurs aroundn = 46, 000 for a 1024 bit key and n = 20, 000 for a768 bit key in Fig. 2, where the trend of the efficiencygain among O(n) and O(n2) is also clearly shown. Incase of 768 bit key, when n = 50, 000, the customer’slocal computation cost in the baseline experiment wouldbe 2.43× more than the proposed scheme. Note thatn = 50, 000 is not an unreasonably large matrix. Manyreal world application, e.g. problems from electromag-netic community, could easily lead to a dense system oflinear equations with more than 200,000 unknowns [10].Though in this work we didn’t try problem size largerthan 50,000, the better efficiency gain for larger-scaleproblems can be easily anticipated (see Table 3) fromthe clear trend among O(n) and O(n2) shown in Fig.2. For example, when n = 500, 000, the anticipatedcomputational saving for customer can be up to 26. 09×.

7.3 Cloud Computation CostThe cloud side computation cost for each iterated al-gorithm execution is given in Table 4. The third andforth columns lists the cloud computation time whenthere is only one instance running. However, as statedpreviously in Section 6.3, we can utilize the cost associa-tivity of cloud computing to speedup the cloud servercomputation via task parallelization without introducingadditional cost to customers. Thus, the fifth and thesixth columns lists the estimated cloud computation timewhen multiple t Amazon EC2 instances are running si-multaneously. By configuring a proper choice of t = 100,even for the largest size of the problem n = 50, 000, thecloud side computation can be finished within around20 minutes for each round. Given the security propertyour mechanism has provided, such time cost should bedeemed reasonable in practice.

8 RELATED WORK

8.1 Secure Computation OutsourcingRecently, a general result of secure computation out-sourcing has been shown viable in theory [20], whichis based on Yao’s garbled circuits [35] and Gentry’s fullyhomomorphic encryption (FHE) scheme [36]. However,applying this general mechanism to our daily computa-tions would be far from practical, due to the extremelyhigh complexity of FHE operation and the pessimisticcircuit sizes that can hardly be handled in practice.Instead of outsourcing general functions, in the securitycommunity, Atallah et al. explore a list of customizedsolutions [26], [27], [37] for securely outsourcing specificcomputations. In [37], they give the first investigation ofsecure outsourcing of numerical and scientific computa-tion, including LE. Though a set of problem dependentdisguising techniques are proposed, they explicitly al-low private information leakage. Besides, the importantcase of result verification is not considered. In [26],Atallah et al. give a protocol design for secure matrixmultiplication outsourcing. The design is built uponthe assumption of two non-colluding servers and thusvulnerable to colluding attacks. Later on in [27], Atallahet al. give an improved protocol for secure outsourcingmatrix multiplications based on secret sharing, whichoutperforms their previous work [26] in terms of sin-gle server assumption and computation efficiency. Butthe drawback is that due to secret sharing technique,all scalar operations in original matrix multiplicationare expanded to polynomials, introducing significantcommunication overhead. Considering the case of theresult verification, the communication overhead must befurther doubled, due to the introducing of additionalpre-computed “random noise” matrices. In short, thesesolutions, although elegant, are still not efficient enoughfor immediate practical uses on large-scale problems,which we aim to address for the secure LE outsourcingin this paper. Very recently, Wang et al. [38] give thefirst study of secure outsourcing of linear programming

12

TABLE 2: Customer computation cost comparison among baseline experiment and the proposed mechanism forsingle algorithm iteration. The asymmetric speedup is the ratio of the time cost in the baseline over the time costin the proposed scheme, which captures the customer efficiency gain. The entry with “×” indicates the positiveefficiency gain is achieved.

Benchmark Baseline cost The proposed ProbSolving cost Asymmetric Speedup# dimension storage size time memory time (768-bit) time (1024-bit) memory 768-bit 1024-bit1 n = 5,000 200 MB 3.68 sec 200 MB 27.46 sec 62.81 sec 1.3 MB 0.13 0.062 n = 8,000 512 MB 14.09 sec 200 MB 43.94 sec 98.24 sec 2.0 MB 0.32 0.143 n = 10,000 800 MB 23.78 sec 200 MB 54.74 sec 122.72 sec 2.6 MB 0.43 0.194 n = 20,000 3.2 GB 113.65 sec 200 MB 115.32 sec 245.81 sec 5.1 MB 0.99 0.465 n = 30,000 7.2 GB 212.33 sec 200 MB 163.29 sec 368.12 sec 7.7 MB 1.3 × 0.586 n = 40,000 12.8 GB 389.76 sec 200 MB 217.20 sec 491.32 sec 10.2 MB 1.79 × 0.797 n = 50,000 20 GB 667.63 sec 200 MB 274.70 sec 612.87 sec 13.0 MB 2.43 × 1.09 ×

TABLE 3: The anticipated cost comparison among baseline experiment and the proposed mechanism for larger-scaleproblems where n > 50000. Under current experiment setting, only the estimated timing results for single algorithmiteration are reported.

Benchmark Baseline The proposed ProbSolving cost Asymmetric Speedup# size time cost time (768-bit) time (1024-bit) 768-bit key 1024-bit key1 n = 200,000 3.1 hours 18.2 mins 40.8 mins 10.28 × 4.60 ×2 n = 500,000 19.8 hours 45.6 mins 1.7 hours 26.09 × 11.65 ×3 n = 1,000,000 79.7 hours 1.5 hour 3.4 hours 52.44 × 23.41 ×

TABLE 4: Cloud side computation cost for different choices of keys and number of simultaneously running EC2instances t.

Benchmark Time cost with one instance Estimated time with instances t = 10

# size 768-bit key 1024-bit key 768-bit key 1024-bit key1 n = 5,000 12 mins 20 mins 1.2 mins 2 mins2 n = 8,000 30 mins 53 mins 3 mins 5.3 mins3 n = 10,000 48 mins 1.3 hours 4.8 mins 7.8 mins

Benchmark Time cost with one instance Estimated time with instances t = 100

# size 768-bit key 1024-bit key 768-bit key 1024-bit key4 n = 20,000 3.2 hours 5.4 hours 1.9 mins 3.2 mins5 n = 30,000 7.2 hours 12.3 hours 4.3 mins 7.4 mins6 n = 40,000 12.9 hours 20.7 hours 7.7 mins 12.4 mins7 n = 50,000 20.2 hours 34.4 hours 12.1 mins 20.6 mins

in cloud computing. Their solution is based on problemtransformation, and has the advantage of bringing cus-tomer savings without introducing substantial overheadon cloud. However, those techniques involve cubic-timecomputational burden matrix-matrix operations, whichthe weak customer in our case is not necessarily able tohandle for large-scale problems.

8.2 Secure Two-party Computation

Securely solving LE has also been studied under theframework of SMC [35]. As we discussed in Introduc-tion, work under SMC may not be well-suited for thecomputation outsourcing model, primarily because theygenerally do not address the computation asymmetryamong different parties. Besides, in SMC no single in-volved party knows all the problem input information,making result verification usually a difficult task. Butin our model, we can explicitly exploit the fact thatthe customer knows all input information and thus de-

sign efficient batch result verification mechanism. Underthe SMC setting, Cramer and Damgard et al. initiatethe study of secure protocols for solving various lin-ear algebra problems [11]. Their work is done in theinformation theoretic multi-party setting, and focus onachieving constant round complexity. Later on, someother theoretical work on this topic have been pro-posed [12]–[14], each improving the previous results interms of different round complexity and communicationcomplexity, respectively. Readers are referred to [14] forthe detailed comparison. Note that these work are alldeveloped under the theoretical SMC framework andmay not be directly applied to our settings of computa-tion outsourcing. Besides, even the most communicationefficient work such as [12], [13] focus only on the directmethod based solution, and thus are not necessarily ableto tackle the large-scale problem that we are handling inthis paper. Du et al. also [16] propose a scheme underSMC setting for solving LE based on secret sharing.

13

But they utilize the heavy oblivious transfer protocolwhich could incur large communication and computa-tion complexity of the two involved parties. Recently,Troncoso-Pastoriza et al. [15] give the first study oniterative methods for collaboratively solving systemsof linear equations under the SMC framework, whichis the closest related work that we are aware of inthe literature. However, their work can only supportvery limited number iterative algorithm execution andcannot support reasonably large-scale systems of linearequations due to the continuing growth of the ciphertextin each iteration, which is not the case in our design. Inaddition to the computation asymmetry issue mentionedpreviously, they only consider honest-but-curious modeland thus do not guarantee that the final solution iscorrect under the possible presence of truly maliciousadversary.

8.3 Computation Delegating and Cheating DetectionDetecting the unfaithful behaviors for computation out-sourcing is not an easy task, even without consider-ation of input/output privacy. Verifiable computationdelegation, where a computationally weak customercan verify the correctness of the delegated computationresults from a powerful but untrusted server withoutinvesting too much resources, has found great interestsin theoretical computer science community. Some recentgeneral result can be found in Goldwasser et al. [39]. Indistributed computing and targeting the specific compu-tation delegation of one-way function inversion, Golle etal. [40] propose to insert some pre-computed results (im-ages of “ringers”) along with the computation workloadto defeat untrusted (or lazy) workers. Szada et al. [41]extend the ringer scheme and propose methods to dealwith cheating detection of other classes of computationoutsourcing, including optimization tasks and MonteCarlo simulations. In [42], Du. et al. propose a method ofcheating detection for general computation outsourcingin grid computing. The server is required to providea commitment via a Merkle tree based on the resultsit computed. The customer can then use the commit-ment combined with a sampling approach to carry outthe result verification, without re-doing much of theoutsourced work. However, all above schemes allowserver actually see the data and result it is computingwith, which is strictly prohibited in secure computationoutsourcing model for data privacy. Thus, the problemof result verification essentially becomes more difficult,when both input/output privacy is demanded. Ourwork leverages the algebraic property of matrix-vectormultiplication and effectively integrate the batch resultverification within the mechanism design, introducingvery small amount of extra overhead on the customer.Difference from Conference Version Portions of thework presented in this paper have previously appearedas an extended abstract in [1]. We have revised thearticle a lot and improved many technical details as

compared to [1]. The primary improvements are asfollows: Firstly, we provide a new mechanism design onsecure outsourcing LE via direct method in Section 3.2.The discussion on the pros and cons of this mechanismjustifies our observation and motivation for choosingiterative method as our base for the secure outsourcingmechanism design. Secondly, we provide a new Sec-tion 6, where we thoroughly discuss the series of practi-cal techniques and mechanism parameter considerationsthat should be taken into account when implementingthe mechanism for specific applications. Thirdly, weprovide a new Section 8.3 in Related Work which includediscussions and comparisons over another importantbranch of research works on data computation delega-tion and result verification, which are closely related toour cheating detection mechanism design. Finally, weprovide a complete yet rigorous security proof for thetheorem 1 in Section 5.2, which is lacking in [1].

9 CONCLUDING REMARKS

In this paper, we investigated the problem of securelyoutsourcing large-scale LE in cloud computing. Differ-ent from previous study, the computation outsourcingframework is based on iterative methods, which hasthe benefits of easy-to-implement and less memory re-quirement in practice. This is especially suitable for theapplication scenario, where computational constrainedcustomers want to securely harness the cloud for solvinglarge-scale problems. We also investigated the algebraicproperty of the matrix-vector multiplication and devel-oped an efficient and effective cheating detection schemefor robust result verification. Thorough security analysisand extensive experiments on the real cloud platformdemonstrate the validity and practicality of the proposedmechanism.

ACKNOWLEDGEMENT

This work was supported in part by the US NationalScience Foundation under grant CNS-0831963.

REFERENCES

[1] C. Wang, K. Ren, J. Wang, and K. M. R. Urs, “Harnessing the cloudfor securely solving large-scale systems of linear equations,” inProc. of the 31st International Conf. on Distributed Computing Systems(ICDCS).

[2] P. Mell and T. Grance, “Draft nist working definition of cloudcomputing,” Referenced on Jan. 23rd, 2010 Online at http://csrc.nist.gov/groups/SNS/cloud-computing/index.html, 2010.

[3] M. Armbrust, A. Fox, R. Griffith, A. Joseph, R. Katz, A. Konwin-ski, G. Lee, D. Patterson, A. Rabkin, I. Stoica, and M. Zaharia, “Aview of cloud computing,” Communications of the ACM, vol. 53,no. 4, pp. 50–58, April 2010.

[4] Cloud Security Alliance, “Security guidance for critical areasof focus in cloud computing,” 2009, online at http://www.cloudsecurityalliance.org.

[5] Sun Microsystems, Inc., “Building customer trust in cloud com-puting with transparent security,” 2009, online at https://www.sun.com/offers/details/sun transparency.xml.

[6] C. Gentry, “Computing arbitrary functions of encrypted data,”Commun. ACM, vol. 53, no. 3, pp. 97–105, 2010.

14

[7] K. Forsman, W. Gropp, L. Kettunen, D. Levine, and J. Salonen,“Solution of dense systems of linear equations arising fromintegral-equation formulations,” IEEE Antennas and PropagationMagazine, vol. 37, no. 6, pp. 96–100, 1995.

[8] A. Edelman, “Large dense numerical linear algebra in 1993:The parallel computing influence,” International Journal of HighPerformance Computing Applications, vol. 7, no. 2, pp. 113–128, 1993.

[9] V. Prakash, S. Kwon, and R. Mittra, “An efficient solution ofa dense system of linear equations arising in the method-of-moments formulation,” Microwave and Optical Technology Letters,vol. 33, no. 3, pp. 196–200, 2002.

[10] B. Carpentieri, “Sparse preconditioners for dense linear systemsfrom electromagnetic applications,” Ph.D. dissertation, CERFACS,Toulouse, France, 2002.

[11] R. Cramer and I. Damgard, “Secure distributed linear algebra ina constant number of rounds,” in Proc. of CRYPTO, 2001, pp. 119–136.

[12] K. Nissim and E. Weinreb, “Communication efficient secure linearalgebra,” in Proc. of TCC, 2006, pp. 522–541.

[13] E. Kiltz, P. Mohassel, E. Weinreb, and M. K. Franklin, “Securelinear algebra using linearly recurrent sequences,” in Proc. of TCC,2007.

[14] P. Mohassel and E. Weinreb, “Efficient secure linear algebra in thepresence of covert or computationally unbounded adversaries,”in Proc. of CRYPTO, 2008, pp. 481–496.

[15] J. R. Troncoso-Pastoriza, P. Comesana, and F. Perez-Gonzalez,“Secure direct and iterative protocols for solving systems oflinear equations,” in Proc. of 1st International Workshop on SignalProcessing in the EncryptEd Domain (SPEED), 2009, pp. 122–141.

[16] W. Du and M. J. Atallah, “Privacy-preserving cooperative sci-entific computations,” in Proc. of 14th IEEE Computer SecurityFoundations Workshop (CSFW), 2001, pp. 273–294.

[17] Y. Saad, Iterative Methods for Sparse Linear Systems, 2nd ed. Societyfor Industrial and Applied Mathematics, 2003.

[18] P. Paillier, “Public-key cryptosystems based on composite degreeresiduosity classes,” in Proc. of EUROCRYPT, 1999, pp. 223–238.

[19] Amazon.com, “Amazon elastic compute cloud,” Online at http://aws.amazon.com/ec2/, 2009.

[20] R. Gennaro, C. Gentry, and B. Parno, “Non-interactive verifiablecomputing: Outsourcing computation to untrusted workers,” inProc. of CRYPTO, Aug. 2010.

[21] C. Wang, N. Cao, J. Li, K. Ren, and W. Lou, “Secure rankedkeyword search over encrypted cloud data,” in Proc. of ICDCS,2010.

[22] J. Li, Q. Wang, C. Wang, N. Cao, K. Ren, and W. Lou, “Fuzzykeyword search over encrypted data in cloud computing,” inProc. of IEEE INFOCOM’10 Mini-Conference, San Diego, CA, USA,March 2010.

[23] S. Yu, C. Wang, K. Ren, and W. Lou, “Achieving secure, scalable,and fine-grained access control in cloud computing,” in Proc. ofINFOCOM, 2010.

[24] G. Dahlquist and A. Bjorck, Numerical Methods. Courier Dover,2003.

[25] T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Intro-duction to Algorithms, 2nd ed. MIT press, 2008.

[26] D. Benjamin and M. J. Atallah, “Private and cheating-free out-sourcing of algebraic computations,” in Proc. of 6th Conf. onPrivacy, Security, and Trust (PST), 2008, pp. 240–245.

[27] M. Atallah and K. Frikken, “Securely outsourcing linear algebracomputations,” in Proc. of ASIACCS, 2010, pp. 48–59.

[28] M. Bellare, J. Garay, and T. Rabin, “Fast batch verification formodular exponentiation and digital signatures,” in Proceedings ofEurocrypt 1998, volume 1403 of LNCS. Springer-Verlag, 1998, pp.236–250.

[29] J. Camenisch, S. Hohenberger, and M. Pedersen, “Batch verifica-tion of short signatures,” in Proceedings of Eurocrypt EUROCRYPT,volume 4515 of LNCS. Springer-Verlag, 2007, pp. 243–263.

[30] C. Orlandi, A. Piva, and M. Barni, “Oblivious Neural NetworkComputing via Homomorphic Encryption,” EURASIP Journal onInformation Security, vol. 2007, pp. 1–10, 2007.

[31] S. Han and W. K. Ng, “Privacy-preserving linear fisher discrimi-nant analysis,” in Proc. of the 12th Pacific-Asia conf. on Advances inknowledge discovery and data mining.

[32] S. Han, W. K. Ng, L. Wan, and V. C. Lee, “Privacy-preservinggradient-descent methods,” IEEE Transactions on Knowledge andData Engineering, vol. 22, no. 6, pp. 884–899, 2010.

[33] M. Armbrust, A. Fox, R. Griffith, A. D. Joseph, R. H. Katz,A. Konwinski, G. Lee, D. A. Patterson, A. Rabkin, I. Stoica,and M. Zaharia, “Above the clouds: A berkeley view of cloudcomputing,” University of California, Berkeley, Tech. Rep. UCB-EECS-2009-28, Feb 2009.

[34] J. Bethencourt, D. X. Song, and B. Waters, “New techniques forprivate stream searching,” ACM Trans. Inf. Syst. Secur., vol. 12,no. 3, 2009.

[35] A. C.-C. Yao, “Protocols for secure computations (extended ab-stract),” in Proc. of FOCS, 1982, pp. 160–164.

[36] C. Gentry, “Fully homomorphic encryption using ideal lattices,”in Proc of STOC, 2009, pp. 169–178.

[37] M. J. Atallah, K. N. Pantazopoulos, J. R. Rice, and E. H. Spaf-ford, “Secure outsourcing of scientific computations,” Advances inComputers, vol. 54, pp. 216–272, 2001.

[38] C. Wang, K. Ren, and J. Wang, “Secure and practical outsourcingof linear programming in cloud computing,” in Proc. of IEEEINFOCOM, 2011.

[39] S. Goldwasser, Y. T. Kalai, and G. N. Rothblum, “Delegatingcomputation: interactive proofs for muggles,” in Proc. of STOC,2008, pp. 113–122.

[40] P. Golle and I. Mironov, “Uncheatable distributed computations,”in Proc. of CT-RSA, 2001, pp. 425–440.

[41] D. Szajda, B. G. Lawson, and J. Owen, “Hardening functions forlarge scale distributed computations,” in Proc. of IEEE Symposiumon Security and Privacy, 2003, pp. 216–224.

[42] W. Du, J. Jia, M. Mangal, and M. Murugesan, “Uncheatable gridcomputing,” in Proc. of ICDCS, 2004, pp. 4–11.