Towards Trusted Services:Result Verification Schemes for MapReduce

Chu Huang, Sencun Zhu and Dinghao WuSchool of Information Science and Technology

Pennsylvania State University, State College, PA 16802{cuh171,sxz16,dwu12}@psu.edu

Abstract—Recent development in Internet-scale data ap-plications and services, combined with the proliferation ofcloud computing, has created a new computing model fordata intensive computing best characterized by the MapReduceparadigm. The MapReduce computing paradigm, pioneered byGoogle in its Internet search application, is an architecturaland programming model for efficiently processing massiveamount of raw unstructured data. With the availability ofthe open source Hadoop tools, applications built based on theMapReduce computing model are rapidly growing.

In this work, we focus on a unique security concern onthe MapReduce architecture. Given the potential security risksfrom lazy or malicious servers involved in a MapReduce task,we design efficient and innovative mechanisms for detectingcheating services under the MapReduce environment basedon watermark injection and random sampling methods. Thenew detection schemes are expected to significantly reducethe cost of verification overhead. Finally, extensive analyticaland experimental evaluation confirms the effectiveness of ourschemes in MapReduce result verification.

Keywords-MapReduce, watermark injection, random sam-pling, result verification, trustworthy

I. INTRODUCTION

Recently, a new wave of large-scale data processing tech-nologies has emerged in various research and business areas,such as genomic analysis, visualization, simulation and busi-ness intelligence, etc. Due to the ever increasing demandsin those fields, more and more people and organizationshave come to realize the increasing needs of computationalresources. MapReduce was first proposed by Google in2004 [1]. As a compensation mechanism to make up forindividual’s lack of computation resources, the MapReduceframework enables huge data processing by dynamicallybuilding up the computation environment which consistsof a large number of computers. Considering its benefits,MapReduce has been widely adopted by a number of largecompanies, such as Google, Yahoo, Amazon, Facebook andAOL [2]. An open source implementation of MapReducecalled Hadoop [3], was then developed by Yahoo. The easeof application development using Hadoop further encourageswide adoption of MapReduce.

*This work was supported by NSF CAREER 0643906

Often operated in the open environment with its emphasison Internet-wide accessibility, Mapreduce provides flexibleand scalable computing services for its clients. However,this also leaves MapReduce vulnerable to third party attacksand misbehavior. Given that MapReduce clients have nocontrol over their data once it gets fed into the distributedapplications, many of them have their worries on commonsecurity issues, such as the violations of data integrity anddata disclosure. Besides all those common privacy threatsand security risks, MapReduce also has its own uniquesecurity deficiencies on data miscalculation while lazy ormalicious servers involved in a MapReduce task. In thispaper, we mainly focus on two main motivations lyingbehind this malicious service providing, including attacksby arbitrary purpose and attacks by strategic purpose.

To address the above challenges, we present twolightweight results verification schemes for detecting thecheating behaviors within the MapReduce framework. Weprimarily focus on some text retrieval related applications,which involve simple operations (e.g., count, addition) onlarge amount of text and graph data. Expanded from thosetext retrieval related applications, we believe our proposedscheme can also be effectively applied to other MapReducebased text processing applications, such as distributed grep,log data processing, speech recognition and machine trans-lation.

II. RELATED WORK

Security of distributed systems has been studied by manyresearchers, but only a few of them really focus on MapRe-duce. Xiao et al. [4] presented an accountable MapReduceplatform which checks all working machines and detectsmalicious nodes in real time. By replaying the tasks executedby workers and matching output with the original results, au-ditors are able to generate verifiable evidence once inconsis-tency occurs. The Airavat system proposed by Roy et al. [5]provides strong security and privacy guarantees for sensitivedata computation of MapReduce. Their work focuses onprotecting privacy issues of untrusted code of data-miningand data-analysis algorithms executed on MapReduce. Weiet al. [6] proposed a secure scheme called SecureMR whichaims at protecting the computation integrity issue of MapRe-

duce. SecureMR detects misbehavior of mappers by sendingsame tasks to multiple mappers, and check consistency ofresults. Existing security frameworks for MapReduce havethe same limitation: reducers and master need to be trusted.This assumption might not be practical in real world.

Several other result checking techniques have been pro-posed in different areas other than MapReduce. Golle andMironov [7] proposed a Ringer scheme to protect againstcoalitions of lazy cheaters, which ensures client that mostof the work are done correctly with high probability. Szajdaet al. [8] extended this ringer scheme, and presented aprobabilistic verification scheme to prevent against mali-cious behavior in grid computing. A generic verificationscheme based on redundancy was presented by Golle andSutbblebine [9]. They described a security framework forcommercial distributed computing by simple task redun-dancy. Szajda et al. [10] presented another redundancy-basedstrategy. Their work requires fewer resources comparedto Golle and Sutbblebine’s, but it can achieve the sameeffective cheating detection rate. Another generic approachwas proposed by Zhao et al. [11]. This result verificationscheme is called Quiz for peer-to-peer grid computing. Theyinject indistinguishable tasks into a job with other normaltasks. Then the verifier checks the correctness of all quizresults.

Most of the above mentioned existing studies are basedon replication techniques, with either tasks being duplicatedand sent to two or more different participants for processing,or re-do partial of a task then match it with the pre-calculatedresult. Such redundant tasks waste resources in the system,and there is lack of an efficient way to detect colludingparticipants. We remove the constraints that master and/orreducers have to be trusted and provide a scheme basedon watermark injection and weighted sampling techniques.Our scheme can efficiently detect cheating behavior ofMapReduce computing in the context of text processingproblems.

III. SYSTEM MODEL

A. MapReduce Programming Model

The MapReduce framework consists of a single masterJobTracker and more than one slave Task Trackers. Masteris responsible for task assignment, scheduling, and manage-ment. A slave that contributes to the computation resourcesis also called worker in this model. Workers execute the tasksas directed by Master. Besides master and workers, the otherimportant entity in MapReduce system is the distributed filesystem. Typically, a file in Hadoop Distributed File System(HDFS) [1], the file system underlying the Hadoop system,is gigabytes to terabytes in size. Our work will follow theHadoop implementation of MapReduce.

As shown in Figure 1, the process of MapReduce dataprocessing can be divided into two phases: map and reduce.In the beginning of the map phase, input data is first sliced

Figure 1: The MapReduce programming model

into n splits. Then master assigns these splits to differentworkers for paralleled processing. Workers on the map phaseare called mappers. In this phase, each mapper processes onesplit, produces an intermediate result and outputs the inter-mediate result in the (key, value) key-value pair format to itslocal disk. The intermediate results from each mapper arethen partitioned into m parts by a partition function. In thereduce phase, workers read partitioned intermediate resultsfrom mappers in the preceding map phase, process themindependently, and merge all intermediate results associatedwith the same key into a final result. Workers running thereduce function are called reducers correspondingly. What tofollow for reducers is storing final results to their local disks.Eventually, the final result will be stored in the distributedfile system.

B. Model of Cheaters

In the MapReduce computation model, we consider anattack model in which the adversary is rational and econom-ically motivated. We classify cheating behaviors using twodifferent adversary models. We assume the input domain forMapReduce is U , and the task for MapReduce is to computeF : f2(f1(x)) for all the input domain, in which f1 is a mapfunction while f2 is a reduce function.

- Lazy Cheating Model: In this model, the cheatingworker follows the master’s computation with two ex-ceptions: 1) it will not start processing its task from thebeginning of its task, or drop a task at any point beforefinishing the task. 2) for every x ∈ U , MapReduce usesG : g2(g1(x)) as the result instead of F : f2(f1(x)),where function G is usually much less expensive thanfunction F . The goal of this kind of cheaters is tosave computation resources and maximize their profitby performing more tasks in the same period of time.

- Malicious Cheating Model: In this model, the behaviorof a malicious worker is either arbitrary or strategic.For any specific x, instead of returning F (x) = (k, v),it would manipulate the final result and return (k′, v′).In other words, the worker intentionally returns wrongresults to client. Arbitrarily cheaters intentionally returnwrong results for the purpose of disrupting the compu-tation. Strategic cheaters falsify only a tiny portion ofthe results with a purpose (e.g., improve its websiterank for search engines), and therefore is more difficultto identify.

Based on the models, we provide two secure schemes toprevent the computation provider from cheating the clientby claiming that they have done the job that they actuallydid not, and also to protect from malicious behavior ofMapReduce.

IV. RESULT VERIFICATION FOR MAPREDUCE

A. Watermark Injection

Our watermark injection scheme is an application-specificverification scheme. The basic idea of watermark injectionis to inject indistinguishable marks into normal documents.Figure 2 shows the process of this watermark injectionscheme. We assume that if the marks we injected are allcomputed correctly, the workers have executed the taskshonestly. Otherwise, workers, one or many, have not con-ducted the task honestly. We outline this watermark injectionscheme as follows.

- Setup (Watermark generation): Before handing the datato the computation provider, the verifier pre-processesthe input files by inserting a number of watermarks intothe task documents.

- Computation: MapReduce performs computation tasksand returns final results to the client.

- Verification: The client verifies the correctness of theresults and accepts the result if no inconsistency isfound.

- Recovery: Then it recovers the injected documents andremoves the impact of the watermark. Otherwise, theclient discards the results.

Because marks are randomly injected and cannot bedistinguished from other data items in the task documents,MapReduce will not be able to identify them in an easymanner. Consequently, if some task executer cheats on a taskthat contains injected marks, its corresponding computationresult will be inconsistent with what was pre-computed. Nextwe use an example application, inverted index, to illustrateour proposed scheme.

1) Inverted Index: An inverted index is a data structurethat maps terms to their locations. Using this approach, asearch engine easily identifies the appropriate documents forterms, without scanning each document in the whole web.

Figure 2: Watermark-based verification scheme. A watermarkgeneration module preprocesses raw data and injects watermarks.Upon received output from MapReduce, the verifier performsverification. If the computation from MapReduce is accepted, theverifier will call a recovery module to remove the effect introducedby injecting watermarks.

Many modern search engines on the web use an invertedindex scheme for data storage.

The large amount of computation may incentivize compu-tation service provider to cheat for saving both storage andcomputational resources. One way to cheat is to skip thecomputation and return wrong (randomly picked) result fromits own vocabulary. For strategic cheaters, they might onlypick certain words and manipulate their corresponding lo-cations. To detect and hence prevent cheating phenomenon,we apply the watermark injection scheme on the invertedindex application.

Let D = {D1, D2, · · · , Dk} be a collection of documentsbeing indexed. Let d1, d2, · · · , dk′ be a collection of water-mark files. The verifier then injects watermarks by encodingcertain words in each selected document, and records thedocument id and the corresponding words. We outline thedetailed process for generating watermark as follows:

- Randomly selects k′ documents d1, d2, · · · , dk′ amongall the input documents before handing over to MapRe-duce

- For each selected documents di(i = 1, 2, · · · , k′), theverifier conducts a simple alphabet substitution [12] onevery ε words. A substitution table T should be pre-defined by the verifier. The verifier also needs to storethe information about injection distance ε, the encodedwords and the corresponding document ID.

- For each encoded term in document collection d ={d1, d2, · · · , dk′}, the verifier builds a posting list. Thecomputation overhead for building a small invertedindex I ′ for all encoded terms is far lower than buildinga complete inverted index I(I ′ ∈ I).

Only the client knows which documents have been chosenand which words have been marked. The verifiable compu-

tation scheme for inverted index is performed as follows:1. Setup: Before handing over the data to a computation

provider, the verifier pre-processes the data and gen-erates watermark following the detailed procedure ofwatermark generation above. Meanwhile, the verifierstores information about which words are encoded andconstructs an index I ′ of encoded (watermark) words.

2. Computation: MapReduce performs the task of con-structing an inverted index I and returns I to client.

3. Verification: The verifier checks the results fromMapReduce by 1) examining whether all encoded termsappearing in the dictionary of I ′ are all included in thedictionary of I; 2) comparing the posting list of eachterm in I ′ with the results from the MapReduce system.If both steps return a matching conclusion, the verifieraccepts the results. Otherwise, the verifier concludesMapReduce has cheated and discards the results.

4. Recovery: If the result is accepted, the verifier needs torecover these words that have been marked. In order todo so, the verifier needs to decode those words basedon the same substitution table T . For each term in theindex I ′: if its corresponding document ID in its posinglists of I , then doing nothing; otherwise, insert a postinglist with idx as document ID into index I .

2) Security Analysis: The security of the watermarkinjection scheme is based on the assumption that workersof MapReduce cannot determine whether the input file hasbeen marked or not. The only way to escape detectionis to correctly compute all encoded words. To determinewhether a word is actually encoded or not, lazy workersof MapReduce needs to perform brute force search over allwords in its dictionary (assuming it contains all words in realworld). The computation overhead of this process is muchheavier than that of indexing itself. If the purpose of thecheating workers were to save computation resource, theywould lose the incentive to cheat. If lazy workers do notprocess the data, or give wrong results by random guess,the results from MapReduce would be inconsistent withthe pre-computed value. Misbehavior is going to be caughtwith a high probability as long as they ever cheat on thecomputation.

To convince readers the high accuracy of the watermarkinjection scheme, we provide a brief theoretical derivation ona simplified scenario. Let D = {D1, D2, · · · , Dn} representthe set of documents to be sent to computation providers.We randomly select p1 percent of totally n documents,np1, to inject watermarks. In each document, we encodep2 percentage of words to do the watermark injection. Ifwe assume that each document contains roughly the samenumber of words, w, the total number of watermarkedword would be np1wp2. Suppose we have m mappers andr reducers in a MapReduce system. MapReduce dividesall n task documents evenly and distributes them to them mappers. Thus each mapper gets n/m documents. The

probability of this situation is

Pr(Detected|one mapper cheats) = 1− (1− p1)n/m

Then the probability that a mapper cheats without beingdetected is (1−p1)n/m. A similar analysis should be appliedto the reducer step. The nw words are randomly assignedto r reducers and each reducer receives on average nw/rwords. The probability that a word is watermarked is p1p2.When a reducer decides to cheat, it escapes the detectiononly if this reducer does not receive any watermarked words.Thus, the probability is (1− p1p2)nw/r.

When there are more mappers or more reducers cheating,the detection rate is even higher. Say, x mappers and yreducers cheat, the probability of detection is

Pr(Detected|x, y) = 1− (1− p1)xn/m(1− p1p2)ynw/r (1)

Let us denote (1 − p1)xn/m(1 − p1p2)ynw/r as f(x, y),describing the probability that the verification scheme failswhen x mappers and y reducers have cheated. Assume eachmapper and each reducer may cheat with a fixed probability,pm and pr, respectively. The probability that x mappers andy reducers cheat is

Pr(x, y) =m!

x!(m− x)!(1−pm)m−xpxm

r!

y!(r − y)!(1−pr)r−ypyr

(2)Thus, the conditional probability of detection is

Pr(Detected|Cheating) =x∑

[0,m]

y∑[0,r]

Pr(x, y)Pr(Detected|x, y)

(3)Using what we have,

Pr(Detected|Cheating) = 1−x∑

[0,m]

y∑[0,r]

Pr(x, y)f(x, y) (4)

In this summation, x ∈ [0,m] and y ∈ [0, r]. Thus

Pr(Detected|Cheating) = 1− (1− pm + pm(1− p1)n/m)m ·(1− pr + pr(1− p1p2)nw/r)r

Let’s suppose m = 15, pm = 0.10 and r = 15, pr = 0.10and look at the detection rate of this verification system.Assume n = 1500 and w = 1500 and we can plot thedetection rate vs. small values of p1 and p2 by using MatLab(as shown in Figure 3). Before proceeding, we examine theequation first. We have n/m = 100 and (1−p1)100 = 0.9100

that is almost 0. Similarly, we have (1 − p1p2)nw/r ≈ 0.Thus, the equation is approximately:

Pr(Detected|Cheating) = 1− (1− pm)m(1− pr)r (5)

This means that when each mapper or each reducer hasto deal with a large number of documents or words andthe system cheats, the probability that the verifier detectsthis cheating is independent of the percentage of the wa-

Table I: The effectiveness of the watermark injection schemefor text problems.

MapReduce cheats MapReduce not cheat

Results Rejected 1− (1− pm)m(1− pr)r 0

Results Accepted (1− pm)m(1− pr)r 1

termarked files/words (as long as they are not too small).Consequently,

Pr(Detected|Cheating) = 0.9576 (6)

Table I shows the effectiveness of the watermark injectionscheme for the inverted index problem. The two cellsrepresent the probability that the verifier rejects or acceptsthe computation results from the MapReduce system, whenthe system cheats or does not cheat. As shown, the accuracyis fairly high and the Type I error rate is 0.

Figure 3: The theoretical performance of the watermark injectionscheme for text problems.

B. Random Sampling

1) PageRank: In the real world, the number of web pagescould be very large and it is still growing exponentially.According to WorldWideWebSize.com [13], the size of theWorld Wide Web contains about 17.5 billion indexed pages.Such a worldwide graph contains billions of nodes andseveral billions of links. If we assume that each URL takes0.5KB to store, the whole Internet, containing trillions ofunique URLs, will take more than 400TB space. Calculatingthe PageRank values frequently using the classic PageRankalgorithm [14], [15] on such a big data set is computationallyexpensive. MapReduce provides a solution to tackle this kindof scaling challenges in a highly distributed manner.

We introduce a basic PageRank model here. The webpages and hyperlinks on the Web can be modeled as nodesand edges in a directed graph G = (V,E). Let N be the totalnumber of web pages and use 1, 2, · · · , N to index these Nweb pages. Let L(u) be the number of out-links from page

u where u = 1, · · · , N . The corresponding adjacency matrixis defined as follows:

Au,v =

{1/L(u) , if there is an edge from u to v;

0 , otherwise.(7)

Note that the value of Au,v can be any number between0 and 1 in this case. The PageRank value for a page v isgiven as follows [14]:

PR(v) =1− dN

+ d∑

u∈B(v)

PR(u)

L(u)(8)

The damping factor [15] d in the equation is used torepresent the probability that a Web surfer follows therandom behavior pattern. The value of d is usually set to0.85.

In order to retrieve a stable solution for the PageRankvalues of all webpages, we set the PR(i) value to be1/N and keep running iterations until stationary values areobtained or only small differences exist between the resultsfrom adjacent iterations. Note that in the rest of our work,we will use the term “node” and “page” interchangeably.

2) Ranking Pages with MapReduce: Suppose a clientrents MapReduce to process a large set of web pages forfinding a way to characterizing structure of the web graphand computing the importance of every web page using theclassic PageRank algorithm. This task needs multiple mapand reduce phases to accomplish. Since values needed foran iteration of a page only depend on previous page ranksof its in-links, every task sent to workers of MapReduce isindependent and can be executed in parallel. This iterationwill continue until convergence.

The watermark injection scheme does not work very wellunder the context of PageRank calculation. The reason isthat in order to do the verification, we need to pre-calculatethe PageRank values for these marked or injected pages.However, doing this calculation means we need to do thecalculation for the whole data set. If so, using MapReduce todistribute the complex task is not meaningful anymore. Wepropose another approach for the problem: verification bysampling. We provide two sampling templates for cheatingdetection for PageRank: naive random sampling and in-degree weighted sampling.

Naive Random Sampling: In this naive random sam-pling scheme, the verifier randomly selects pages withan equal probability for each page. Using the PageRankscores returned from MapReduce, the verifier checks if thePageRank equation holds. The PageRank values of the in-nodes of the sampled nodes and the out-degree of thesein-nodes need to be collected as well, according to theequation. Computation providers cannot possibly guess ormake a reasonable solution that holds the equation for allthe nodes in the network without actually computing thepage rank. Thus, under any type of cheating models, there

must be a great number of webpages on which the equationdoes not hold. Thus, the detection rate of this naive samplingverification scheme is almost 1.

However, there might be cases when computationproviders intentionally modify the PageRank value for cer-tain webpages for their personal sakes. The percentage ofthese modified pages may take only a small portion of thewhole data set. Consequently, the naive sampling approachcannot guarantee a high detection rate in this scenario.Thus, we propose a weighted sampling scheme to tacklethis concern.

In-degree Weighted Sampling: When selecting onepage and checking if Equ.8 holds for this page, we areactually checking this page as well as all its in-nodes. Ifthe page rank of one of its in-nodes of itself is modified,the equation will not hold and hence we have detectedthe cheating behavior. On the basis of this fact, when weselect a page that has a high in-degree, we are actuallychecking a number of pages in the set. Thus, an intuitivesuggestion is to sample those pages with high in-degreesas much as possible. In order to prevent the computationproviders figure out our verification approach, we shouldadd a randomness degree to the sampling. In the in-degreeweighted scheme, we add a weight that is proportional to in-degree to each page and adopt a weighted sampling method.Thus, pages with higher in-degrees have a higher probabilityto be selected.

3) Security Analysis: When computation providers inten-tionally change the value of a small portion of pages, suchas increasing the PageRank value of their own pages anddecreasing the value of their opponent’s pages, there mightbe only a small number of nodes for which Equation (1) doesnot hold. Let’t look at the effectiveness of the two samplingschemes here. Assume N is the total number of pages ander is the error rate—the percentage of pages which do notsatisfy Equation (8). The verifier selects M pages by oneof the two sampling schemes. As mentioned, the in-degreevalues of these M pages affect the verification effectiveness.Let’s use f(x) to describe the in-degree distribution of allpages in the data set and x in the following equations refersto the in-degree value. Use c to represent the detection rate.When we only check one page whose in-degree is x, we areactually checking this node and all the x in-nodes. Thus,the detection rate is g(x) = 1 − (1 − er)x+1. The overalldetection rate should be the expectation value of g(x), withrespects to the distribution of x. Thus,

c(Naive) = E(1− (1− er)M(1+x))

=

∫f(x)(1− (1− er)M(1+x))dx

According to Jamakovic and Uhlig [16], in a regulatorynetwork, the in-degrees follow an exponential distribution.Let’s assume that x follows an exponential distribution, wehave, f(x) = K exp(−Kx), x ∈ [0, inf).

Table II: The effectiveness of the watermark injectionscheme for text problems.

MapReduce cheats MapReduce not cheat

Results Rejected 1− (1− pm)m(1− pr)r 0

Results Accepted (1− pm)m(1− pr)r 1

Substituting f(x) into the detection rate equation, weobtain,

c(Naive) = 1− K(1− er)M

1− (1− er)MerK(9)

when (1− er)MerK < 1, which can be easily satisfied.For the in-degree weighted sampling algorithm, instead

of f(x), we should have (x+ 1)f(x)/A in the integration.(x+1)A represents the effect of the weights and A =

∫(x+

1)f(x)dx is only a normalization coefficient. Thus,

c(Weighted) = E(1− (1− er)M(1+x))

=

∫(x+ 1)f(x)(1− (1− er)M(1+x))dx/A

For the same exponential distribution of x, we obtain,

c(Naive) = 1− K(1− er)M

(1− (1− er)MerK)2A(10)

The relationship between A and K is that A=1+1/K. Let’sset K = 1/4 (the average in-degree number is 4) and er =0.05. Let’s increase M from 10 to 100 and compare thetwo schemes under this simple scenario. Figure 4 showsthe plots for the detection rates. Apparently, the weightedscheme outperforms the naive one.

For the Internet, Jamakovic and Uhlig [16] concluded thatthe in-degrees of the Internet follow a power law distributionf(x) = ax−γ , x ∈ [0, inf). Under this circumstance, we canconduct a similar derivation and obtain the following results.

c(Naive) =

∫[1− (1− er)M(1+x)]f(x)dx

= 1− (1− er)Ma∫

[(1− er)M ]x · x−γdx

c(Weighted) =

∫[1− (1− er)M(1+x)]xf(x)/Adx

= 1− (1− er)Ma/A∫

[(1− er)M ]x · x1−γdx

Computational simulations lead to similar numeric results asshown in Figure 4.

Table II shows the effectiveness of the weighted samplingscheme for the PageRank calculation problem.

V. EXPERIMENTAL EVALUATION

To evaluate the effectiveness of our verification scheme, aprototype of watermark injection and random sampling ver-ification schemes is implemented to test the overall method

Figure 4: The theoretical performance of the two samplingschemes for PageRank.

performance. Experiments conducted in this research aim tomainly measure the detection rate under different types ofcheating models.

A. Experiment Setup

The experiment is running on a small cloud with 10machines. We use 9 host machines as workers that offerMapReduce services and one host as a master. All host ma-chines run on a 3.4GHz Intel Core Quad-core i7 processor.Each virtual machine has 512MB of memory and 20GB diskand was installed Ubuntu Linux 10.04.2, Sun JDK 6 andHadoop 0.20.2. We conduct our experiment using invertedindex application and PageRank calculation based on theGoogle PageRank algorithm. Since this experiment is notfocused on optimization of Hadoop configuration, we justuse the default configuration of Hadoop.

B. Performance Analysis

Figure 5: The performance of the watermark injection schemefrom computational simulations

Figure 6: The performance of the two sampling schemes undercomputational simulations

1) Watermark Injection: Every data point shown in Fig-ure 5 can be viewed as a test conducted with settings ofvaried watermark injection proportions within documentsand words. Given that 200 simulations are run for each ofthose tests, mean values are calculated as the data pointsas shown in this figure. Consistent with the conceptualmodeling, p1 in this case is the percentage of documentswith injected watermarks, ranging from 1% to 5%. In thesame way, p2, also with the same range values, indicatesthe proportion of words within each document which areencoded with watermarks. One can interpret from the abovefigure that among all the 16810 results points, only less than0.5% of them have a detection rate less than 95%. Moreover,more than half of the simulations have their detection rateslarger than 99%.

Besides, when considering p1 and p2’s effects on the over-all detection rates, we further find that, compared with p1,p2 seems to have non-significant impact on the experimentresults. It seems that detection rates do not change very muchalong with the variances in the percentage of watermark in-jected words. However, if viewing across the p1 axis, one cannotice that along with the growth of the watermark injecteddocument numbers, detection rates of cheating behaviorsbegin to increase until the injection percentage reaches 3%.In other words, it shows relatively lower detection rate whenthe watermark injected document proportions are less than3%. Beyond that injection percentage, detection rates beginto reach about 100%.

2) Random Sampling: In order to simulate the cheatingbehavior of the malicious and lazy workers, intentionalmiscalculations of PageRank values were injected into thefinal evaluation dataset at an error rate of 1% to 5%,respectively.

Figure 6 demonstrates the distributions of the experimen-tal detection rates over the increasing number of sampling

ratios. Overall, we can see that the random sampling ap-proaches, both naive and weighted, perform nicely on thePageRank cheating detection task under the MapReduceenvironment.

As defined as the percentage of miscalculations, errorrates ranging from 1% to 5% are represented by five differentcolors as shown in the figure. Dashed lines are correspondingto the naive-based random sampling approach, whereas solidlines are for the more advanced weighted-based randomsampling methods. Detection rates increase dramaticallywith lower sampling ratios, in particular when samplingratios are less than 0.15% in the weighted approach or lessthan 0.3% in the naive-based method.

Almost all of the weighted experiments reach high detec-tion rates of over 80%, while in contrast, results of the naivebased approach are not as desired as the weighted ones undersmall sampling sizes. Weighted samplings gradually lost itsabsolute superiority in detection rates as the sampling ratiosincreases. Except naive samplings under 1% and 2% errorrates, all sampling settings tested, both naive and weightedschemes, in this study, achieve about the same detection rateswhen sampling more than 35 pages out of the total 10,000ones.

VI. DISCUSSION AND CONCLUSION

In this work, we have proposed two verification schemesfor detecting lazy and malicious behaviors that MapReducesystems may have when they are conducting specific tasks.Although we have only explained our schemes on two textretrieval related applications for easy-to-understand purpose,we would like to remark that our work can also be gen-eralized straightforward to other text-intensive tasks, suchas word count, distributed grep, log data processing, speechrecognition and machine translation. In fact, given any com-putational task with text input, watermark injection enablesMapReduce task executors to verify the results based onthose watermarked items. Likewise, for computational taskswith graph input, our proposed random sampling schemecan also be effectively applied, considering the probabilisticgeneralizations of the general random sampling approach.

Although as mentioned above, our proposed schemescan be effectively extended to some other MapReduce textprocessing applications, they may not be very well gener-alized to numerical data-intensive computational tasks, suchas knowledge discovery, pattern recognition, and more ad-vanced statistical analysis. Also limited by the scope of thisstudy, only lazy and malicious attacks have been considered.Although they do cover a large proportion of the possiblesecurity risks, still there are threats in specific purpose fromthose strategic attackers. For instance, attackers who onlymanipulate their proposed keywords in the inverted indexapplication. In that case, our watermark injection methodwould not work as well as it did while detecting the lazyand malicious attacks. Therefore, with small watermark

percentages, it would be hard to detect this kind of cheatingbehavior using the watermark scheme as we proposed.Considering all those aforementioned limitations, developinga more generic solution for MapReduce applications remainsan open question for our future research.

REFERENCES

[1] J. Dean and S. Ghemawat, “Mapreduce: Simplified dataprocessing on large clusters,” Communications of the ACM,vol. 51, no. 1, pp. 107–113, 2008.

[2] H. Karloff, S. Suri, and S. Vassilvitskii, “A model of com-putation for mapreduce,” in Proceedings of the Twenty-FirstAnnual ACM-SIAM Symposium on Discrete Algorithms. So-ciety for Industrial and Applied Mathematics, 2010.

[3] Hadoop, “Apache hadoop.” [Online]. Available:http://hadoop.apache.org

[4] Z. Xiao and Y. Xiao, “Accountable mapreduce in cloudcomputing,” in SCNC 2011.

[5] I. Roy, S. Setty, A. Kilzer, V. Shmatikov, and E. Witchel,“Airavat: Security and privacy for mapreduce,” in Proceedingsof the 7th USENIX conference on Networked systems designand implementation. USENIX Association, 2010.

[6] W. Wei, J. Du, T. Yu, and X. Gu, “Securemr: A serviceintegrity assurance framework for mapreduce,” in ComputerSecurity Applications Conference, 2009.

[7] P. Golle and I. Mironov, “Uncheatable distributed computa-tions,” Topics in Cryptology—CT-RSA 2001.

[8] D. Szajda, B. Lawson, and J. Owen, “Hardening functionsfor large scale distributed computations,” In Proceedings ofIEEE Symposium on Security and Privacy, May 2003.

[9] P. Golle and S. Stubblebine, “Secure distributed computingin a commercial environment,” in Financial Cryptography.Springer, 2002.

[10] D. Szajda, B. Lawson, and J. Owen, “Toward an optimalredundancy strategy for distributed computations,” in ClusterComputing, 2005. IEEE International. IEEE, 2005.

[11] S. Zhao, V. Lo, and C. Dickey, “Result verification and trust-based scheduling in peer-to-peer grids,” in Proc. Fifth IEEEIntl Conf. Peer-to-Peer Computing (P2P 05), Sept. 2005.

[12] Wikipedia, “Substitution cipher.” [Online]. Available:http://en.wikipedia.org/wiki/Substitution cipher

[13] M. de Kunder, “Daily estimated size of the world wideweb.” [Online]. Available: www.worldwidewebsize.com

[14] S. Brin and L. Page, “The anatomy of a large-scale hyper-textual web search engine,” Computer Networks and ISDNSystems, vol. 30, no. 1-7, pp. 107–117, 1998.

[15] L. Page, S. Brin, R. Motwani, and T. Winograd, “The pager-ank citation ranking: Bringing order to the web.” StanfordDigital Library Technologies Project, Tech. Rep., 1998.

[16] A. Jamakovic and S. Uhlig, “On the relationships betweentopological measures in real-world networks,” Networks andHeterogeneous Media, vol. 3, no. 2, p. 345, 2008.