Topic 7: Shortcomings in the MapReduce Paradigm

7: Shortcomings in the MapReduce Paradigm

Zubair Nabi

zubair.nabi@itu.edu.pk

April 19, 2013

Zubair Nabi 7: Shortcomings in the MapReduce Paradigm April 19, 2013 1 / 31

Outline

1 Hadoop everywhere!

2 Skew

3 Heterogeneous Environment

4 Low-level Programming Interface

5 Strictly Batch-processing

6 Single-input/single output and Two-phase

7 Iterative and Recursive Applications

8 Incremental Computation

Outline

2 Skew

Users1

Adobe: Several areas from social services to unstructured data storageand processing

eBay: 532 nodes cluster storing 5.3PB of data

Facebook: Used for reporting/analytics; one cluster with 1100 nodes(12PB) and another with 300 nodes (3PB)

LinkedIn: 3 clusters with collectively 4000 nodes

Twitter: To store and process Tweets and log files

Yahoo!: Multiple clusters with collectively 40000 nodes; largest clusterhas 4500 nodes!

1http://wiki.apache.org/hadoop/PoweredByZubair Nabi 7: Shortcomings in the MapReduce Paradigm April 19, 2013 4 / 31

Users1

But all is not well

Over the years, Hadoop has become a one-size-fits-all solution to dataintensive computing

As early as 2008, David DeWitt and Michael Stonebraker asserted thatMapReduce was a “major step backwards” for data intensivecomputingThey opined:

I MapReduce is a major step backwards in database access because itnegates schema and is too low-level

I It has a sub-optimal implementation as it, makes use of brute forceinstead of indexing, does not handle skew, and uses data pull instead ofpush

I It is just rehashing old database conceptsI It is missing most DBMS functionalities, such as updates, transactions,

etc.I It is incompatible with DBMS tools, such as human visualization, data

replication from one DBMS to another, etc.

But all is not well

As early as 2008, David DeWitt and Michael Stonebraker asserted thatMapReduce was a “major step backwards” for data intensivecomputing

They opined:I MapReduce is a major step backwards in database access because it

negates schema and is too low-levelI It has a sub-optimal implementation as it, makes use of brute force

instead of indexing, does not handle skew, and uses data pull instead ofpush

But all is not well

I It is just rehashing old database concepts

I It is missing most DBMS functionalities, such as updates, transactions,etc.

I It is incompatible with DBMS tools, such as human visualization, datareplication from one DBMS to another, etc.

But all is not well

I It is incompatible with DBMS tools, such as human visualization, datareplication from one DBMS to another, etc.

But all is not well

Outline

2 Skew

Introduction

Due to the uneven distribution of intermediate key/value pairs somereduce workers end up doing more work

Such reducers become “stragglers”

A large number of real-world applications follow long-tailed distributions(Zipf-like)

Introduction

Wordcount and skew

Text corpora have a Zipfian skew, i.e. a very small number of wordsaccount for most occurrences

For instance, of 242,758 words in the dataset used to generate thefigure, the 10, 100, and 1000 most frequent words account for 22%,43%, and 64% of the entire set

Such skewed intermediate results lead to uneven distribution ofworkload across reduce workers

Wordcount and skew

Page rank and skew

Even Google’s implementation of its core PageRank algorithm isplagued by the skew problem

Google uses PageRank to calculate a webpage’s relevance for a givensearch query

I Map: Emit the outlinks for each pageI Reduce: Calculate rank per page

The skew in intermediate data exists due to the huge disparity in thenumber of incoming links across pages on the Internet

The scale of the problem is evident when we consider the fact thatGoogle currently indexes more than 25 billion webpages with skewedlinks

For instance, Facebook has 49,376,609 incoming links (at the time ofwriting) while the personal webpage of the presenter only has 4 (=))

Page rank and skew

Even Google’s implementation of its core PageRank algorithm isplagued by the skew problemGoogle uses PageRank to calculate a webpage’s relevance for a givensearch query

Page rank and skew

I Map: Emit the outlinks for each page

I Reduce: Calculate rank per page

Page rank and skew

Zipf distributions are everywhere

Followed by Inverted Indexing, Publish/Subscribe systems, frauddetection, and various clustering algorithms

P2P systems have Zipf distributions too both in terms of users andcontent

Web caching schemes as well as email and social networks

Outline

2 Skew

Introduction

In the MapReduce model, tasks which take exceptionally long arelabelled “stragglers”

The framework launches a speculative copy of each straggler onanother machine expecting it to finish quickly

Without this, the overall job completion time is dictated by the sloweststraggler

On Google clusters, speculative execution can reduce job completionby 44%

Introduction

Hadoop’s assumptions regarding speculation

1 All nodes are equal, i.e. they can perform work at more or less thesame rate

2 Tasks make progress at a constant rate throughout their lifetime

3 There is no cost of launching a speculative cost on an otherwise idleslot/node

4 The progress score of a task captures the fraction of its total work thatit has done. Specifically, the shuffle, merge, and reduce logic phaseseach take roughly 1/3 of the total time

5 As tasks finish in waves, a task with a low progress score is most likelya straggler

6 Tasks within the same phase, require roughly the same amount of work

Assumptions 1 and 2

Both breakdown in heterogeneous environments which consist ofmultiple generations of hardware

Assumptions 1 and 2

Assumption 3

Breaks down due to shared resources

Assumption 3

Breaks down due to shared resources

Assumption 4

Breaks down due the fact that in reduce tasks the shuffle phase takesthe longest time as opposed to the other 2

Assumption 4

Breaks down due the fact that in reduce tasks the shuffle phase takesthe longest time as opposed to the other 2

Assumption 5

Breaks down due to the fact that task completion is spread across timedue to uneven workload

Assumption 5

Breaks down due to the fact that task completion is spread across timedue to uneven workload

Assumption 6

Breaks down due to data skew

Assumption 6

Breaks down due to data skew

Outline

2 Skew

Introduction

The one-input, two-stage data flow is extremely rigid for ad-hocanalysis of large datasets

Hacks need to be put into place for different data flow, such as joins ormultiple stages

Custom code has to be written for common DB operations, such asprojection and filtering

The opaque nature of map and reduce functions makes it impossible toperform optimizations, such as operator reordering

Introduction

Outline

2 Skew

Introduction

In case of MapReduce, the entire output of a map or a reduce taskneeds to be materialized to local storage before the next stage cancommence

Simplifies fault-tolerance

Reducers have to pull their input instead of the mappers pushing it

Negates pipelining, result estimation, and continuous queries (streamprocessing)

Introduction

Outline

2 Skew

Introduction

1 Not all applications can be broken down into just two-phases, such ascomplex SQL-like queries

2 Tasks take in just one input and produce one output

Introduction

1 Not all applications can be broken down into just two-phases, such ascomplex SQL-like queries

2 Tasks take in just one input and produce one output

Outline

2 Skew

Introduction

1 Hadoop is widely employed for iterative computations

2 For machine learning applications, the Apache Mahout library is usedatop Hadoop

3 Mahout uses an external driver program to submit multiple jobs toHadoop and perform a convergence test

4 No fault-tolerance and overhead of job submission

5 Loop-invariant data is materialized to storage

Introduction

Outline

2 Skew

Introduction

1 Most workloads processed by MapReduce are incremental by nature,i.e. MapReduce jobs often run repeatedly with small changes in theirinput

2 For instance, most iterations of PageRank run with very smallmodifications

3 Unfortunately, even with a small change in input, MapReducere-performs the entire computation

Introduction

References

1 MapReduce: A major step backwards:http://homes.cs.washington.edu/~billhowe/mapreduce_a_major_step_backwards.html

2 Matei Zaharia, Andy Konwinski, Anthony D. Joseph, Randy Katz, andIon Stoica. 2008. Improving MapReduce performance inheterogeneous environments. In Proceedings of the 8th USENIXconference on Operating systems design and implementation(OSDI’08). USENIX Association, Berkeley, CA, USA, 29-42.

3 Christopher Olston, Benjamin Reed, Utkarsh Srivastava, Ravi Kumar,and Andrew Tomkins. 2008. Pig latin: a not-so-foreign language fordata processing. In Proceedings of the 2008 ACM SIGMODinternational conference on Management of data (SIGMOD ’08). ACM,New York, NY, USA, 1099-1110.

References (2)

4 Tyson Condie, Neil Conway, Peter Alvaro, Joseph M. Hellerstein,Khaled Elmeleegy, and Russell Sears. 2010. MapReduce online. InProceedings of the 7th USENIX conference on Networked systemsdesign and implementation (NSDI’10). USENIX Association, Berkeley,CA, USA.

5 Michael Isard, Mihai Budiu, Yuan Yu, Andrew Birrell, and DennisFetterly. 2007. Dryad: distributed data-parallel programs fromsequential building blocks. In Proceedings of the 2nd ACMSIGOPS/EuroSys European Conference on Computer Systems 2007(EuroSys ’07). ACM, New York, NY, USA, 59-72.

6 Derek G. Murray, Malte Schwarzkopf, Christopher Smowton, StevenSmith, Anil Madhavapeddy, and Steven Hand. 2011. CIEL: a universalexecution engine for distributed data-flow computing. In Proceedings ofthe 8th USENIX conference on Networked systems design andimplementation (NSDI’11). USENIX Association, Berkeley, CA, USA.

References (3)

7 Pramod Bhatotia, Alexander Wieder, Rodrigo Rodrigues, Umut A.Acar, and Rafael Pasquin. 2011. Incoop: MapReduce for incrementalcomputations. In Proceedings of the 2nd ACM Symposium on CloudComputing (SOCC ’11). ACM, New York, NY, USA.

Topic 7: Shortcomings in the MapReduce Paradigm

Technology

Master–worker model for MapReduce paradigm on the TILE64 ...ychung/journal/FGCS-2014.pdf · FutureGenerationComputerSystems36(2014)19–30 Contents lists available atScienceDirect

Secure Matrix Multiplication with MapReduce · The MapReduce programming paradigm allows to process big data sets in parallel on a large cluster of commodity machines. The MapReduce

PaperClip Compliant Email Service Whitepaper · PaperClip believes any chance to overcome the current market solution shortcomings and gain wide adoption is to change the paradigm

7 design patterns db copy - GitHub Pages...Design patterns • In parallel/distributed systems, synchronisation of intermediate results is difﬁcult • MapReduce paradigm offers

Paper 1748-2014 Simulation of MapReduce with the Hash of … · 2014-03-24 · THE GROWING USEFULNESS OF THE MAP-REDUCE PARADIGM The MapReduce framework has now emerged as a highly

Introduction to MapReduce Paradigm for Data Mining COSC 526 Class 2 Arvind Ramanathan Computational Science & Engineering Division Oak Ridge National Laboratory,

Hadoop/MapReduce Computing Paradigm Spring 2015 Taken from: WPI, Mohamed Eltabakh 1

Shortcomings of Cdma

Hadoop/MapReduce Computing Paradigm 1 CS525: Special Topics in DBs Large-Scale Data Management Presented By Kelly Technologies

MapReduce in MPI for Large-scale Graph Algorithmssjplimp/papers/pc11.pdf · 1 Introduction MapReduce is the programming paradigm popularized by Google researchers Dean and Ghe-mawat

VSB - Board Shortcomings

Comparative Analysis of MapReduce, Hive and Pigijoes.vidyapublications.com/paper/Vol17/07-Vol17.pdf · 2016-01-29 · comparative study of Hadoop’s programming paradigm (Map reduce)

MapReduce - University of Technologycse.hcmut.edu.vn/~ptvu/ppds/MapReduce.pdf · More examples: Distributed Grep, Count of URL access frequency, etc. ... master-worker paradigm to

Lecture 11: Graph algorithms - GitHub Pages · Course content • Introduction • Data streams 1 & 2 • The MapReduce paradigm • Looking behind the scenes of MapReduce: HDFS &

MapReduce Paradigm

Automatically Leveraging MapReduce Frameworks for Data ... · MapReduce is a popular programming paradigm for running large-scale data-intensive computation. Recently, many frameworks

Hadoop/MapReduce Computing Paradigm 1 Shirish Agale

MapReduce: Programming - Fordhamstorm.cis.fordham.edu/zhang/cs5950/slides/MapReduceProgramming.pdfOutline • Review and demo • Homework 1 • MapReduce paradigm: hadoop streaming

Data model for analysis of scholarly documents in the MapReduce paradigm

Übung Datenbanksysteme II Web-Scale Data Management · MapReduce: Introduction MapReduce … is a paradigm derived from functional programming. is implemented as framework. operates