GRASS: Trimming Stragglers in Approximation Analytics

Ganesh Ananthanarayanan, Michael Hung, Xiaoqi Ren, Ion Stoica, Adam Wierman, Minlan Yu

Approximation Jobs

• Data deluge timely answers from part of dataset are often good enough

• Deadline-bound Jobs: Maximize accuracy within a deadline– E.g., get me the best ad to show within 1s

• Error-bound Jobs: Minimize time taken to achieve desired accuracy – E.g., get me the #cars sold to the nearest 1000

Prioritize Tasks

• Spawn jobs on large dataset; subset of tasks need to complete– Accuracy α (Subset of completed tasks)

• Prioritize tasks– Jobs run only part of their tasks at a time (multi-

waved)• NP-Hard but many known heuristics…

Stragglers

• Occur despite prevention mechanisms• Speculation: Spawn a duplicate, earliest wins

Challenge: dynamically prioritize while choosing between speculative/unscheduled tasks

• Speculative copies consume extra resources

T3

Opportunity Cost

T2

time

Slot 1

Slot 2

Slot 3

T1

50 109

Is speculation worth the

payoff?

T1

Roadmap

• Two extreme approaches

• Switching between the two approaches

• Evaluation

Greedy Scheduling (GS)

• Schedule the task that greedily improves accuracy, i.e., finish earliest

T1

T2 T3time

Slot 1

Slot 2

10 3

T4 T5 T6

6

T7

Task ID T1 T2 T3 T4 T5 T6 T7 T8 T9

trem 5 --- -- --- --- --- --- --- ---

tnew 2 --- 1 1 1 1 1 1 3

T1Deadline = 6

Accuracy = 7/9

Resource Aware Scheduling (RAS)

• Speculate only if (c)trem > (c+1)tnew

time

T1

T2 T1

Slot 1

Slot 2

10

T6

T3

T4

T5

3 6

T7

T8

Task ID T1 T2 T3 T4 T5 T6 T7 T8 T9

trem 5 --- -- --- --- --- --- --- ---

tnew 2 --- 1 1 1 1 1 1 3

T1Deadline = 6

Accuracy = 8/9

GS vs. RAS

T1

T2 T3time

Slot 1

Slot 2

10 3

T4 T5 T6

6

T7

T1Deadline = 6

Accuracy = 7/9

time

T1

T2 T1

Slot 1

Slot 2

T6

T3

T4

T5

T7

T8

T1Deadline = 6

Accuracy = 8/9

Deadline = 3

Deadline = 3

Accuracy = 3/9T1

Accuracy = 2/9

GS

RAS

How to pick between GS and RAS?

• When to consider opportunity cost?• Analytical model [TBD]

Guideline

• Be “conservative” early in the job, and “aggressive” near its completion

For jobs with >2 waves of tasks use RAS, otherwise use GS

GRASS = GS + RAS

• Start with RAS and switch to GS when 2 waves of tasks remain

• How to estimate two remaining waves?– Wave boundaries are not strict– Task durations are uncertain– Wave-width is not constant

Learning switching point in GRASS

• GRASS starts with RAS and switches to GS close to the deadline/error-bound

• Learn the switching point from completed jobs– Generate GS-only and RAS-only samples

• Switch when the GS-only samples are better for the remaining deadline

GRASS Scheduling

• GS Greedy Scheduling• RAS Opportunity cost aware Scheduling

• Switch from RAS to GS close to deadline/error-bound

• Learn switching point

Implementation Details

• Implemented inside Hadoop 0.20.2 and Spark 0.7.3– Modified Fair Scheduler

• Task Estimators– trem is extrapolated from remaining data; progress

reports at 5% intervals– tnew is estimated from completed tasks

Evaluating GRASS

• Workload from Facebook and Bing traces– 1000’s of nodes, Hadoop and Dryad jobs– How was it generated?

• Baselines: LATE or Mantri

• 200 node EC2 deployment; trace-driven sim.

Deadline-bound Jobs

Error-bound Jobs

Value of Switching

Related Work

Conclusion