BDM - project management in big data context.pptx

Great Minds Search Alike™By Jean-Louis QuéguinerSenior Big data Developer at Wajam

Agile Project Management in a Big Data context

Great Minds Search Alike™

Wajam is a leader in social search and social advertising technology.

Wajam gives you recommendations from friends on Google, Bing, eBay, Amazon, Wikipedia, TripAdvisor and more.

What is Wajam?

© Wajam 2015

1. Context

2. Architecture

3. Proposed process

CONTENT

© Wajam 2015

● 350 GB of logs every day● 7-10 Millions active users● 20 Millions of ads every day● More than 200 Specific events● Around 1 to 2 events deployed every week

CONTEXT

© Wajam 2015

ARCHITECTURE

© Wajam 2015

PROPOSED PROCESSBacklog Sync Dev Test Eval

SizeAdapt

storageBackward

CompatibiltyMerge & Reload

Done

Sync : 1- Define with BI their needed output2- Define with Dev our needed input

Eval Size:1- Will it fit in queue?2- Will it fit in HDFS

Adapt Storage : 1- file structure2- adapt MySQL columns / index/ tables

Backward Compatibilty:If output structure changed &If output(t) is used as input for t+1then adaptation is needed

© Wajam 2015

PROPOSED MARKER

NAME + PRIORITY

TASK NAME + DESCRIPTION

ADD MOUSE ON HOVER AD EVENT

ALEX 1

© Wajam 2015

PROPOSED PROCESSBacklog Sync Dev Test Eval

SizeAdapt

storageBackward

CompatibiltyMerge & Reload

Done

TODO 1

TODO 2

TODO 3

A 1 A 2

A 3

J 1J 2J 3

© Wajam 2015

CONTINUOUS IMPROVEMENT & CONCLUSIONS● Re-evaluate priorities every morning● Re-communicate priorities every morning● Custom Steps● Parallel task tracking● Help focus● Easy to track● More detailed and flexible than classic scrum approach

THANK YOU

© Wajam 2015