12
©Accordant Media 2013. All Rights Reserved BigData@Accordant Media June 17 2013 Balaji Rao Director, Technology

Big data@accordantmedia - oanyc summit

Embed Size (px)

Citation preview

Page 1: Big data@accordantmedia - oanyc summit

©Accordant Media 2013. All Rights Reserved

BigData@Accordant MediaJune 17 2013Balaji RaoDirector, Technology

Page 2: Big data@accordantmedia - oanyc summit

©Accordant Media 2013. All Rights Reserved. 2

Accordant Media is an independent programmatic media buying specialist, focused on digital media buying and optimization.

Page 3: Big data@accordantmedia - oanyc summit

©Accordant Media 2013. All Rights Reserved. 3

Bidding Mechanics

Exchanges Bidders

Bid RequestAsset Information

Bid ResponseMax Offer

40 Billion Events Daily | 200ms per Auction|70 TB a day

Winning BidSent Out

Ad inventory from 500k+ domains sent in

Page 4: Big data@accordantmedia - oanyc summit

©Accordant Media 2013. All Rights Reserved. 4

Amassing Actionable Data

All Media-Driven Activity(53 fields)

Ad Verification Data(35 fields)

• User ID• Date/Time• Creative• Size• CPM• Bid Info• Est. Pricing• Clear Pricing• IP• Geo• OS • Browser• Language

• Inv. Source

• Site• Currency• Advertiser• Campaign• Recency • Frequency• Imp/Click/

Conv• Fees• Revenue• Order ID

Bid-Stream (won & lost)(40 fields)

• User ID• Date/Time• Size• Est. Pricing• IP• Geo• OS • Browser• Language

• Inv. Source

• Site• Age• Gender• Timezone• Within iFrame• Segments

(30,000+)

• User ID• Date/Time• Campaign• Creative• Inv. Source• Site• Site category• Geo• In View?• Time in view• # Views

• Engaged?• Engage Time• Hovers? • Hover Time• X/Y coord. of

Hovers• Alerts

Up to 30,000+ segments for each bid request

Page 5: Big data@accordantmedia - oanyc summit

©Accordant Media 2013. All Rights Reserved. 5

What is the Big Picture ?

Page 6: Big data@accordantmedia - oanyc summit

©Accordant Media 2013. All Rights Reserved. 6

Hosted on leading Cloud Hosting Providers Listening 40~70 TB a day, EMR 2TB a day(on an avg.) 31TB compressed as of May @ ~3 TB per month Polyglot persistence AWS- EMR (Hive/Hadoop), S3, EC2, etc., Redshift, NoSQL,

Columnar DB, SQLServer Google- BigQuery, GCE, Google Storage Anything new, cheap, fast ….• S3, Google Storage (Compressed)

• Partitioned, Year/Month logs at 1 per day Year/Month/Day for logs > 1 per day

• ETL- Talend, Elastic Mapreduce Hadoop/HIVE• Monthly Jobs- Forecasting for partners, Weekly- Audience Audits• Daily- Reporting Summary Daily, Hourly- e.g. Reporting Summary, Segmentation

Technology at a glance

Page 7: Big data@accordantmedia - oanyc summit

©Accordant Media 2013. All Rights Reserved 7

Date Impressions Clicks Conversions Total Cost Client Revenue eCPM eCPC eCPA eRPA CTR Conv Rate Broad Reach 18283318 5041 848 14,952.00 611.73 0.82 2.97 17.63 0.72 0.028% 0.005%

8/10/2010 800254 224 18 533.97 0 0.67 2.38 29.66 - 0.028% 0.002%8/11/2010 2318422 642 140 1,680.68 0 0.72 2.62 12.00 - 0.028% 0.006%8/12/2010 1945529 527 126 1,516.63 159.7 0.78 2.88 12.04 1.27 0.027% 0.006%8/13/2010 2122654 596 68 1,701.33 85.36 0.80 2.85 25.02 1.26 0.028% 0.003%8/14/2010 1893693 576 40 1,497.68 0 0.79 2.60 37.44 - 0.030% 0.002%8/15/2010 1907657 612 67 1,518.79 0 0.80 2.48 22.67 - 0.032% 0.004%8/16/2010 1899805 523 106 1,459.14 185.91 0.77 2.79 13.77 1.75 0.028% 0.006%8/17/2010 1475451 344 82 1,342.23 79.85 0.91 3.90 16.37 0.97 0.023% 0.006%8/18/2010 1770367 437 82 1,737.09 100.91 0.98 3.98 21.18 1.23 0.025% 0.005%8/19/2010 2149486 560 119 1,964.45 0 0.91 3.51 16.51 - 0.026% 0.006%

Broad Reach Driven Remarketing 234269 84 609 409.09 271.34 1.75 4.87 0.67 0.45 0.036% 0.260%8/10/2010 1711 1 0 1.46 0 0.85 1.46 #DIV/0! #DIV/0! 0.058% 0.000%8/11/2010 6591 3 22 5.35 0 0.81 1.78 0.24 - 0.046% 0.334%8/12/2010 10138 5 20 8.29 16.54 0.82 1.66 0.41 0.83 0.049% 0.197%8/13/2010 12059 1 32 10.13 26.6 0.84 10.13 0.32 0.83 0.008% 0.265%8/14/2010 11669 2 38 10.15 0 0.87 5.07 0.27 - 0.017% 0.326%8/15/2010 15157 7 31 13.26 0 0.88 1.89 0.43 - 0.046% 0.205%8/16/2010 24550 13 39 36.15 82.25 1.47 2.78 0.93 2.11 0.053% 0.159%8/17/2010 46890 16 113 100.10 32.78 2.13 6.26 0.89 0.29 0.034% 0.241%8/18/2010 44564 18 145 95.72 113.17 2.15 5.32 0.66 0.78 0.040% 0.325%8/19/2010 60940 18 169 128.48 0 2.11 7.14 0.76 - 0.030% 0.277%

Standard Remarketing 2848262 1723 8012 5,347.48 5962.25 1.88 3.10 0.67 0.74 0.060% 0.281%8/10/2010 127528 60 274 110.17 0 0.86 1.84 0.40 - 0.047% 0.215%8/11/2010 183476 83 520 168.08 0 0.92 2.03 0.32 - 0.045% 0.283%8/12/2010 171385 79 447 159.33 923.25 0.93 2.02 0.36 2.07 0.046% 0.261%8/13/2010 147360 78 349 137.16 569.47 0.93 1.76 0.39 1.63 0.053% 0.237%8/14/2010 121316 65 240 114.55 0 0.94 1.76 0.48 - 0.054% 0.198%8/15/2010 137568 74 429 130.28 0 0.95 1.76 0.30 - 0.054% 0.312%8/16/2010 259435 141 543 459.98 1546.54 1.77 3.26 0.85 2.85 0.054% 0.209%8/17/2010 527275 358 1663 1,268.31 878.39 2.41 3.54 0.76 0.53 0.068% 0.315%8/18/2010 546179 359 1793 1,328.21 2044.6 2.43 3.70 0.74 1.14 0.066% 0.328%8/19/2010 626740 426 1754 1,471.42 0 2.35 3.45 0.84 - 0.068% 0.280%

Grand Total 21365849 6848 9469 20,708.57 6845.32 0.97 3.02 2.19 0.72 0.032% 0.044%

Client Reporting: 24/7 Access

Client Client

Client Client

Client Client

Client

Client

Client

Client

Page 8: Big data@accordantmedia - oanyc summit

©Accordant Media 2013. All Rights Reserved 8

Proprietary Forecasting Tool

Accordant’s proprietary tool can forecast given highly dynamic targeting parameters

Page 9: Big data@accordantmedia - oanyc summit

©Accordant Media 2013. All Rights Reserved 9

BigData Visualized- Audience Audit

Page 10: Big data@accordantmedia - oanyc summit

©Accordant Media 2013. All Rights Reserved 10

BigData Algorithm (Re)Model

Campaign Parameters Campaign Past Performance

Bid StreamExchange1st Party2nd Party3rd Party (With Data Provider Agreement)DomainsLookalikes

Past Performance

Campaign (re)model

Algorithm models tell the bidders to bid (and at what price) or not to bid.Campaigns are continuously re-modeled for optimum performance

Page 11: Big data@accordantmedia - oanyc summit

©Accordant Media 2013. All Rights Reserved 11

BigData –Lessons Learned

• Pay as you go vs Huge upfront cost• Don’t get locked down

• Choose technology wisely • Keep tech babble away

• Try new technologies • Some will fail

• Keep analysis on BigData simple• Deep analytics fail

Page 12: Big data@accordantmedia - oanyc summit

©Accordant Media 2013. All Rights Reserved 12

Q & A

• Thank You!

• (…And we’re hiring)!• [email protected]