WSO2Con US 2013 - Keynote: APIs Driving the Stubhub Business

APIs driving StubHub Business

Sastry Malladi Chief Architect

StubHub

Ø  An eBay owned company Ø  Over 25 million users and growing Ø  We sell one ?cket per second Ø  ~8.5 million page views a day, on an average Ø  ~ 3 million addi?onal page views per day on Mobile devices Ø  ~10 M ?ckets for sale in sports, concerts and others. Ø  ~ 1 TB of data processed monthly by the analy?cs infrastructure

– This number will significantly go up as we bring in data from many of the unstructured data sources

Ø  ~300 Million SQL execu?ons/day

Some Fun Facts about StubHub

Ø Our Business goal Ø API centric Architecture Ø Challenges Ø Our approach Ø  Leveraging Big Data Platform Ø Summary

Overview

Our business goal

Make StubHub the worldwide destination for an end-to-end experience for all fans – this

includes discovery, access and sharing post event experiences

Ø  We need to build an echo system and developer community to drive more traffic to our business (not necessarily website)

Ø  Integrate with many partners in the life cycle of discovery and event going experience

Ø  Go to places where people/apps are as opposed to expecting them all to come to our site

Ø  But … Ø  We had a monolithic architecture/codebase …

To achieve that goal …


Overview

7

TODAY TOMMOROW

FUTURE

BriQle, hardwired and monolithic applica?on

Shared, reusable components

Domain decomposition Domain decomposition / Service Orientation

The functional interface is a contract between two blocks of lego and is referred to as a Service

Service

Ø Service Ø  Internal terminology Ø Re-usable component with a well defined

functional contract

Service vs. API

Ø  API

Ø  Externally exposed “Service” – external developer terminology Ø  Includes a developer program Ø  Typically has a “func?onal” as well as a “non-‐func?onal” (e.g SLA)

contract Ø  Poten?ally orchestrated across mul?ple services

API Centric Architecture

Ø  An API is a business capability delivered over the network to internal or external consumers Ø  Network accessible func?on Ø  Available using standard web protocols Ø  With well-‐defined interfaces Ø  Designed for access by third-‐par?es

Ø  A Managed API is:

Ø  Ac?vely adver?sed and subscribe-‐able Ø  Available with SLAs Ø  Secured, authen?cated, authorized and protected Ø  Monitored and mone?zed with analy?cs

11

API Centric Architecture - Increases Business Benefits

Ø  Business agility Ø  Faster ?me to market on building brand new experiences -‐ mobile,

tablet, etc. Ø  Faster ?me to market on user experience changes on the site

Ø  Seller Benefits Ø  Sellers have flexibility to build their own customized solu?ons Ø  Scales well to inventory and traffic from large brokers

Ø  Buyer Benefits Ø  Faster ?me to market on partner integra?ons Ø  Opens new door on partner integra?ons Ø  Allows developer community to extend SH fan experience

12

API Centric Architecture – Builds Scale for Organization

Ø  Productivity Ø  Parallel development Ø  Parallel deployment Ø  Reduced complexity of the code Ø  Increased maintainability of the application

Ø  Testability Ø  Easier to automate testing for smaller components Ø  Easier to identify and triage bugs Ø  Increases overall quality of product

API Brands Enhance revenue

API brands build mindshare

Mindshare increases visibility

Visibility fosters discovery and evalua?on

Evalua?on triggers collabora?on and API adop?on

API adop?on builds community


Overview

Challenges

Ø  Changing / re-‐architec?ng a running business (mid-‐flight) Ø  Consump?on paQerns are not fully baked at the ?me of building

an API Ø  Dealing with cross domain calls – CORS Ø  SLAs and Chargeback models aren’t clearly published / baked in

most cases Ø  Preven?ng Fraudsters leveraging the API, and impersona?ng

and collec?ng confiden?al data Ø  Products in this space are Maturing and constantly evolving


Overview

Ø  Do the domain decomposi?on and build the APIs, with a thin UI layer for crea?ng experiences

Ø  Re-‐Build Na?ve mobile apps and a unified experience for mobile and desktop

Ø  Use WSO2 ESB and API Manager to Publish, Protect and manage the APIs Ø  For CORS problem

Ø  Using a combina?on of F5 rules and API gateway mediators to support CORS protocol

Ø  For Fraud preven?on Ø  Use risk scores and profiles based on contextual data sent along with API requests Ø  Analysis of the paQerns using Big Data plaform

Approach overview

Approach to domain decomposition

User Inventory

Catalog

Search Purchase

Reco Payment

Domains

Ideal decomposition Intersection of any two domains = 0

Sum of all domains = StubHub / Enterprise

Stubhub as an enterprise

Service

Domain Meta Model

*In this diagram Functional Domain = Functional Area Domain Architect = Architect responsible for a Functional Area Domain Model = “Domain Model” for “Stubhub

Domain Model: Functional view

End-‐state Architecture

Purchase Inventory Account Mgmt Fulfillment Services/

APIs layer

Data en<ty services (user, events, <ckets) with caching Data Access Layer

Presenta<on layer

Website mWeb Mobile Apps Partner Apps

Data

API gateway

User Others

APIs allow innovation, business agility and operational excellence

WSO2 API Gateway: Roles & Functions

Publisher Store / Subscriber

Identity Server Gateway

²  API - CRUD ²  API Lifecycle

²  Manage Apps ²  Subscribe To APIs ²  Generate App

Keys

²  User Authentication ²  Key Management ²  JWT Assertion

²  Routing Rules ²  Rate Limiting ²  Authorization

External API Invocations

Developer portal


Overview

APIs needed for these use cases

Personaliza<on / Recommenda<ons •  Buyer Recommenda?ons: Personalized experience •  Seller Recommenda?ons Ticket Pricing

Social Interac<ons •  Support an enriched experience for a Fan for the complete event life cycle – from

discovery to access to post event experience. Business Analy<cs

•  Making data available for tradi?onal business analy?cs repor?ng, including fraud analysis

Customer 360 / Insights •  A mul?-‐faceted view of a customer, including buying and / or selling preferences,

sea?ng preferences, pricing preferences, preferred events, friends, likes/dislikes, transac?on history etc.

Ø  Many external and internal data sources Ø  Both structured, semi-‐structured as well as unstructured data Ø  Feeding the aggregate data back into OLTP environment as real-‐

?me as possible Ø  Aggregate data is served through the tradi?onal EDW, but not

all data (e.g. graphs) can be represented there. Ø  Building a platform for the future while continuing to

support existing platform/tools Ø  Re-platforming a 10 year old data platform patched up

over the years

Challenges

Ø  Structured Ø  Transactional Data (orders, listings) Ø  Catalog data Ø  CS / Siebel HR data

Ø  Semi to Unstructured Ø  Click stream data Ø  External user segmentation data Ø  External survey data Ø  Email data Ø  Social data Ø  Meta classification data Ø  Referral data

Few Example data sources

Big Data Platform enabling APIs

ETL Data Adapter

1 Data

Adapter 2 Data

Adapter N

EDW

Data Sources: Structured / Unstructured

Transac<ons Click Stream Social weblogs Catalog …….

Hive oozie Pig

HDFS Mahout

sqoop MapReduce

HBase

Solr

Big Data Platform

Solr

HBase

Website/Mobile apps/partner apps

Reco svc

other Replication

OLTP world

Analytics world

Analytics engine (R / Mahout)

BI Tools

Sandbox data

Replication


Overview

•  Building on domain models and API centric architecture is a key enabler for business agility and growth

•  But there are challenges, as is the case with any ini?a?ve. The good news is that you are not alone.

•  Most of the challenges can be overcome. Need to follow quick “test and learn” itera?ve approach.

•  Laying down a solid founda?onal architecture upfront helps to a great extent.

•  Leveraging Big Data comes in handy for many use cases, and helps provide appropriate data through the API

Summary

Thank you ! [email protected]

Questions ?

Technology

WSO2Con US 2013 - Keynote: APIs Driving the Stubhub Business