The implications of Big Data for

BTS and COS

George Kershoff

Presented at the 7th joint EC-OECD workshop on “Recent developments in Business and Consumer Surveys” held in Paris on 30 November and 1 December 2015

Outline

− What is Big Data?

− Implications of Big Data for BTS and COS

– Competitive advantage of Big Data

– Competitive advantage of BTS and COS

− Purpose of this paper and presentation:

– Create awareness and stimulate discussion

– Allow producers of BTS and COS to pre-emptively take action

What is Big Data?

− Large datasets created through multiple technologies for disparate purposes in real time

– Internet search terms

– Twitter feeds and other internet sources (e.g. news, datasets)

– Sensors (e.g. mobile phones, personal fitness monitors, vehicle trackers, traffic loops / toll gantries)

– Administrative records

– Private sector (proprietary) datasets− Active: loyalty cards; internet transactions and usage

− Secondary: crawling / scraping; datafication of content, merging of multiple datasets

− Data from back-end operations

− Fuzziness be specific

Examples of how Big Data from the private sector is used to monitor the macro economy

Consumer confidence Datafied tweets and news MarketPsych

Business confidence / GDP growth

Interbank payments messages SWIFT

Vehicle traffic data INRIX

Business activity Pallet movements retail sales CHEP

Satellite imagery economic activity / trade SpaceKnow

PricesWeb Crawling Billion Prices

Taking pictures of products and their prices Premise

EmploymentMerging various data sources and web crawling Real Time M

Job searches Indeed

Real Estate Merging various private and public data sources Zillow

Analogue era (BTS & COS) Digital era (Big Data)

SizeScarce Ample / overwhelming

Aggregate Granular

Availability After a (short) lag in time In real-time

Collection methodActive Passive

Random sampling Exhaustive / everyone

Data attributes Organised Messy

Production costs Expensive Low relative to size of the data

Research designStatisticians and economists Data scientists and engineers

“See what you asked for” “Ask what you see”

InterpretationReadily available Scarce

Causation Correlation

Barriers to entry High Low

Implications: competitive edge of Big Data

Analogue era (BTS & COS) Digital era (Big Data)

SizeScarce Ample

Aggregate Granular

Availability After a (short) lag in time In real-time

Collection methodActive Passive

Random sampling Exhaustive / everyone

Data attributes Organised Messy

Production costs Expensive Low relative to size of the data

Research designStatisticians and economists Data scientists and engineers

“See what you asked for” “Ask what you see”

InterpretationReadily available Scarce

Causation Correlation

Barriers to entry High Low

Implications: competitive edge of BTS & COS

Analogue era (BTS & COS) Digital era (Big Data)

SizeScarce Ample

Aggregate Granular

Availability After a (short) lag in time In real-time

Collection methodActive Passive

Random sampling Exhaustive / everyone

Data attributes Organised Messy

Production costs Expensive Low relative to size of the data

Research designStatisticians and economists Data scientists and engineers

“See what you asked for” “Ask what you see”

InterpretationReadily available Scarce

Causation Correlation

Barriers to entry High Low

Other competitive edges of BTS and COS

− Measure expectations

− Historical time series

− Micro data

Copyright for this presentation is held by Stellenbosch University. Although great care is exercised to record and interpret all information correctly, Stellenbosch University, its division BER and the author do not accept any responsibility for any direct or indirect loss that might result from accidentally inaccurate data and interpretations by third parties. Stellenbosch University further accepts no liability for the consequences of any decisions or actions taken by any third party on the basis of information provided in this presentation. The view, conclusions or opinions contained in this presentation are those of the author and do not necessarily reflect those of BER or Stellenbosch University.