The scraping problem in ticketing - Martin Zetterlund from Sentor

THE SCRAPING

PROBLEM

IN TICKETING

Martin Zetterlund,

Founder of Sentor Managed Security Services

SCRAPING DEFINED

“To automatically interact with a

website through the use of a computer

program.“

We also include manual labor in low

cost countries that are sometimes

utilized to do the work of a computer

program

HOW COMMON IS IT

• Our clients averaged 23 % of

scraped traffic.

• Those hardest hit experienced in

excess of 50 % of scraping traffic to

their sites.

Data from The Scraping Threat Report 2014

SCRAPING: AN INDUSTRY

“The companies stealing your data or scraping your tickets might have more

resources than you expect and they change their behavior in order to bypass

safety methods”

Examples of professional scraping companies that sell their services to

anyone are screenscraper.com and mozenda.com

THE LEGAL SITUATION

• Unclear and fairly complicated

situation. Ticketmaster seem to

have had some success in US and

have an interesting case against

higs tickets right now.

• O‟leary won the right to call resellers

„worthless ticket touts‟ in ads.

OUR CLIENT

• Major ticket vendor based in London

with a world-wide reputation

• Experienced issues with waiting

room engaging due to scrapers

• Negative publicity as well as site

overload

THE PROBLEM

Second hand sales

• Half empty venues

due to tout having

problem selling

tickets

• Bad PR

Site overload

• Will cause more

expensive

infrastructure

• Bad user experience

Functionality issues

• Seats/tickets

unavailable to users

due to bots

occupying them by

use of website

functionality

LESSONS LEARNED

Our client attempted to solve the problem by using manual log analysis to find

IP addresses that hit their site or seemed automated in their behavior.

Time consuming

Reactive

No automated blocking

Old data logs

HOW TO STOP SCRAPING

• Rate limiting

Against non-distributed scrapers

• Captcha

Effects user experience but can

be effective against scrapers

• Blacklisting

IP addresses, User-agents

http://www.scrapesentry.com for

more details

KEY SUCCESS FACTORS

Resources

Needs both dev and

analysis to work together

over time

Technology

Platform needs to be

able to adapt and

evolve over time

Persistence

Problem will not go

away. Staff for long

time.

SCRAPESENTRY IN BRIEF

BENEFITS OF SCRAPESENTRY

• Correlates scraper IP information over a

global network of clients

• Real-time analysis of user behaviour on

website in order to identify scrapers

• Continuously developed to stay ahead

of scrapers

• Supported 24/7 by security analysts and

security operators from the Sentor

Security Operations Centre.

THANK YOU!

martin.zetterlund@sentormss.com

For more information visit: www.ScrapeSentry.com

CONTACT ME: