Learning to remove Internet advertisements

Dec 6, 2004 12004 Michigan Technological University

Nicholas KushmerickNicholas Kushmerick

Department of Computer Science,Department of Computer Science,

University College Dublin, IrelandUniversity College Dublin, Ireland

Learning to remove Internet Learning to remove Internet advertisementsadvertisements

Presented by Bo ZhangDepartment of Computer Science Michigan Technological University


OverviewOverview

BackgroundBackground

Introduction of ADEATERIntroduction of ADEATER

Design of ADEATERDesign of ADEATER

EvaluationEvaluation

Related WorkRelated Work

Conclusion and Future WorkConclusion and Future Work


BackgroundBackground Negative Impact of advertisement images on InternetNegative Impact of advertisement images on Internet

Slow down the speed of browsing Consume resources of computer Extra costs for users

Advertisement Image

Advertisement Image

Advertisement Image


Introduction of ADEATERIntroduction of ADEATER

Definition:Definition:

- A browsing assistant that automatically removes advertisement images from Internet pages.

Property:Property:

Rules generated from learning algorithm


Introduction of ADEATERIntroduction of ADEATER ExamplesExamples


Design of ADEATER Design of ADEATER

System ArchitectureSystem Architecture


Design of ADEATERDesign of ADEATER Encoding instanceEncoding instance

Fixed–width feature vector

Images enclosed in anchor tag <A> is a candidate advertisement

Geometric features of an image: -Height <IMG height=90> -Width <IMG width=90> -Aspect ratio (ratio of width to height)

Local feature: -Whether destination URL and image URL are in the same internet

domain www.ee.mtu.edu/page.html www.cs.mtu.edu/image.jpg YES

www.dell.com/notebook.html www.cs.mtu.edu/image.jpg No



Encoding instanceEncoding instance


Caption feature: -Words occuring in enclosing <A> tag with phrase length<K

and phrase count >M -K is maximum phrase length -M is minimum phrase count

Alt Feature -Set of “alternate” words in the <IMG> tag (<IMG alt=“ad”>)

with phrase length<K and phrase count >M -K is maximum phrase length -M is minimum phrase count





Ubase, Udest, Uimg

-Words occuring in base URL, destination URL, image URL with phrase length<K and phrase count >M -K is maximum phrase length -M is minimum phrase count

Stop list -Low-information terms (“http”, “www”, ”jpg”, etc.)




Samples of HTML page



Encoding of samples



Encoding of samples (cont)



Gathering examplesGathering examples

AD samples are generated by ADGRABBER browsing assistant

Identifier candidate advertisements

Generate vector encoding

NON-AD samples are generated by a custom-built Internet spider

Extract images from randomly-generated URLs.



Learning rules

Algorithm - C4.5 decision tree learning algorithm

Properties - Quick on-line execution of classifier - Not be overly sensitive to missing features or noises - Scale well and insensitive to irrelevant features

Examples of rules - If aspect ratio > 4.5833, alt doesn’t contain “to” but does

contain “click+here”,and Udest doesn’t contain “http+www”, then instance is an AD

- If Ubase does not contain “messier”, and Udest contains the “redirect+cgi”, then instance is an AD



Removing advertisementsRemoving advertisements

Process

- Fetch HTML pages from Internet - Identify candidate advertisements - Classify instances with learned rules - Replace the image’s URL with the URL of an inconspicuous low-bandwidth image

Implementation

- Removal module as a proxy server


Evaluation

Speed and accuracySpeed and accuracy

Experiment setting

Total samples - AD: 459 examples

- NON-AD: 2820 examples

10-fold cross-validation - Training set: 90% examples - Test set: 10% examples

Off-line training phase: 5.8 minutes

On-line classification phase: 70 msec/image

Average accuracy: 97.1%


Evaluation Learning curvesLearning curves

Simple methodology - Not recalculate feature set Realistic methodology - Recalculate feature set


Evaluation

Alternative encodingsAlternative encodings


Related Work Muffin: Filtering web pages

ImageKill Filter: Hand-crafted rules

ImageKill.minheight

- Only remove images which are at least n pixels high

ImageKill.minwidth

- Only remove images which are at least n pixels wide

ImageKill.ratio

- Remove images which are more than n times as wide as

they are high

ImageKill.exclude

- Don't remove images that match the given string/regexp


Related Work

WebFilter: Filtering web pages

Solution

- User provides a list of URL templates and corresponding

filter scripts


Related Work

Junkbuster: Junkbuster: Filtering web pages

Solution

- User provides a block file


Related Work

Smokey: Detect abusive messagesSmokey: Detect abusive messages

Solution

- Training samples and generate rules by training - Parse messages and generate feature vector - Classify the feature vector with rules generated


Conclusion and Future Work

ConclusionConclusion

High accuracy

Modest resource cost (processing time, training samples)

Future WorkFuture Work

Incremental learning algorithm

More efficient feature selection mechanism


Thank you!Thank you!

Documents

Learning to remove Internet advertisements