A Crawler-based Study of Spy-ware on the Web Lingfeng Mo

Spy-ware on the Web

A Crawler-based Study

A Crawler-based Study of Spy-ware on the Web

Lingfeng Mo

Spy-ware on the Web


Malicious spy-ware poses a significant threat to desktopsecurity and integrity. But we talk about the threat froman Internet perspective. Using a crawler, they performed alarge-scale study of the Web, sampling both executables

and conventional Web pages for malicious objects.

Spy-ware threaten

Spy-ware on the Web


Spy-ware has become the Internet’s most “popular” download. A recent scan performed by AOL/NCSA (America Online and the National Cyber Security Alliance ) of 329 customers’ computers found that 80% were infected with spy-ware programs.

Results : 1: In a May 2005 crawl of 18 million URLs, Spy-ware in

13.4% of the 21,200 executables.2. Scripted “drive-by download” attacks in 5.9% of the

Web pages which were processed.

Introduction

Spy-ware on the Web


Where Spy-ware comes from?

Spy-ware typically installs itself through one of two following methods:1, download software to which piggy-backed spy-ware code has been attached->file-sharing software2, visit a Web page that invisibly performs a “drive-by download” attack, the user’s browser will install software without the user’s consent.

Spy-ware on the Web


Method of this study:

1: Examine executable file content for piggybacked spyware programs and Web pages for drive-by download attacks.

2: Analysis spyware (a) which areas of the Web are most infected(b) Which fraction of spyware that contains malicious

functions (modem dialing, Trojan downloading).3: how spyware on the Web has changed overtime.

Spy-ware on the Web


What this study did for Examine executable file ?

1. Determine whether a Web object contains executable software. How to determine?

2. Download. install and rum executable files in VM. (Problems)3. Using commercial anti-spyware tool ?= executable contain piggy-

backed spyware found by our Web crawler. How to do it?Study spyware from:

1: executable Web content contains spyware programs2: Web pages contain enbeded drive-by download attacks.

Spy-ware on the Web


How to determine?

1. If Content-type HTTP header provided by the Webserver when downloading the object was associated withan executable (e.g., application/octet-stream)

2. Extension with executables and installers (e.g., .exe, .cab, or .msi).

3. Also looked for well-known signatures at the beginning of the file to identify its type.

If (exe.) {analyze}; else {not analyze}; Problem?

Spy-ware on the Web


Problem in finding EXEs:1. If EXE embedded in archives (ZIP,RAR)?Solution: download->extracted EXE. From extension

information2. EXE whose URLs are hidden in Java Script.Solution: use web crawler scan and find URL.If find any->add they to a list of pages to crawl.

GO TO SECOND STEP

Spy-ware on the Web


Problems in installation:

1: Installation frameworkSolution: develop a tool that can Auto-click simple button (eg,. Next, install) .2: Forms (E-mail, name, company)Solution: Auto-fill form with some dummy information

GO TO THIRD STEP

Spy-ware on the Web


1. Run anti-spyware tool (Lava Software AdAware) in a clean VM.

2. Collect infection analysis from its emitted logs.3. Use log information to identify which spyware program

were installed.4. Using online databases of spyware and also manually

classified which function those spyware program contained. (Keystroker logging, Adware, Trojan backdoors,

or browser hijacking)

Limitation of this analysis method?

Spy-ware on the Web


Limitations of this analysis method:1. AdAware can only detect spyware programs that have

signatures within its detection database, that is, our analysis misses spyware programs that AdAware does not find.

2. We only collected information about spyware software that is installed. Though many anti-spyware tools such as AdAware also identify malicious cookies or registry entries as spyware threats, we excluded these.

How to Analysis spyware?

Spy-ware on the Web


Web Crawling

Tool: Web Crawler (Heritrix: available at: http://crawler.archive.org/)

1. Crawled sites from 8 different categories in order to understand how spyware had penetrated different regions of the Web (adult entertainment sites, celebrity-oriented sites, games-oriented sites, kids’ sites, music sites and etc.)

2. Some random sites.

How spyware on the Web has changed over time?

Spy-ware on the Web


How spyware on the Web has changed over time?

1. Do the same above both in May and October.

2. Update the spyware database of Adaware.

Spy-ware on the Web


Result:*While the absolute number of spyware-infected executables dropped

substantially between the crawls, this is due primarily to a single site whose number of infected executables declined from 1,776 in May to 503 in October.

*Overall, about 1 in 20 of the executable files we crawled ontained spyware, an indication of the extent of the spyware problem.

Spy-ware on the Web


Spyware prevalence

Spy-ware on the Web


Top 10 spyware programs and sites.

Spy-ware on the Web


Spy-ware on the Web


Are some Web categories more dangerous than others?

Spy-ware on the Web


What kinds of spyware do we find?

Spy-ware on the Web


Number of spyware programs installed

Spy-ware on the Web


Defense:

1: signature-based tools

2: construct blacklists of URLs or domains that are suspected to contain spyware

Spy-ware on the Web


Can signature-based tools keep up?

Re-analyzed all of the spyware-infected executables we foundin October, but using the older AdAware signature databasethat was available in May. If new spyware threats werereleased between May and October, this older version ofAdAware might not have signatures that match them.

RESULTS SEE TABLE 5

Spy-ware on the Web


Conclusion: it is important to keep an anti-

spyware signature

database up-to-date

Spy-ware on the Web


How effective is blacklisting?

overall conclusion we can draw is that blacklists are ineffective in two ways: many blacklisted sites contain no spyware, and many nonblacklisted sites do contain spyware.

Spy-ware on the Web


Continue compare data:

Conclusion: blacklisted spyware programs tend to contain a greater fraction of keyloggers, dialers, and Trojan downloaders. It appears as though blacklists tend to focus on spyware that contains more dangerous functions.

Spy-ware on the Web


Summary

1: Our results show that spyware piggybacked on executables isa significant threat. 2: 1 in 20 of the executables we identified were infected with spyware in our October crawl – a surprisingly high fraction.3: some Internet zones, such as game or celebrity sites, have a higher incidence of executable spyware than others. While some changes have occurred in the time between our crawls, at a high level they show a similar level of risk to Web users.

Spy-ware on the Web


Limitations:

1. Did not crawl the entire Wen -> base on sampling.2. Can not explain any relationship between density and thepresence of threats on the desktop, since the latter is basedon the behavior of real users.3. Limited by anti-software (Adaware).

Spy-ware on the Web


Comparing LBP to other local descriptors

Why the tested methods work well with the easiest fb probe set?

They are robust with respect to variations of facial expressions, whereas the results with the fc probe set show that other methods than LBP do not survive changes in illumination.

Spy-ware on the Web


References:

1. Alexander Moshchuk, Tanya Bragin, Steven D. Gribble, and Henry M. Levy. A Crawler-based Study of Spyware on the Web, 2005

2. Ad-Aware. http://www.lavasoftusa.com/software/adaware/.3. Internet Archive. The Heritrix web crawler project. http://crawler.archive.org/.4. Webroot Software, Inc. Automated threat research.

Described at http://research.spysweeper.com/automated_research.html.

Documents

A Crawler-based Study of Spy-ware on the Web Lingfeng Mo