Day Zero Security Erick Engelke University of Waterloo erick@uwaterloo.ca OTHELLO

Day Zero Security

Erick Engelke

University of Waterloo

erick@uwaterloo.ca

OTHELLO

CANHEIT | Power Through Collaboration | May 27-30, 2007 | OTHELLO

The Problem

Viruses and other malware, hackers

Antivirus products only catch viruses that have been discovered, analyzed and confirmed, and then updates tested and finally distributed worldwide

We are seeing an increase in the number of DAY ZERO viruses, ones not yet protected by the antivirus companies

Kapersky: multiple updates daily, not fast enough

Also hackers – leave non-virus tools like VNC, services

CANHEIT | Power Through Collaboration | May 27-30, 2007 | OTHELLO

Manual Virus Discovery

Observe a problem or symptoms

Manually sort through programs, services, DLLs– processes and files on disk– Comparing filenames with known MS filenames– Comparing remaining files with web filename databases– Looking for suspicious file sizes, date stamps

Executables don’t have to end in EXE or DLL

Not very scientific or automated – tedious work

CANHEIT | Power Through Collaboration | May 27-30, 2007 | OTHELLO

Goal

A mechanism to detect suspicious programs– prior to antivirus vendor issuing new signatures– immune to common hacker tricks like

• filenames similar to MS filenames

• identical filenames, similar path names

• misleading vendor “version” strings

• hacked versions of vendor files

Run once detection – no need for prior baseline (client owned)

Support continuous detection in real-time (servers and more)

CANHEIT | Power Through Collaboration | May 27-30, 2007 | OTHELLO

Recent Day Zero Viruses at UW

December 2006– numerous machines infected – Cerberus (like tripwire) detected the day zero service

January 2007– numerous machines infected– Othello identified two different day zero viruses– the filenames and paths were misleadingly familiar

CANHEIT | Power Through Collaboration | May 27-30, 2007 | OTHELLO

Evidence of Hacking or Viruses

Malware most “useful” when executing in memory– today’s hackers want to “own” machines– as a service which is started on bootup– as an application started at boot or logon time– as a DLL loaded by other processes

Sometimes use a root kit to hide above types– new rootkits are rare– hard to write, other listed file types are easier– F-Secure BlackLight is currently free

CANHEIT | Power Through Collaboration | May 27-30, 2007 | OTHELLO

What We Will Scan

Scan scanning process for suspicious DLLs

Scan all active services

Scan bootup and logon time processes

Scan currently executing processes

CANHEIT | Power Through Collaboration | May 27-30, 2007 | OTHELLO

Othello Strategy

We want to verify that programs were written by someone somewhat trustworthy

At first, we only seem to need to know three things:– Who in the world wrote the software– Whether the author is trustworthy– Whether the software has been modified (hacked) and

thus no longer trustworthy

Aren’t these hard problems? 6½ billion people!

CANHEIT | Power Through Collaboration | May 27-30, 2007 | OTHELLO

Othello

Uses cryptography to verify programs are from trusted vendors

Non-verifiable programs are suspect, and reported back to the user

Verifiable programs are presented, they may still be rogue

Suitable for system administrators and help desks, assist checking computers

Includes button to test with multiple AV engines

CANHEIT | Power Through Collaboration | May 27-30, 2007 | OTHELLO

Cryptography Review

Cryptography is communication in the presence of adversaries

Includes – encryption

• keeping data secret

– signatures • verifying who wrote something

• verifying it has not been changed

CANHEIT | Power Through Collaboration | May 27-30, 2007 | OTHELLO

Public Key Cryptography Review

In public key systems, there are two keys– a published public key

• like a phone number in a phone book

– a private key we use but keep secret

Both keys are very long numbers – impossible for anyone else to guess,

CANHEIT | Power Through Collaboration | May 27-30, 2007 | OTHELLO

Public Key Cryptography

We will use the private key like an old fashioned wax seal on an envelope

anyone can open the message (it’s not encrypted)

the seal verifies who sent it

but possessing one message doesn’t allow you to fake the wax seal on another

CANHEIT | Power Through Collaboration | May 27-30, 2007 | OTHELLO

How We Use Cryptography

Software vendor buys a public/private key set

They use the private key to sign software– They place a special number in the file which solves a tough

mathematical question as only that private key can– anyone can test the key to see if it solves the problem using the

public key, proving the ownership

Hacking the program changes the numbers, so it no longer solves the math problem

CANHEIT | Power Through Collaboration | May 27-30, 2007 | OTHELLO

Our End

If software is signed, we check that it matches the public key

We look up that public key at only a few trusted registries where vendors are allowed to register

Check for key revocation – no longer trusted

CANHEIT | Power Through Collaboration | May 27-30, 2007 | OTHELLO

Who Is Trustworthy

We trust bankers with money

We trust police in emergencies

We do not know the individuals

They undergo background checks before being trusted, and they know they will be held responsible for misdeeds

CANHEIT | Power Through Collaboration | May 27-30, 2007 | OTHELLO

How Can We Trust Software

We must be able to trust someone – can we trust this software author?

That person must be verified by several other trusted people before they are granted a key – all before they were able to sign the software

If any software associated with that person is malware – the registry repeal their trusted status

CANHEIT | Power Through Collaboration | May 27-30, 2007 | OTHELLO

Public Key Infrastructure : PKI

Public key cryptography is only useful with PKI

Certificate Authorities vouch for:– authenticity of the public key (like a phone book)– money has been paid– to some extent: that the individual is responsible

• require proof from several authenticated sources• through possible repeal of the certificates, they can

squash the reputation, cert investment is lost

CANHEIT | Power Through Collaboration | May 27-30, 2007 | OTHELLO

Does Author Sign Signed Software

Usually no!

Many programmers, one signer

Some individual in the company has the responsibility, and signs all the software the company officially releases

Similar to certified accountant who signs off on all financial reports – responsibility is real, company image at stake

CANHEIT | Power Through Collaboration | May 27-30, 2007 | OTHELLO

Revised Goals

PREVIOUSLYWho in the world wrote the software

Whether the author is trustworthy

Whether the software has been modified (hacked) and thus not trustworthy

NOWOnly need to know that software was signed by someone on behalf of a trusted company

That the signer met original criteria, has not been revoked

That the signature is mathematically valid – file not modified

CANHEIT | Power Through Collaboration | May 27-30, 2007 | OTHELLO

Have Code Signatures Failed?

Once: Jan 2001, VeriSign issued two certs to Microsoft impersonator

VeriSign found the problem 6 weeks later– notified Microsoft– posted public notice– used Certificate Revocation List (CRL)

Othello checks for CRL entries

CANHEIT | Power Through Collaboration | May 27-30, 2007 | OTHELLO

OTHELLO: Sample Trusted Programs

CANHEIT | Power Through Collaboration | May 27-30, 2007 | OTHELLO

OTHELLO: Program Without Signature

CANHEIT | Power Through Collaboration | May 27-30, 2007 | OTHELLO

VirusTotal Results

CANHEIT | Power Through Collaboration | May 27-30, 2007 | OTHELLO

Email from Cerberus

Date: Wed, 23 May 2007 21:38:27 -0400From: "cerberus@JUSTINE" <cerberus@JUSTINE.uwaterloo.ca>To: erick@engmail.uwaterloo.caSubject: JUSTINE: Authenticode warnings

Cerberus detected the following Authenticode warnings

No Signature: d:\program files\apache software foundation\apache2.2\bin\httpd.exe Apache2

No Signature: c:\nexus\class\nclasssvc.exe classd

No Signature: d:\program files\apache software foundation\apache2.2\bin\httpd.exe

CANHEIT | Power Through Collaboration | May 27-30, 2007 | OTHELLO

The Bad News

Othello detects existing compromises

Cerberus gives real time compromise detection

Both are “too late” to prevent initial compromises

Both can alert staff to rebuild needs and help determine compromise patterns

CANHEIT | Power Through Collaboration | May 27-30, 2007 | OTHELLO

“Othello” Name

Shakespeare: a tragedy about ill-placed trust

“Othello” sounds like “auth” in Authenticode

CANHEIT | Power Through Collaboration | May 27-30, 2007 | OTHELLO

Downloads

Othello – single use program– no prior baseline required– http://www.eng.uwaterloo.ca/~erick/software/othello

Cerberus– continuous system checking– uses a baseline and reports changes (Email,Syslog)– can be deployed automatically via GPO– http://www.eng.uwaterloo.ca/~erick/software/cerberus

CANHEIT | Power Through Collaboration | May 27-30, 2007 | OTHELLO

Rootkit Detection Downloads

Rootkit Detection– F-Secure Blacklight (Google it) is a free download– the Sysinternals program is too outdated to be useful

CANHEIT | Power Through Collaboration | May 27-30, 2007 | OTHELLO

Thank you