EICAR 2009, 12 May 2009

Checkvir Realtime Anti-Malware Testing and Certification

Dr. Ferenc Leitold, Veszprog Ltd. fleitold@veszprog.hu

www.checkvir.com

EICAR 2009, 12 May 2009

Contents

• Purpose of Checkvir testing

• Testing methodology– Technical background– Testing procedures

• Current state

• Difficulties

• Questions

EICAR 2009, 12 May 2009

Purpose of Checkvir testing

AVG 1,7

ESET 2,6

F-PROT 1,2

F-Secure 5

Kaspersky 23,2

McAfee 35,4

Panda 44,7

Sophos 5,4

Sunbelt 0,6

Symantec 233,4

VirusBuster 1

source: AV-Test.org

Number of updates / dayProblems:

Big number of updates

Cloud technology

Solutions are continually changing

Testing all versions are impossible

EICAR 2009, 12 May 2009

Purpose of Checkvir testingTesting all versions are impossible

Executes tests as frequently as possible• Automatic methods have to be developed• Big number of computers have to be used

EICAR 2009, 12 May 2009

Purpose of Checkvir testing

The main purposes:• Provide reliable, correct and exact information

mainly about:- effectiveness- performance

in a balanced way

(AMTSO’s principle)

• Provide naming cross-reference information

performanceeffectiveness

EICAR 2009, 12 May 2009

Pack and save the new image

AV update

Unpack previous image

Publish results

Unpack last image

New version?

Initialize testing

Execute test(s)

Save results and reports

Analyze results

yes

no

Testing methodologyupdate test

EICAR 2009, 12 May 2009

Testing methodologyTechnical background

clients

“malware proxy” serverwebserver

controller

archiver

firewall

firewall & router

EICAR 2009, 12 May 2009

Testing methodologyTesting procedures

• Malware knowledge (detection, disinfection)– against known, unknown malware and clean files

– on-demand, on-access and proactive executions

• “Container” checking capabilities– archives, email clients’ data files, …

• Speed– on-demand, on-access

– boot time

• Functionality• Stability• …

speed

knowledge

EICAR 2009, 12 May 2009

Testing methodologyTesting procedures

Why the speed is so important?

EICAR 2009, 12 May 2009

Testing methodologyTesting procedures

EICAR 2009, 12 May 2009

Testing methodologyTesting procedures

Testing bootup time

What is more important?

BOOTUP TIME or SECURE BOOTING

DEMO

EICAR 2009, 12 May 2009

Testing methodologyTesting procedures

EICAR 2009, 12 May 2009

Testing methodologyTesting procedures

Avast

AVG

Avira

Bitdefender

Eset

e-Trust

F-Prot

F-Secure

Fortinet

Ikarus

Kaspersky

Microsoft

Rising

Sophos

Symantec

Trend Micro

VirusBuster

Bootup protection test

EICAR 2009, 12 May 2009

Testing methodologyTesting procedures

Avast

AVG

Avira

Bitdefender

Eset

e-Trust

F-Prot

F-Secure

Fortinet

Ikarus

Kaspersky

Microsoft

Rising

Sophos

Symantec

Trend Micro

VirusBuster

Bootup protection test

EICAR 2009, 12 May 2009

Testing methodologyTesting procedures

Avast

AVG

Avira

Bitdefender

Eset

e-Trust

F-Prot

F-Secure

Fortinet

Ikarus

Kaspersky

Microsoft

Rising

Sophos

Symantec

Trend Micro

VirusBuster

Bootup protection test

EICAR 2009, 12 May 2009

Testing methodologyTesting procedures

Avast

AVG

Avira

Bitdefender

Eset

e-Trust

F-Prot

F-Secure

Fortinet

Ikarus

Kaspersky

Microsoft

Rising

Sophos

Symantec

Trend Micro

VirusBuster

Bootup protection test

EICAR 2009, 12 May 2009

Testing methodologyProactive tests vs. AM cloud technology

Problems:

• AM products use cloud technology– > traffic should be allowed

• Malware use cloud technology– > traffic should be allowed– > How can we protect the world?– > How can we provide exactly the same

environment for solutions?

EICAR 2009, 12 May 2009

Testing methodologyProactive tests vs. AM cloud technology

clients

“malware proxy” serverwebserver

controller

archiver

firewall

firewall & router

EICAR 2009, 12 May 2009

Testing methodologySettings

• By default, DEFAULT settings are used

• Minimal functionality is required: – Execute tests without user interaction– Automatically clean the infected file

(if not possible -> delete)

– Report file generation

EICAR 2009, 12 May 2009

Current state

What is working now?

• The frame system

• The website

• Automatic procedures of some products

• Preliminary selection and validation of the samples

EICAR 2009, 12 May 2009

Current state

EICAR 2009, 12 May 2009

Current state

EICAR 2009, 12 May 2009

Current state

EICAR 2009, 12 May 2009

Difficulties

• Viewpoint of the average user Automatic methods

• Testing environment

• Funcionality problems– Truncate report file

• Stability problems

EICAR 2009, 12 May 2009

Questions