1
Part 6: System Security
Malware Safe Coding Software Trust Virus Detectors Software Signatures “Kernel Integrity Checkers” “Application Integrity Checkers”
2
Computer Systems
Security – keep the systems safe No malware No applications that can be attacked No data stolen No corruption or harmful activity User accounts are not hacked ….. HOW?
Safe softwareVirus checksSandboxing
Integrity checkingCrypto File Systems
Smartcards
3
Safe Coding
Not well understood Software has vulnerabilities, ensure attack free code at
implementation time Stack Guard and Lib Safe and a host of techniques are available Type safe languages are available Not clear if these approaches work
Teach programmers to be aware of attack techniques on software
Safe coding practices
4
Bounds Checking
Array bounds often get exceeded and yet nothing happens Very common in C and C++ code Optimized compilers for all languages disable bounds checking Have to be checked in software, at all points
Buffer overflow of many kinds use this For every data transfer we need to check the size of the
data Function calls Data copying Reading input more
5
Input Validation
Input to a program has to be validated From network From files From user
Almost never done A program “expects” the data to be formatted correctly,
specially if that program generated the data Suppose WORD creates a DOC file When WORD reads it, the data is expected to be in WORD format Simple checks are NOT enough
This vulnerability is ingrained into all legacy software
6
Calling Routines
When routines are called, the caller and the callee must perform sanity checks on arguments
Specially when system calls and library routines are called Many stack and heap smashing attacks use such
vulnerabilities If a program has a call to the “system” routine in Unix If arguments are tampered with, any program can be run by an
attacker
7
Root Access Attacks
“set uid” programs can gain root access Used heavily for Unix Similar features in all operating systems
Many attacks have been designed to misuse this feature in many non-intuitive ways
Databases are particularly attackable via setuid attacks
8
Heap tricks
Put crafted data on heap Most input is read into the heap area anyway
Ensure that a particular pointer to an argument to another function call has been tampered with
Next time the function is called, it will be called with attacked arguments
Hard to detect and protect against
9
Open Source vs. Closed Source
What is safer – Open Source or Closed Source? Open Source:
Many eyes theory When found, vulnerabilities quickly fixed
Closed Source: Stringent code review and coding security policies may be in effect Even if vulnerabilities exist, they may not be found, as the code is
unknown Vendor may have liability
This debate has no obvious outcome, both have advantages and disadvantages
10
Data Execution
Many attacks inject executable code as data and then transfer control to it
Solution is to ensure data is never executed Code resides in code segment, data resides in data segment Code segment must be non-writable Data segment must be non-executable
Hardware support needed NX or XD bits in page tables Existed in the old VAX architecture
11
Software Issues
Software is inherently untrustable Even if source code is visible (e.g. Thompson Attack)
Many techniques for obfuscation Easy to make software do things other than what the source code
seems to do Programmers can inject things into programs that can be
harmful By design By accident
How do we know that software is to be trusted? OS Applications Drivers Downloaded things
No good answer
12
Feature Creep
Features of a program causes unforeseen vulnerabilities e.g. Visual Basic Scripting in Outlook
Why does an email reader have to be programmable? ….because it can be done!
Complex features of programs are attractive to designers and to end users
Impossible to analyze the security impacts Many tricks are then invented by attackers Old adage: Keep it simple
Code will be cleaner Features would be understandable Unforeseen consequences will be minimized
13
Malware
Virus – Trojan – Worm Adware, spyware Rootkits Can be
detectable, but only if detector in
operating system
Not detectable
14
Virus Detectors
Current state of the art in virus prevention is detection of viruses in all incoming files and existing files
Any file that a computer accesses is “scanned” by a virus detector
Hooks to the “open” and other system calls Makes system performance degrade
Scanning via hash matching Needs to know all the hashes of known viruses
Detects known viruses Does not detect changes to existing programs Have to frequently download “virus definitions”
15
Downside of Virus Detectors
Can be easily disabled (via buffer overflow, for example) Rootkits can uninstall it, or disable it
Cannot detect rootkits
Performance is slow New viruses can spread without fear for some time The virus detector is itself a virus?
Yes, but a good one No, but can be, via a little social engineering
Can store hashes of good files, but…. A virus can change the definitions and escape detection
16
Smart Viruses
Polymorphic Viruses Viruses that attack detectors Viruses that hide from detectors
Camouflaged Hidden names
17
Rootkits
Rootkits are kernel patches + processes Processes may run with Admin/root privileges Special routines are embedded in the kernel Many kernel routines may be changed or patched
How to detect Take checksum of kernel Take checksum of routines that may change Look at kernel tables and watch for changes Check for perturbations
18
Undetecatable
If a rootkit is not worried about rootkit detectors then it is detectable
Rootkits can be designed to be evasive If we take hashes of the kernel, it will patch the routine that
computes the hash If we track kernel tables, the tracking routine will be patched
A “new” rootkit detector will find an old rootkit As long as the rootkit writer does not know of the detector and its
techniques A rootkit can patch itself later to become invisible to new rootkit
detector Cat and Mouse game
19
Sandboxing
Run a program with some flags set in the kernel, designating it to be a sandboxed process
Every time this process calls a “system call” Check whether it is an allowable operation Check whether the arguments are “allowed” Interpret the call and ensure it falls under the “safe” category Policies have to be stated and enforced …. Loopholes?
20
Integrity Checking
Every executable file should have a pre-computed hash and a signature and an issuer certificate
Hash ensures any changes to the executable is detectable Signature ensures that no one updates the hash Issuer certificate ensures the signature is not faked This information is stored, separately from the file itself (hopefully
securely)
Integrity Checking Check hash and signature every time the file is executed Check hash occasionally while the program execution is in progress
(to detect dynamic corruption)
Problem: Who does the checking?
21
Application Integrity Checking
Assume: Operating System is not compromised OS does Application Integrity Checking
OS stores hashes and signatures Unsigned application either not allowed, or signed by OS Dynamic checking is also done
Ensures freedom from downloaded viruses Ensures some freedom from buffer overflow attacks
Cannot be completely detected, but the harm caused is limited
Runs only “trusted applications” Either signed applications or authorized by the user
Help prevention of undetected downloads, spyware and many such malware
22
Kernel Integrity Checking
How to check the OS for rootkits? Use a hardware device
TCG “Trusted Computing Group” approach “Co-Pilot” approach
Use a software device The VMM “Virtual Machine Manager” approach
Concept: A trusted device has keys and signatures for the operating system
components This device can access the OS and check it for integrity Possible to detect rootkits, as long as the checker is not
compromised
23
Hardware Checking
TPM (trusted platform module) from TCG A hardware device, that contains keys and hashes for checking
integrity of operating systems Attestation of software and digital signatures possible An elaborate framework of security modules, protocols and
monitoring of applications Can be used to enforce software licenses
Co-Pilot Project Hardware on the PCI bus, can take over the computer Performs memory checks on the OS and interrupt routines Can detect rootkits, even dynamic ones
24
Software Checking
Run the Operating System as a host on a Virtual Machine Manager
The VMM is hard to compromise Small, hence free of vulnerabilities (assumption) OS has to be rootkitted to attack VMM, but the VMM can detect
such rootkits
Check OS integrity from the VMM (Terra Project) Check OS integrity from a separate host operating system
called the Security Monitor ASU project
Work in progress, not tested well yet
25
Secure I/O
All security solutions outlined so far suffers from a spoofing vulnerability
A piece of software can be flagged to be “trusted” by faking human input by a program (virus)
TO ensure spoof freedom there is a need for a secure I/O channel from the core security system to the human user
TCG-TPM, Co-Pilot, Terra --- all are vulnerable
Secure I/O needs separate hardware channel, separate keypad
An SSL connection to a small communicator may be a solution.
26
Crypto File Systems
All files are stored encrypted Where is the key?
Derived from a passphrase Everything is “weak” as along the computer is “running”
MS-CFS uses PKI
Advantages Lost/stolen disks are secure Proper implementations may disallow some viruses from accessing
some files Diadvantrages
Files are decrypted on the fly and CFS is transparent to users and hance viruses
Attacking the key is possible
27
Current Status
Integrity checking is almost non-existent in deployed software
Hence, all systems are soft against viral attacks Big security problem, Device driver signing is used in Windows, quite ineffective
The TCG system is the closest to deployment Also the weakest system Has political implications
No other solutions are being deployed may not be entirely true