View
221
Download
2
Category
Preview:
Citation preview
Presentations on www.slideshare.net/xen_com_mgr/presentations
Lars KurthCommunity Manager, Xen Project
Chairman, Xen Project Advisory Board
Director, Open Source, Citrix lars_kurth
Cloud1. Virtual Machine Introspection (with demo)
2. Vulnerability Management In Xen Project
3. Live Patching (with demo)
Embedded & Automotive5. Why Virtualize Embedded Systems
8. Additional Security Properties of Xen
9. Xen Project In Security Applications
10. Xen Project in Embedded and Automotive (with demo)
12. Conclusion
Bonus Material: Embedded & Automotive (Slide 49)6. Hypervisor Architectures on ARM
7. PV Drivers and Protocols for Embedded Use-Cases
11. Schedulers and interrupt latency
Bonus Material: Security (Slide 67)4. Assessing a FOSS Project’s Security Record
www.xenproject.org
www.xenserver.org
A new way to protect against malware
Developed by Zentific, Citrix, Bitdefender, Intel and others
VM3
Guest OS
App
VMn
Guest OS
App
VM2
Guest OS
App
Dom0
Dom0 Kernel
Drivers Agent(s) Agent(s) Agent(s)
Installed in-guest agents, e.g. anti-virus software,
VM disk & memory scanner, network monitor, etc.
Can be disabled by rootkits
Several
VM3 VMnVM2Dom0
Dom0 Kernel
Drivers
VM3
Guest OS
App
VMn
Guest OS
App
VM2
Guest OS
App
Security
Appliance
VM1
IntrospectionEngine
Protected area
authentication mechanism to protect the IF
Uses HW extensions to monitor memory (e.g. Intel EPT) Low Intrusion
Register rules with Xen to trap on and inspect suspicious activities
(e.g. execution of memory on the dynamic heap)
All malware need an attack technique to gain a footholdAttack techniques exploit specific software bugs/vulnerability
Most exploits use one of a small set of attack techniques Buffer Overflows, Heap Sprays, Code Injection, API Hooking, …
Because VMI protects against attack techniquesIt can protect against entirely new malware
Verified to block these advanced attacks in real-timeAPT28, Energetic Bear, DarkHotel, Epic Turla, Regin, ZeuS, Dyreza, EternalBlue… solely by relying on VMI
WannaCry/EternalBlue blocked in real installations1
1 businessinsights.bitdefender.com/hypervisor-introspection-defeated-enternalblue-a-priori
Rootkits Exploit 0-days in Operating Systems/System SoftwareCan disable agent based security solutions (mask their own existence)
VMI solutions operate from outside the VMThus, it cannot be disabled using traditional attack vectors
BUT:VMI is not a replacement, for traditional security solutionsIt is an extra tool that can be used to increase protection
Pratap Sankar @ Flickr
Documentationwiki.xenproject.org/wiki/Virtual_Machine_Introspection
Products
Bitdefender HVIXenServer
www.bitdefender.com
Protection & Remedial
Monitoring & Admin
Citrix Ready
Zentific ZazenXen & XenServer & …
www.zentific.com
Protection & Remedial
Monitoring & Admin
Forensics & Data gathering
Malware analysis
AIS IntrovirtXenServer
www.ainfosec.com
Pratap Sankar @ Flickr
https://www.youtube.com/watch?v=qpQPBvOniUU
Result of several community consultations
R: Vulnerability reported to security@xenproject.org
P: Vulnerability pre-disclosed on xen-security-issues@lists.xenproject.org
R P
Fixing Security Bugs:
Dedicated security team =
security experts from within
the Xen Project Community
Security Team:
Triage
Creation of fix/patches
Validation of fix/patches
Assignment of CVE
Issue description and risk analysis
AR P
Fix their systems/software:
Eligible Xen Project Users
are informed under embargo
of the vulnerability
Eligible Users = Pre-disclosure list members:
Product Companies, Open Source & Commercial Distros (e.g. Huawei, Debian)
Service/Cloud Providers (e.g. Alibaba)
Large Private Downstream (e.g. Google)
Allowed to share information via
xen-security-issues- discuss@lists.xenproject.org
R: Vulnerability reported to security@xenproject.org
P: Vulnerability pre-disclosed on xen-security-issues@lists.xenproject.org
A: Vulnerability announced on xen-announce@lists.xenproject.org & xenbits.xen.org/xsa
A XR P
General Publication:
Information about
vulnerability is made public
Everyone else:
Patches their systems either through
security updates from distros/products or
builds them from source.
Users of service/cloud providers will
not be impacted
R: Vulnerability reported to security@xenproject.org
P: Vulnerability pre-disclosed on xen-security-issues@lists.xenproject.org
A: Vulnerability announced on xen-announce@lists.xenproject.org & xenbits.xen.org/xsa
A XR P
R: Vulnerability reported to security@xenproject.org
P: Vulnerability pre-disclosed on xen-security-issues@lists.xenproject.org
A: Vulnerability announced on xen-announce@lists.xenproject.org & xenbits.xen.org/xsa
Product Vendors:
Create and test live patches
Product customers:
Apply live patches here
Service Providers:
Create, test and deploy live patches
Users of service/cloud providers will
not be impacted
Security Team:
Can a Livepatch can be created?
No? If possible, re-write fix/patches
A tale of close collaboration withinthe Xen Project Community
const char *xen_extra_version(void)
{
return XEN_EXTRAVERSION;
}
push %rbp
mov %rsp,%rbp
lea 0x16698b(%rip),%rax
leaveq
retq
const char *xen_extra_version(void)
{
return “Hello World”;
}
push %rbp
mov %rsp,%rbp
lea 0x29333b(%rip),%rax
leaveq
Retq
Replacing compiled functions with new code, encoded in an ELF file called
payload, while the hypervisor is running without impacting running guests.
Design: xenbits.xenproject.org/docs/unstable/misc/livepatch.html
The exact source tree used to
build the running Xen instance.
The .config from the original
build of Xen.
A build-id onto which the
livepatch will be applied.
A source patch.
livepatch-
build-
tools
The exact same
compilation toolchain
used to build the
running Xen.
Livepatch
payload
Supports stacking of different payloads; payloads depend on build-id
Functionality:
list: lists loaded and applied live patches
upload: load & verify a live patch
unload: unload a live patch
apply: apply a live patch
revert: un-apply a live patch Xen 4.8.1
XSA 213
XSA 214
XSA 215
Depends on
build-id of 4.8.1
Depends on
build-id of XSA 213
Depends on
build-id of XSA 214
Target
Dom0 &
Guest
Linux
Kernel
Hypervisor
Technology
Kernel
Live Patching
kPatch
(RedHat)
Xen
LivePatch
kSplice
(Oracle)
kGraft
(SUSE)
Function +
Data
✔
✔
✔ Xen 4.7
✔
✔
Inline f()
patching
✗
✗
✗ Future
✔
✗
Data
Structures
✗
✔ via hooks
✔
✔
✗
Xen 4.8
via hooks
XenServer
LivePatch
Integrates
different solutions
into a single user
experience
For Dom0
(CentOS)
For Xen
Source patches
+ other
build artifacts
Hot Fixes containPer valid patch level: a Xen or Dom0 Live Patch
Matching RPMs for most recent patch level In case of a reboot or for Xen/Dom0 not capable of Live Patching
Extensive Verification and Validation:
The process of patching a live hypervisor or kernel
is not an easy task. What happens is a little bit like open
heart surgery. The patient is the hypervisor and/or Dom0
itself, and precision and care are needed to get things right.
One wrong move and it is game over.
Live
Patch
Live
Patch
Live
Patch
buildfor each
patch level
package
Hot Fix
LPsRPMs
RPMRPMRPM
buildfor most recent
patch level
Hot Fix
LPsRPMs
Publication
SigningValidation
Verification
Q&A
(livepatch-build
or kpatch-build)
(iso)
XAPI
Toolstack
Hot Fix
LPsRPM
downloadHot Fix
LPsRPM
Dom0
Dom0 Kernel(CentOS)
Hypervisor
XenCenter
or xe
Initiates
host update
SysAdmin
Running System
instance that supports
live patching
Disk
updates(such that
after reboot
the patches
are applied)
works out
correct LP
& updates
(using native
live patching
tools or APIs)
Pratap Sankar @ Flickr
xenbits.xenproject.org/people/larsk/LCC17 - Build LivePatch.mp4xenbits.xenproject.org/people/larsk/LCC17 - Apply LivePatch.mov
Pratap Sankar @ Flickr
Xen Project LivePatch Specification & Statusxenbits.xenproject.org/docs/unstable/misc/livepatch.htmlwiki.xenproject.org/wiki/LivePatch
Xen Project LivePatch Presentations & Videosxenbits.xenproject.org/people/larsk/FOSDEM17-LivePatch.pdf (Short)people/larsk/XPDS16-LivePatch.pdf (Long)
Xen Project LivePatch Videosfosdem.org/2017/schedule/event/iaas_livepatxen/
XenServerxenserver.org
ConsolidationReduce cost, size, weight, power consumption
and heat emission (already an issue today in cars)
CAN bus scalability / Architecture complexity
Reduce development costs: platform independence
Security and SafetySeparate safety critical apps from general apps
Safety Certification of the Hypervisor
Embedded Requirements (Bonus)Minimal IRQ latency
Low or 0 scheduling overhead
Drivers for special I/O devices
Flexible architecture
System Partitioning Sandboxing drivers & system components
Fine-grain control of VM capabilities
Enables multi-layered security approach
Other Security FeaturesTrusted Execution Environment (TEE)
Virtual Machine Introspection, alt2pm
Live Patching
Guest Kernel
Xen Project Hypervisor
Driver Domain Guest OS*: Linux, BSD, MiniOS, unikernel, …
DiskController
Guest Kernel*
Storage Domain
Disk Driver
Guest Kernel*
Network Domain
Network Driver
NetworkController
BlockFront Driver BlockBack Driver
Dom0
Kernel
Application
NetFront Driver NetBack Driver
Attack Surface Reduction
Similar to Linux Security Modules/SELinux
Same policy syntax as SELinux
Different types, roles, users and attributes
Same tools for policy compilation / verification (checkpolicy)
VM
hypervisor domain(self) domain(other) memory (grant, mmu, shadow)
inter-VM communicationpassthroughsecurity config
Fine-grained policy, controlling
which hypervisor functionality is
accessible to this (class of) VM
Effect: limit what an exploit in
this VM could do
Pratap Sankar @ Flickr
Crucible:Defensestarlab.io
Xen Project based virtualization
platform for technology protection,
cyber-hardening, and system integrity
for aerospace & defense systems
Documentationwiki.xenproject.org/wiki/Dom0_Disaggregationwiki.xenproject.org/wiki/Xen_Security_Modules_:_XSM-FLASK
Products & Projects
Qubes OSwww.qubes-os.org
Secure OS
OpenXTwww.openxt.org
FOSS Platform for security research,
security applications and embedded
appliance integration building on
Xen & OpenEmbedded
User defined App VMs for individual
apps or groups of apps
USB
Service
Domain
Banking
Domain
Personal
Domain
Firewall VMenforces network
policies
Network
Domain
Dom0
Secure UI and
sysadmin domain
Pratap Sankar @ Flickr
AISainfosec.com
BAE Systems
baesystems.com
Galoisgalois.com
Maintain FreeRTOS Xen Port
Developed and maintain HalVM
Dornerworksdornerworks.com/xen
Consulting
Xen Embedded Distros
Xen for Xilinx Zynq
Xen for NXP i.MX 8
ARLX HypervisorDO-178 (EAL6+), IEC 62304, ISO 26262
MILS EAL
FACE, VICTORY, ARINC 653
Starlabstarlab.io
Crucible and Crucible:Defense
Xen embedded hypervisorIn progress: DO-178, MILS EAL
Uses a minimal Dom0 using
MiniOS, disaggregation and
XSM/FLASK
Precedents of military grade certification for Xen based systems
www.slideshare.net/xen_com_mgr/art-certification & www.youtube.com/watch?v=UyW5ul_1ct0
xenbits.xenproject.org/people/larsk/XPDS14 - Xen and the Art of Certification.pdf
www.linux.com/news/xen-project/2017/2/how-shrink-attack-surfaces-hypervisor
Pratap Sankar @ Flickr
LG ElectronicsDemobit.do/lg-xen-demo-2016
Bosch Car GmbHContributions10 smaller features in 2016
Perseus (stealth)Founded by Xen maintainerbit.do/perseus-2017
Demo at IAA 2017 in Frankfurt, Germany
ADITJoint venture Bosch & DENSOLittle known at this stage
GlobalLogicProduct: Nautilusbit.do/gl-nautilus
First product in production
expected in Q1 2018
Supports:HW: Renesas R-Car Gen2 & Gen3,
TI Jacinto6, Intel Apollo Lake, Qualcomm
410C, Sinlinx A33
Guests: Linux up to 4.9 Android M, N,
N-Car QNX, ThreadX, FreeRTOS
PV Drivers for: GPU, Audio, HW
accelerated Video codecs, DRM, …
Contributions:27 smaller features from 2013 to 2016
EPAMDemoNext slide
Interesting Features:Container based telematics applications
running in a Xen VM that can be
downloaded from a cloud service
Ongoing Contributions:ABIs for PV Sound, PV Display & PV DRM
Leading development of co-processor
sharing framework
Pratap Sankar @ Flickr
xenbits.xenproject.org/people/larsk/
LCC17 - The Internet of Transportation[1080P].MP4
AWS
Dom0 - Control DomD – HW Drivers & Cluster
Wayland/Weston
OpenGL ES
Linux Kernel with GPU and other HW Drivers
ALSA wPV_ALSAS_BE
DomU Fusion
Container mgmt tool
Linux Kernel w/oHW Drivers
Minimal rootfswith systems
library
Telematics simulation Agent (Acceleration, Braking, Corning, GPS)
DomU – Linux IVI
MW Frameworks
PVDISPLAY
Linux Kernel with GPU and w/o other HW Drivers
PVEVENTS
PVSOUND
IVI Simulation App Trusted Apps
TrustZone
Hypervisor
R-Car H3 Platform
OP-TEE OS
TZ monitor
Driver Behavior Based Insurance Backend
Telematics Simulation Agent ver 2.0
Telematics Simulation Agent ver 1.0
Monitoring Dashboard
Wayland BE(Events/Display)
Cluster Simulation AppDom0 Services
Minimal rootfs
Linux Kernel w/o HW Drivers
Containers
Picture by Lars Kurth
Xen Project & Security in Cloud:
Only Hypervisor with VMIProtection from new classes of malwareSeveral security companies working with XenServer
Live PatchingDisruption free application of vulnerabilitiesUsed by several cloud providersUsed best in commercial products, e.g. XenServer
Industry Leading Vulnerability ProcessIncludes QEMU and Kernel XSAs
Picture by Lars Kurth
Xen Project in Embedded & Automotive:
Extremely Flexible and VersatileProven in many different marketsEasy to port to new environmentsEasy to develop new PV drivers (see bonus 2)Highly customizable
Security and ResilienceIsolation, Partitioning, Security Features
SafetyExamples of Military Grade CertificationBUT: looking at ways to make this easier and cheaper
Challenges still being addressedStandardization of more I/O devices via PV protocolsStandardization of GPU and co-processor sharingRTOS or other minimal OS as Dom0Testing of embedded Hardware by the project
Picture by Lars Kurth
www.slideshare.net/xen_com_mgr/presentations
Picture by Lars Kurth
Picture by Lars Kurth
EL0/PL0 least privileged mode used for applications (user mode)
EL1/PL1 privileged mode used for running kernels such as the Linux kernel
EL2/PL2 This has a higher level of privilege and can be used to run a hypervisor which takes control
of the system and can host multiple "guest" operating systems
EL2
EL1
EL0
Guest Kernel
Guest Kernel
Guest
Userspace
Guest
Userspace
Host
Userspace
Host Kernel + Hypervisor Native DDs
Type 2 with VHE/ARMv8.1 (e.g. KVM)
Guest Kernel
Guest Kernel
Guest
Userspace
Guest
Userspace
Guest
Userspace
Guest Kernel
Hypervisor
Traditional Embedded Type 1 Hypervisor
Native DDs
EL1
EL2
EL0
Guest Kernel
Guest Kernel
Guest
Userspace
Guest
Userspace
Host Kernel + Hypervisor Native DDs
Type 2 with VHE/ARMv8.1 (e.g. KVM)
Guest Kernel
Guest Kernel
Host
Userspace
Guest
Userspace
Guest
Userspace
Hypervisor
Guest Kernel
Native DDs
Traditional Embedded Type 1 Hypervisor
Host
Userspace
EL2
EL1
EL0
Xen Project Hypervisor
Guest Kernel
Guest Kernel
Guest
Userspace
Guest
UserspaceStrong Isolation Device Drivers run in EL1,
not EL2
Protected Address Spaces:
Grant tables
Trusted Computing
Base (TCB)
Dom0
Kernel
Native DDs
Dom0
Userspace
Toolstack
EL1
EL2
EL0
Xen Project Hypervisor
Guest Kernel
Guest Kernel
Guest
Userspace
Guest
Userspace
Control PlaneServer: sysadmin
Embedded: config/setup, system
health monitoring (watchdog),
maintenance, SW updates, …
Dom0
Kernel
Native DDs
Dom0
Userspace
Toolstack
Dom0
Kernel
Guest Kernel
HWI/O
Native Driver
Xen Project Hypervisor
*Back Driver *Front Driver
Application
Existingnet, block, consolekeyboard, mouse, USBframebuffer, GPU sharing*
New in Xen 4.99pfs (share a filesystem between VMs)Pvcalls (forward POSIX calls across VMs)multitouch, sound, display, DRM
Developing New OnesEasy to write (GPL and BSD samples)Kernel and User Space
*) A number of different approaches by different vendors in different market
segments are being deployed, which are PV-like, but not strictly a PV
protocol
Xen supports several different schedulers with different properties.
Xen supports several different schedulers with different properties.
Regular VM
scheduler (Credit)Hard real-time
(ARINC653)
Dedicated to 1 VCPU via pinning and Null scheduler
no scheduler overheads
Soft real-time
(RTDS)
Scheduler Use-cases Today Future plans
Credit General Purpose Supported
Default
Supported
Optional
Credit 2 General Purpose
Optimized for lower latency, higher VM density
Supported Default
RTDS Soft & Firm Real-time
Multicore
Embedded, Automotive, Graphics & Gaming in
the Cloud, Low Latency Workloads
Experimental
Better XL support
<1μs granularity
Supported
Hardening
Optimization
ARINC 653 Hard Real-time
Single core
Avionics, Drones, Medical
Supported
Compile time
Null Hard Real-time Experimental Supported
vCPU 0
pCPU 0
vCPU 1
pCPU 1
irq 109
virq 109
IRQ injection
Always on the CPU running the vCPU
vCPU 0
pCPU 0
irq 109
virq 109
vCPU 1
pCPU 1
IF
vIRQ target changes or vCPU is moved
THEN
vIRQ is moved immediately
virq 109
vCPU 0
pCPU 0
vCPU 1
pCPU 1
irq 109
virq 109
IRQs always shadow the vIRQ
minimizes latency
Xilinx ZynqMP board
(four Cortex A53 cores, GICv2)
WARM_MAX (excluding the first 3 interrupts): <2000ns
Without Null scheduler
See blog.xenproject.org/2017/03/20/xen-on-arm-
interrupt-latency/
Developer Portal: bit.do/xen-devsXen on ARM whitepaper: bit.do/xenarm-whiteXen on ARM wiki: bit.do/xenarm-wiki
Port Xen to a new SOC: bit.do/xenarm-portingAdd Xen support Xen to your OS: bit.do/xenarm-os
Device Passthrough presentation: bit.do/xenarm-ptOE meta-virtualization Xen recipe: bit.do/xenmetaOpenXT (Xen + OpenEmbedded): openxt.orgXenbedded presentation: bit.do/xenbedded
Monthly ARM Community Call: bit.do/xenarm-call
Picture by Lars Kurth
Security Process Number of Vulnerabilities Media CoverageOther Considerations
A XR P
Responsible Disclosure: fix critical systems/software before publication
R: Vulnerability reported to security@...
P: Vulnerability pre-disclosed to eligible users
A: Vulnerability announced publicly
F: Fix available
Full Disclosure, immediate (no-fix): public disclosure without a fix
A XR F
A XR
Full Disclosure, post-fix: public disclosure with a fix
F
F
4) New members: http://seclists.org/oss-sec/2017/q2/6385) http://www.openwall.com/lists/oss-security or devel list6) https://wiki.qemu.org/index.php/SecurityProcess
Only handles x86 KVM bugs (no ARM or other bugs)
1) Is the CVE severity used to handle vulnerabilities differently?2) Days embargoed (information is classified)3) D = Distros/Products, S = Public Service, P = Private Downstream4) http://oss-security.openwall.org/wiki/mailing-lists/distros
Responsible only
Days 2 Who? 3FOSS Project Bug Severity 1 Process Type
14-19 D 4
Linux Kernel via
OSS-security distros 4 ≥ Medium – Critical Responsible Disclosure
14-19 D 4
QEMU (KVM) via
QEMU Security
Process 6≥ Medium – Critical
≤ Low
Responsible Disclosure
Full Disclosure, no-fix
3-5 D, S, POpenStack OSSA
OpenStack OSSN
≥ Medium – Critical
≤ Low
Responsible Disclosure
Full Disclosure, post-fix
Xen HypervisorIncludes Linux & QEMU
vulnerabilities in supported Xen
configurations
Low – Critical Responsible Disclosure 14 D, S, P
14-19
Linux Kernel via
OSS-security distros 4
OSS-security 5≥ Medium – Critical
≤ Low
Responsible Disclosure
Full Disclosure, no-fix
Impacts how long a user (aka you) is at riskIs my distro/vendor on a pre-disclosure list?
A surprisingly large number of distros are not: including a few on Linux.com’s Best Linux Distros of 2016 list
Impacts cloud / service providers As a user, are security issues fixed before public disclosure?
Low Severity vulnerabilities can still be High Risk Temporal and Environmental CVSS scores are not covered by CVE databases (neither cvedetails.com or nvd.nist.gov)
Vulnerabilities can be chained together, making the combo High Severity(e.g. Hot Potato used 3 old unpatched vulnerabilities to gain root access)
cvedetails.com (bit.do/guide-cvedetails)
Easy to use interface for vulnerability data
Data from several sources
Browsable by vendor, product, version, type, date…
Vulnerability statistics, trends, reports
BUT: rigid➜ getting data outside pre-defined vendor/product categories is near-impossible
vulners.com (good guides on slideshare.net – search for vulners)
In many ways more accurate and flexible than cvedetails
type:cve AND (description:kvm OR description:qemu) AND published: [2012 TO *] ➜307
type:cve AND (description:xen) AND published: [2012 TO *] ➜ 245
…
Works best when used through its API (in particular if you want to visualize the data)
Vulnerability data from vulners.com
0
50
100
150
200
250
300
350
2012 2013 2014 2015 2016 2017
Linux Kernel
KVM w. QEMU
Xen Project
*) Data up to Sept 4th, 2017
169
517
222
197
129
120
51
7
63
143
34
5
Legend: CVSS
Score Distribution
Low
Medium
High
Critical
*
Data covering September 2016 – September 2017: from vulners.com, mention.com and theregister.com
Clips covering
vulnerabilities
(US only)
% of Vulnerability
Stories on
The Register155370 314
7283
0
1000
2000
3000
4000
5000
6000
7000
8000
Xen KVM QEMU Linux
0
50
100
150
200
250
300
350
400
450
Xen KVM QEMU Linux
Number of
Vulnerabilities
0%
5%
10%
15%
20%
25%
30%
35%
40%
Xen KVM QEMU Linux
33% of Xen stories
cover Vulnerabilities
2.5%
16%
6.5%
Does the Project Look for Vulnerabilities?
Approach to Quality and Testing: e.g. Fuzzing, Audits of components
Does the project award Bug Bounties (e.g. NetBSD)?
Do vendors supporting the project offer Bug Bounties?
Infrastructure related to VulnerabilitiesTransparency: How well are processes documented
Vulnerability Testing: XTF (in Xen)
Vulnerability Tooling: XSATool, XSAMatch (in Xen)
Recommended