How to configure esx to pass an audit

How to Configure your ESX Hosts to How to Configure your ESX Hosts to Successfully Pass an Audit… Successfully Pass an Audit… GUARANTEED! GUARANTEED!

Greg Shields, MVP, vExpertGreg Shields, MVP, vExpertHead Geek, Concentrated Technologywww.ConcentratedTech.com

This slide deck was used in one of our many conference presentations. We hope you enjoy it, and invite you to use it

within your own organization however you like.

For more information on our company, including information on private classes and upcoming conference appearances, please

visit our Web site, www.ConcentratedTech.com.

For links to newly-posted decks, follow us on Twitter:@concentrateddon or @concentratdgreg

This work is copyright ©Concentrated Technology, LLC

http://www.ConcentratedTech.com/

Four Documents of NoteFour Documents of Note

In Order of Usefulness– VMware’s VI3.5 Security Hardening Guide

High-level guidance for security and auditing.– The DISA’s STIG for ESX Server & “Virtual

Computing”Very specific guidance on security. Required if DoD-connected.

– CIS’s VMware ESX Server 3.0 BenchmarkThis document is aged, but serves as an additional data point for learning / education on common ESX topics.

– CIS’s Virtual Machine Security GuidelinesThis document, while also aged, is fairly general in its guidance.

DISA STIG GuidanceDISA STIG Guidance

Remember that ESX has roots in RHEL v3, specifically(Kernel 2.6.18-128.ESX).– Thus, protecting/auditing ESX starts by

protecting/auditing RHEL v3.

DISA Requirement ESX0010 states:– The IAO/SA will configure the ESX Server in

accordance with the UNIX STIG and Checklist. This is not applicable to ESX Server 3i.

Any ESX Server must first meet DISA’s general UNIX STIG, then also the ESX STIG.


Once met, DISA’s ESX STIG adds nearly 120 individual requirements for modifying / verifying the configuration of that server’s ESX functionality...


Once met, DISA’s ESX STIG adds nearly 120 individual requirements for modifying / verifying the configuration of that server’s ESX functionality...

Stepping through these items isn’t value added. Let’s instead discuss high-level security and

auditing requirements.

Guidance for Virtual Guidance for Virtual MachinesMachines

Secure virtual machines in the same ways you would physical machines.– Updates, A/V, A/M, firewalls.

Disable unnecessary functions.– OS services, physical devices, screen savers

(particularly important).

Leverage templates when possible– Templates ensure that every VM has a common

start point, common security/auditing settings.

– Eases config documentation.

Guidance for Virtual Guidance for Virtual MachinesMachines

Set Limits/Reservations to prevent resource overuse.– Greg’s Advice: Be careful with setting too many

limits/reservations.

– Don’t forget host reserve to protect host functions.

Isolate VM networks.– Physically separate VM interfaces from VMotion &

management connection interfaces to prevent data leakage. Very important.

– Leverage VLANs if your security policies allow.

– Use dVSs when possible to reduce configuration error, centralize management of virtual switches.

– Create isolated management network with high security level.

Spec ESX hosts with lots of network cards!

Guidance for VMX File Guidance for VMX File CustomizationCustomization

Disable Remote Copy/Paste Operations between Guest OS and Remote Console– Can be used as vector for data leakage. Typically

unsecured.isolation.tools.copy.disable = TRUEisolation.tools.paste.disable = TRUEisolation.tools.setGUIoptions.disable = FALSE

Prevent Log Overflow– VM logs to VI datastore can overflow log space.

– Set rotation size and count of logs to keep.log.rotatesize = 100000log.keepOld = 10


Do not permit use of nonpersistent disks.– These disks revert back to snapshot when VM is

rebooted.

– Can be used by would-be attacker to cover tracks.

– Verify in VM settings.

Verify that unauthorized devices are not connected.– Unnecessary peripherals should not be connected.

– Prevent user from connecting devices from within the guest OS.floppy<x>.presentserial<x>.presentparallel<x>.presentisolation.tools.connectable.disable = TRUE


Verify correct assignment of guest OS– While not necessarily a security risk, improper guest OS

assignment will have an impact on system performance.

Verify proper permissions on disk files.– .VMX files should be 755 (o+rwx, g+rx)

– .VMDK files should be 600 (o+rw)

– User and group should be root.

Guidance for ESX Service Guidance for ESX Service ConsoleConsole

Configure Service Console with default firewall settings.– Add additional settings as necessary for approved

services.

Suggested Firewall Suggested Firewall ExclusionsExclusionsPort # Purpose Traffic Type 5989/TCP CIM Secure Server Incoming

22/TCP SSH Server Incoming

5988/TCP CIM Server Incoming

427/TCP & 427/UDP CIM SLP Incoming & Outgoing

80,443/TCP vSphere Web Access Incoming

443,902/TCP VMware Consolidated Backup Outgoing

902/UDP VMware vCenter Agent Outgoing

3260/TCP Software iSCSI Client (If Used) Outgoing

123/UDP NTP Client Outgoing

80,9000-9100/TCP VMware Update Manager Outgoing

Add exclusions as necessary. Remember that many “odd” faults are Firewall-based.


Minimize use of VI Console– Console access can be substantial impact on VM

performance.

– Remote access protocols slightly better, but…

– Stop managing infrastructure from any consoles! Use remote tools!

Limit use of Service Console for administration– VI Client and VirtualCenter leverage well-defined APIs for

management.

– Service Console leverages Linux-based administration.

– More opportunity for mistakes with Linux-based administration.

– If scripting/automation is necessary, leverage Remote CLI, VI Perl Toolkit, or PowerShell Toolkit for scripting rather than shell scripting. Well-defined interfaces.


Authenticate via a Directory Service– Centralization of authentication via directory service

reduces chance of mistake or malicious (hidden) account creation./usr/sbin/esxcfg-auth --enablead --addomain mydomain.com --addc mydc.mydomain.com --krb5realm=mydomain.com --krb5kdc mydc.mydomain.com --krb5adminserver mydc.mydomain.com

Control root privileges– Disallow root logins to Service Console. Enforce

sudo.cat /dev/null > /etc/securityNote: This may impact iLO and DRAC functionality.

– Limit sudo to users in wheel group only.auth required /lib/security/$ISA/pam_wheel.so use_uid


Disable accounts after three failed logins– Common requirement in many compliance regs.

auth required /lib/security/pam_tally.so no_magic_rootaccount required /lib/security/pam_tally.so deny=3no_magic_root

– Create file for logging failed login attempts.touch /var/log/faillogchown root:root /var/log/faillogchmod 600 /var/log/faillog

Always remember that ESX Console is not Linux.– Don’t manage like Linux. Only install ESX-compatible

software.

Guidance for Logging / Guidance for Logging / AlertingAlerting

Configure NTP– Accomplished through VI Console.

Enable remote syslog logging– Most compliance regulations require offsite and

protected log storage.– Configure /etc/syslog.conf.

Add the line @<loghost.company.com> after each message type.Kill –SIGHUP `cat /var/run/syslogd.pid`

Create and store key file hashes (/etc, /etc/vmware)– sha1sum <fileName>– This process can be eased through Tripwire /

ConfigureSoft

Guidance for Logging / Guidance for Logging / AlertingAlerting

Configure SNMP. Use SNMP v3 where Possible.– Modify /etc/snmp/snmpd.conf– (Details of this configuration are out of scope for today’s class)– If SNMP v3 not possible, use isolated network for SNMP traffic.

Guidance for NetworksGuidance for Networks


Mask and Zone FC SAN resources correctly.– Ensure that LUNs are only presented to interfaces which need

them.

Leverage iSCSI Authentication– iSCSI CHAP authentication is per HBA/NIC, not per-target.– No Kerberos available. No encryption available.– Ensure that iSCSI traffic is always isolated (security + DoS

prevention).

Leverage VM-based firewalls for intra-ESX ACLing.– ESX’s internal layer 2 firewall terminates network ACLs.

– External Switch Tagging (EST) VLANs terminate at pSwitch

– Virtual Switch Tagging (VST) VLANs terminate at vSwitch




vSphere + Cisco Nexus overcomes this limitation.


Replace self-signed certificates– ESX’s native self-signed certificates can be a MitM attack

vector.– Replace existing certificates with CA-signed certificates.– Refer to the VMware document Replacing VirtualCenter

Server Certificates for detailed specifications: http://www.vmware.com/pdf/vi_vcserver_certificates.pdf.

Disable Promiscuous Mode, MAC Address Changes, Forged Transmissions where possible.– Disabling MAC Address Changes can impact some

clusters.– Promiscuous Mode required for IDS/IPS. Isolate if needed.

http://www.vmware.com/pdf/vi_vcserver_certificates.pdf

Guidance for vCenterGuidance for vCenter

Limit administrator access. Ensure separation of duties.– vCenter includes high-level administrator access, but also

discrete task assignment. Ensure that tasks are assigned as needed.

Limit database access after installation.– vCenter database creation at installation requires DB

Owner rights.

– Database operations only requires Invoke/execute stored procedures, select, update, insert, and delete.

Segregate VMware Update Manager and VMware Converter Enterprise roles to isolated computers.– This maintains the security position of the vCenter server.

Consider AutomationConsider Automation

Tripwire ConfigureSoft

Sample Audit ProgramSample Audit Program

Stop by www.ConcentratedTech.com to download an actual ESX 3.5 audit program.

This audit program includes the exact steps an auditor (from this particular group) must use to verify settings on an ESX sever.

Follow this document, and pass that audit…GUARANTEED!

http://www.concentratedtech.com/

VirtualizationVirtualization’’s Four Horsemens Four Horsemen

Hypervisor Ubiquity– There is a singular hypervisor upon which

everything sits.

VM Dormancy– Powered down virtual machines don’t get

patched.

Virtual Networking– Intra-ESX network ACLs don’t exist.

VM Collocation– VMotioning can collocate VMs that should be

segregated.

This slide deck was used in one of our many conference presentations. We hope you enjoy it, and invite you to use it

within your own organization however you like.

For more information on our company, including information on private classes and upcoming conference appearances, please

visit our Web site, www.ConcentratedTech.com.

For links to newly-posted decks, follow us on Twitter:@concentrateddon or @concentratdgreg

This work is copyright ©Concentrated Technology, LLC

http://www.ConcentratedTech.com/