Systems TSC VSP

Anthony McNamara

X64 Workshop Driving the Platform

Day 1 AM Driving A Platform: Agenda

Introduction

X64 Family

AMD vs Intel Differences

Family

BIOS Overview

Service Processor Overview

Connectivity

Useful links

Lab

X64 Workshop Introduction

Workstations

U20(M2) / U24 / U40(M2)

Servers

V20Z / V40z

X2100 / X2100M2 / X2200M2

X4100 / X4140 / X4200 / X4240 / X4440 / X4500 / X4600

X4150 / X4450

Blades

More than I can mention on this slide

The X64 Family

X64 Workshop Introduction

Single or dual core AMD Opteron CPUs on U20 / U40

Dual and quad core Intel Core 2 CPUs on U24

DDRI (DDRII on M2 and U24) memory

Up to two SATA drives on U20

Up to four SATA drives on U24 / U40

Up to eight SATA drives on U40M2

Up to four SAS drivers on U24

Up to eight SAS drives on U40M2

Up to two Gig-E ports

PCI-E and PCI cards supported

The X64 Family U20 / U24 / U40 (M2)

X64 Workshop Introduction

Up to two single or dual core AMD CPUs (v20z)

Up to four single or dual core AMD CPUs (v40z)

DDRI memory (Up to 200MHz if 2 DIMMs per CPU)

Two Ultra SCSI drives (V20z)

Up to six Ultra SCSI drives (V40z)

Dual Gig-E ports

PCI-X cards supported

The X64 Family V20z/V40z

X64 Workshop Introduction

One single or dual core AMD CPU

One dual or quad core AMD CPU (X2100M2)

Up to two dual or quad core AMD CPUs (X2200M2)

DDRI (DDRII M2)

Up to two SATAII drives (SAS with PCIe card kit)

Dual Gig-E ports (four Gig-E ports M2)

PCI-E 8x cards supported

The X64 Family X2100 / X2100 M2 / X2200 M2

X64 Workshop Introduction

Up to two single or dual core AMD CPUs

Up to two dual core AMD CPUs (M2 single CPU plug required)

Quad core CPUs not supported

DDRI memory / DDRII (M2)

Up to for SAS drives on X4100 (DVD option removed)

Up to four SAS drives on X4200 (With DVD option)

Four Gig-E ports

PCI-X cards supported

PCI-E cards additionally supported on M2 version

The X64 Family X4100 (M2) / X4200 (M2)

X64 Workshop Introduction

Two dual core AMD CPUs

Quad core CPUs not supported

DDRI memory only

Up to 48 SATA-II drives

No hardware RAID

Four Gig-E ports

2 x PCI-X cards supported

The X64 Family X4500

X64 Workshop Introduction

Up to eight single core AMD CPUs

Up to eight dual core AMD CPUs

Up to eight quad core AMD CPUs

DDRI memory on original X4600

DDRII memory on upgraded X4600 or X4600M2

Up to four SAS drives

Four Gig-E ports

PCI-E cards supported

The X64 Family X4600 (M2)

Memory DDR-IAMD Opteron 100Socket 939U20X2100Memory DDR-IAMD Opteron 200Socket 940Memory DDR-IAMD Opteron 800Socket 940v20zX4100X4200X4500X4600V40ZFamily - 1st Gen AMD Opteron Systems

Memory DDR-2AMD Opteron 1000Socket AM2U20M2X2100M2Memory DDR-2AMD Opteron 2000Socket FMemory DDR-2AMD Opteron 8000Socket FX2200M2X4100M2X4200M2X4600M2U40M2Family - 2nd Gen AMD Opteron Systems

AMD Opteron Identification

First generation AMD Opteron CPU's use a three digit identifier XYY;

X Family scalabilityYY Relative performance

Second generation AMD Opteron CPU's use a four digit identifier XYZZ;

X Family scalabilityY Socket generationZZ Relative performance

Model suffixes denote high or low power versions SE or HE

X64 Workshop Introduction

Intel dual core / quad core and quad core extreme Core 2 CPUs on workstations

Intel dual core / quad core Xeon CPUs on servers and blades

FBDIMMs for server products (DDRII / DDRIII for workstation products)

FSB replaces HyperTransport on 2008 models

FSB EOL late 2008 and replaced with Intel QuickPath technology

Separate northbridge chip handles memory and primary I/O

ICH9 or ESB2 handles secondary I/O

All server / blade platforms have eLOM hardware (AST2000 ARM9)

eLOM EOL late 2008 and replaced with recompiled iLOM code

Diagnostic tools require rewrite for Intel's machine check architecture

PC Check default for all platforms now and built into BIOS initiated ROM

The X64 Family Intel Differences

HyperTransport Overview

8GB/sec performance

1GHz x bus width x 2 transfers/clk x 2 (bidi)/8bit

Multiple bus widths supported

2 bit -> 32 bit

AMD Opteron CPUs have 3 HT links per module

1-way family have no coherent links

2-way family have x1 coherent link

8-way family have x3 coherent links

QuickPath Overview

Via Online Demo:

http://www.intel.com/technology/quickpath/index.htm?iid=tech_arch_nextgen+body_quickpath_bullet

AMD HyperTransport Technology

Intel QuickPath Technology

System Features Summary

BIOS Overview

The Role Of BIOS

What is it?

Set of API's that are run after power on

Initiates POST

Configures a useable system

Direct interface to system hardware

BIOS Overview Continued

BIOS Startup

Check CMOS for custom settings

Load interrupt handlers and drivers

Initialise registers and power management

Run POST

Load PCI option ROM and execute

Determine boot devices

Initiate boot process

Power on -> CPU loads BIOS Boot Program -> POST -> looks for BIOSes on optional cards such as GPU, HDD, etc ->You can break into BIOS at this stage to change or confirm BIOS setting -> memory tests and performs system inventory -> detect plug and play devices -> BIOS searches for a bootable device

BIOS determiens whether it is a cold boot or a reboot...if it's a reboot then POST is skipped.

BIOS Overview Continued

The Functions Of BIOS

Hardware initialisation

ACPI (Advanced Configuration and Power Interface)

Hardware recognition and power configuration

Power management

BIOS provides the OS with power control

Allows for power state switching

APIC (Advanced Programmable Interrupt Controller)

Local and I/O

Distributes interrupt signals

I/O APIC routes interrupts from the bus to the LAPIC

More sophisticated than older PIC (More IRQ's available etc)

Global states;

G0 normal workingG1 is sleep or standby several levels depending on implementationG2 is soft off some components retain power can be woken up by attached device (kbd, etc) still requires boot so cannot resume OSG3 is mechanical off

PIC allows 16 IRQ'sAPIC requires both IO and Local APICS to create a APIC bus

Edge-triggered interrupts result in all associated interrupt sector routines being run against a particular interrupt vector and 'hopes' that whatever caused the interrupt is corrected.

Level-triggered interrupts a device 'drives' the signal to it's active level where it is held until serviced, the problem is when you have multiple devices sharing the same interrupt status change in these devices won't be detected until the original event is serviced.

BIOS Overview Continued

The Functions Of BIOS

Option ROM load and execution

Boot device management

RAID Management

Both onboard and optional RAID cards

Platform dependent

We use both IPMI 1.5 (x2100) and IPMI 2.0

IPMI consists of the BMC and satellite controllers around the system, sensors report to the BMC via the IPMB (Intelligent Platform Management Bus a sort of I2C bus. Adminc use the BMC to remotely control system state

Service Processor

Connectivity

Functionality

Management

Console and remote console

ALOM, eLOM and iLOM interface (ALOM emulation)

IPMI

Intelligent Platform Management Interface

OS independent remote administration tool

Introduction to the SP

Service Processor Continued

Accessible through;

Web browser

CLI (Serial and LAN)

IPMI

SNMP (Sun N1, IBM Tivoli, HP OpenView)

Functionality is interface dependent

SP Connectivity

Galaxy health state = allmanage system state remotely only via ipmi, Web and CLI

Service Processor Continued

System management

Host and SP power state control

Component state monitoring

Event and console logging

Service Processor administration

SP user account maintenance

Virtual device management

Console redirection control

SP Functionality

Service Processor Continued

SP/BIOS updates

Can be updated via the WebGUI or SP CLI

Patches include changes to;

BIOS

SP firmware

PC Check tests

Onboard device updates (RAID, NIC, Video)

SP Management

Service Processor Continued

Multiple console options

KVMS / JavaR

Serial over LAN

Serial

Video

Connecting to the Console

Service Processor Continued

KVMS / JavaR

Launched via the WebGUI

Offers virtual CD-ROM and floppy device control

Offers virtual CD-ROM and floppy image control

Provides mouse and keyboard redirection

Issues?

Solaris and Windows requires 'Absolute' mouse mode

Linux and VMware requires 'Relative' mouse mode

SP Console KVMS / JavaR

KVMS = Keyboard, Video, Mouse & StorageAbsolute = grid referenceRelative = movement is offset against last known mouse position

Relative is considered to allow 'finer' mouse movement but can also lead to mouse jumps and erratic movement.

Service Processor Continued

Serial / LAN

Provides CLI console access

Console needs to be redirected depending on the connection method - Serial or SoL, eg:

set COM port to SYSTEM for ttya on X2200 M2

set COM to BMC and agentInfo to agent for ttyb on X2100 M2

System specific redirection details are available in the product notes, install and administration documents

SP Console Serial / LAN

Service Processor Comparison

Service Processor Comparison

Host

Supported OS

Solaris 9 and 10

Possible selling of OpenSolaris 05/08 contracts

Red Hat Enterprise Linux 3, 4 and 5

(Red Hat Enterprise 3 does not support ACPI/SMBIOS fully)

Suse 8, 9 and 10

Windows XP, 2003, Vista and 2008

VMWare ESX 3.1 - 3.5

Useful Links

http://systems-tsc/twiki/bin/view/Main/WebHome

http://systems-tsc/twiki/bin/view/Teams/TeamHomeX64

http://www.sun.com/products/index.jsp

http://www.sun.com/ {PRODUCT}

http://www.amd.com/us-en/Processors/ProductInformation

http://www.hypertransport.org

http://www.intel.com

Lab

Connecting To Remote iLOM Platforms

Ultra 24vi64-u24a-kvm-gmp03.uk

X2100/X2200va64-x2100g-sp-gmp03.uk

X4100/X4200Not available

X4150Not bookable

X4600va64-x4600a-sp-gmp03.uk

Booting using a remote .iso image file

Anthony McNamara

Anthony.McNamara@Sun.COM

X64 Workshop Driving the Platform

Sun Microsystems, Inc.

Page

Click to edit the title text format

Click to edit the outline text format

Second Outline Level

Sun Confidential: Internal Only

Click to edit the notes format

Page

Click to edit the title text format

Presenters Name

Presenters Title

Presenters Company

Click to edit the notes format

Page

SystemCode NameCPU TypeSocketMemoryDiskI/OSPU20MarrakeshAMD Opteron 100939DDR-ISATAPCI / PCI-E-U24UrsaIntel Duo / Quad775DDR-IISATAPCI / PCI-E-U40SiriusAMD Opteron 800940DDR-IISATAPCI / PCI-E-U20M2MunichAMD Opteron 1000AM2DDR-IISATA-IIPCI / PCI-E-U40M2StuttgartAMD Opteron 2000FDDR-ISASPCI / PCI-E-V20ZStinger2PAMD Opteron 200940DDR-IUSCSI 320PCI-XSPV40ZStinger4PAMD Opteron 800940DDR-IUSCSI 320PCI-XSPX2100AquariusAMD Opteron 100939DDR-ISATAPCI-ESMDCX2100M2LeoAMD Opteron 1000AM2DDR-IISATA-IIPCI-EELOMX2200M2TaurusAMD Opteron 2000F DDR-IISATA-IIPCI-EELOMX4100Galaxy1AMD Opteron 200940DDR-ISASPCI-XILOMX4100M2Galaxy1AMD Opteron 2000F DDR-IISASPCI-EILOMX4140DoradoAMD Opteron 2000FDDR-IISASPCI-EELOMX4150DoradiIntel Duo / Quad Xeon775FB-DIMMSASPCI-EELOMX4200Galaxy2AMD Opteron 200940DDR-ISASPCI-XILOMX4200M2Galaxy2AMD Opteron 2000FDDR-IISASPCI-EILOMX4440TucanaAMD Opteron 2000FDDR-IISASPCI-EELOMX4450TucaniIntel Duo / Quad Xeon775FB-DIMMSASPCI-EELOMX4500ThumperAMD Opteron 200940DDR-ISATA-IIPCI-XILOMX4600Galaxy4AMD Opteron 800940DDR-ISASPCI-E / PCI-XILOMX4600M2Galaxy4AMD Opteron 8000FDDR-IISASPCI-E / PCI-XILOM

???Page ??? (???)27/05/2008, 15:45:27Page / FeatureSun Systems Group Service Processor

X64 Enterprise SystemsX64 HPC SystemsSPARC Systems

OldOld CurrentOldCurrentOldCurrent

V20z & V40z SP ShippingILOM Release 1.0 ShippingILOM Release 1.1 June 2006ILOM Release 1.2 Jan 2007ILOM Release 1.3 May 2007ILOM Release 2.0 October 2007 X2100 System Management Daughter CardLeo/Taurus Embedded SPALOM 1.5 shippingiLOM Mid 2007 on

???Page ??? (???)26/05/2008, 14:36:17Page / Monitoring & LoggingSNMP MonitoringYesYesYesYesYesYesNoYesVia SunMCYes

SNMP ControlYesNoYesYesYesYesNoNoNoYes

Monitoring of FRUs/componentsYesYesYesYesYesYesLimited**YesYesYes

Notification of alertsYesNoNoYesYesYesNoYesYesYes

Watchdog TimerYesYesYesYesYesYesYesYesYesYes

Event LoggingYesYesYesYesYesYesYesYesYesYes

Configurable Alert ThresholdsYesYesYesYesYesYesYesYesYesYes

Auditing CapabilityNoNoNoYesYesYesNoNoNoYes

AccessAccess to SP while powered downYesYesYesYesYesYesYesYesYesYes

Configure SP from BIOS screenNoYesYesYesYesYesNoNoNoN/A

Multiple User Account administrationYesYesYesYesYesYesNoYesYesYes

ControlForced Power OffYesYesYesYesYesYesYesYesYesYes

Graceful Shutdown/Power CyclingYesYesYesYesYesYesYesYesYesYes

Front LED System IdentificationYesYesYesYesYesYesYesYesYesYes

Rear System LED IdentificationNoYesYesYesYesYesNoYesNoYes

FRU data visible through SPYesYesYesYesYesYesYesYesYesYes

FirmwareUpgradeable firmware/BIOSYesYesYesYesYesYesYesYesYesYes

Reboot later option for firmware updateNoNoNoYesYesYesNoNoNoYes

Rollback capability (dual images)NoNoNoNoNoYesNoNoNoYes

Simple CLI & GUI upgrade procedureNoYesYesYesYesYesNoYesYesYes

???Page ??? (???)26/05/2008, 14:36:17Page / FeatureSun Systems Group Service Processor

X64 Enterprise SystemsX64 HPC SystemsSPARC Systems

OldOld CurrentOldCurrentOldCurrent

V20z & V40z SP ShippingILOM Release 1.0 ShippingILOM Release 1.1 June 2006ILOM Release 1.2 Jan 2007ILOM Release 1.3 May 2007ILOM Release 2.0 October 2007 X2100 System Management Daughter CardLeo/Taurus Embedded SPALOM 1.5 shippingILOM Mid 2007 onward

SP Cost (per server)+ $0+ $0+ $0+ $0+ $0+ $0+ $149+$0+ $0+$0

InterfacesVirtual media media capabilityNoYesYesNoYesNoNo

Remote KVMNoYesYesNoYesNoNo

Fully comprehensive CLIYesNoYesYesYes

DMTF 'SMASH' CLP supportNoYesYesNoYesNoNo

Browser-based GUI (English, i18n)NoYesYesNoYesNoYes

Serial Over LAN (console redirection)YesYes*YesYesYes

Dedicated 10/100 Management PortYesNoYesYesYes

SecuritySSH 2.0YesNoYesNoYes

LDAP supportYesNoYesYesYes

RADIUS supportNoNoYesYesNoYesNoYes

Microsoft Active Directory supportNoNoNoNoYesYesNoYesNoNo

Role Based Access ControlYesNoNoYesYes

Turn on/off GUINoNoNoNoNoYesNoNoNoYes

Turn on/off individual portsNoNoNoNoNoYesNoNoNoYes

Initial password allocationYesYesYesNoNo

Password reset capabilityYesYesYesYesYes

Standards supportIPMI v1.5 InterfaceYesYesYesNoNo

IPMI v2.0 InterfaceYesNoYesNoNo

SNMP V1 InterfaceYesNoNoNoYes

SNMP V2c InterfaceYes (trap only)NoNoYes

SNMP V3 InterfaceNoYesYesNoYesNoYes

WS Management NoNoNoNoYesYesNoYesNoYes

???Page ??? (???)26/05/2008, 14:36:17Page /