Upload
emily-rosa-white
View
221
Download
1
Tags:
Embed Size (px)
Citation preview
1
Windows NT Scalability
Jim Gray
Microsoft [email protected]
http/www.research.Microsoft.com/~Gray/talks/
2
OutlineOutline
• Scalability: What & Why?• Scale UP: NT SMP scalability• Scale OUT: NT Cluster scalability• Key Message:
– NT can do the most demanding apps today.
– Tomorrow will be even better.
Scale OutScale Out
Scale Up
Scale DownDown
3
Scale OutScale Out
Server ClusterServer ClusterWhat is Scalability?
• Grow without limits– Capacity– Throughput– Availability
• Do not add complexity– design– administer– Operate– UseS
cale
Do
wn
Do
wn
Win TermWin TermNetPCNetPC
HandheldHandheld
PortablePortable
TVTV
Sca
le U
pSuperSuperServerServer
ServerServer
PC PC WorkstationWorkstation
4
Scalability is Important• Automation benefits rising
– ROI of 1 month....
• Slice price going to zero– Cyberbrick costs 5k$
• Design, Implement & Manage cost going down
– Internet tools, Database tools– DCOM & Transactions, clusters.
• Billions of clients imply millions of HUGE servers.
• Thin clients imply huge servers.
ServerServer
5
1987: 256 tps $ 14 million computerA dozen peopleTwo rooms of machines
1997: 1,250 tps $ 50 k$ computerOne person1 micro-dollar per transaction (1,000x cheaper)
What Happens in 10 Years?
Ready for the next 10 years?
6
Q: Why Does Microsoft Care? A: Billions of clients need millions of servers
Microsoft wants to do a GREAT! job on Scaleable Windows NT and Scaleable BackOffice.
Key technique: INTEGRATION.
0300600900
1,2001,5001,8002,1002,4002,700
1994 1995 1996 1997 1998 1999 2000 2001
UnixUnix
WindowsNT WindowsNT ServerServer
NetWareNetWare
Servers Shipped per year
(97-01 are MS estimates)
7
OutlineOutline
• Scalability: What & Why?• Scale UP: NT SMP scalability• Scale OUT: NT Cluster scalability
Scale OutScale Out
Scale Up
Scale DownDown
8
How Scaleable is NT??The Single Node Story
• 64 bit file system in NT 1, 2, 3, 4, 5• 8 node SMP in NT 4.E, 32 node OEM• 64 bit addressing in NT 5• 1 Terabyte SQL Databases (PetaByte capable)
• 10,000 users (TPC-C benchmark)• 100 Million web hits per day (IIS)• 50 GB Exchange mail store
next release designed for 16 TB
• 50,000 POP3 users on Exchange (1.8 M messages/day)
• And, more coming…..
9
Windows NT ServerEnterprise Edition
• Scalability – 8x SMP support (32x in OEM kit)– Larger process memory (3GB Intel)– Unlimited Virtual Roots in IIS (web)
• Transaction– DCOM transactions (Viper TP mon) – Message Queuing (Falcon)
• Availability– Clustering (WolfPack)– Web, File, Print, … servers fail over.
10
Manageability Windows NT 5.0 and Windows 98
• Active Directory tracks all objects in net• Integration with IE 4.
–Web-centric user interface• Management Console
–Component architecture• Zero Admin Kit and Systems
Management Server• Plug&Play, Instant On, Remote Boot,..• Hydra and Intelli-Mirroring
11
Windows NT ServerWindows NT Serverwith “Hydra” Serverwith “Hydra” Server
Dedicated Dedicated Windows Windows terminalterminal
Existing, Existing, Desktop PC Desktop PC
MS-DOS, MS-DOS, UNIX, UNIX, Mac Mac clientsclients
Net PCNet PC
Thin Client Supportlower per-client costs
12
The benefits of the PC plus The benefits of the PC plus centralized computing advantagescentralized computing advantages
Windows NT 5.0IntelliMirror™
• Files and settings mirrored on client and server
• Great for disconnected users• Facilitates roaming• Easy to replace PCs• Optimizes network performance
13
Scaleup To Big Databases?• NT 4 and SQL Server 6.5
– DBs up to 1 Billion records, – 100 GB– Covers most (80%) data warehouses
• SQL Server 7.0 – Designed for Terabytes
• Hundreds of disks per server.• SMP parallel search
– Data Mining and Multi-Media– Multi-Media is probably main
• TerraServer is good example
TTBB
ExcelExcelspreadsheetspreadsheet
Manhattan phone book Manhattan phone book (15MB)(15MB)
Human GenomeHuman Genome (3GB) (3GB)
Dayton-HudsonDayton-HudsonSales recordsSales records(300GB)(300GB)
SatelliteSatellitephotos of photos of
Earth (1 TB)Earth (1 TB)
14
Database ScaleUP: TerraServer™• Build a 1 TB SQL Server database
– Demo NT and SQL Server scalability– Stress test SQL Server 7.0
• Data must be– 1 TB– Unencumbered– Interesting to everyone everywhere– And not offensive to anyone anywhere
• Loaded – 1.1 M place names from Encarta World Atlas– 1 M Sq Km from USGS (1 meter resolution)– 2 M Sq Km from Russian Space agency (2 m)
• Will be on web (world’s largest atlas)• Sell images with commerce server.• USGS CRDA: 3 TB more coming.
15
TerraServer System
• DEC Alpha 4100 (4x smp) + • 324 StorageWorks Drives (1.4 TB)• RAID 5 Protected• SQL Server “Sphinx”• USGS 1-meter data
(30% of US)• Russian Space data
Two meterresolutionimages(2 M km2 2% of earth)
SPIN-2SPIN-2
16
Demo by Tom Barclay
Http://t2b2
17
How Can We Measure Scalability?• SpecMarks™ ?
–Just measures processor• Transactions per Minute
– Transaction Processing Performance Council– Measures whole system– Price and Performance– 63 data points
• 3 year history• all vendors
– audited results– www.tpc.org
tpmC vs Time
05,000
10,00015,00020,00025,00030,00035,000
Jan-95 Jan-96 Jan-97
tpm
C
18
NT vs UNIX SMPs• NT typically runs on 1 to 4 cpus
– Scales near-linear on them• UNIX boxes: 32-64 way SMPs
– They do 3x more tpmC– They cost 10x more.
• 10 way NT machines are available– They cost more– They are faster
• My view (shared by many)– Need clusters for availability– Cluster commodity servers to make huge systems– a la Tandem, Teradata, VMScluster, IBM Sysplex, IBM SP2
tpmC vs Time
05,000
10,00015,00020,00025,00030,00035,000
Jan-95 Jan-96 Jan-97
tpm
C
tpmC vs Time
05,000
10,00015,00020,00025,00030,00035,000
Jan-95 Jan-96 Jan-97
tpm
C
hh
UnixNT
tpmC vs Time
05,000
10,00015,000
20,00025,000
30,00035,000
Jan-95 Jan-96 Jan-97
tpm
C
hUnix
NT
19
Transaction Throughput TPC-C• On comparable hardware, NT scales best• SQL Server & NT Improving 250% per year• Best Price Performance (2x cheaper)tpmC on Intel CPUs
0
2,000
4,000
6,000
8,000
10,000
12,000
14,000
0 1 2 3 4 5 6 7 8 9 10
tpm
C
NT
UNIX
h
h
hhhh
tpmC vs Intel CPUs
0
2,000
4,000
6,000
8,000
10,000
12,000
14,000
0 1 2 3 4 5 6 7 8 9 10
tpm
C
NT all
NT Best
Unix best
hh
20
NT Scales Better Than Solaris• Microsoft SQL
NTIntel scales to 6x
• Beats Sybase Solaris UltraSPARCup to 11-way
0
5,000
10,000
15,000
20,000
0 10 20cpus
tpm
C
Sybase/Solaris
/UltraSPARC
MS
SQL/N
T/Inte
l
21
Only NT Has Economy of Scale
• NT is 2x less expensive
• Only NT has economy of scale
• Unix has dis-economy of scale
Transactions/k$ by vendor
0.0
5.0
10.0
15.0
20.0
25.0
0 10,000 20,000 30,000tpmC
tpm
C/k
$
DB2/UnixSybase/Unix
Informix/Unix
Microsoft/NT
Oracle/Unix
22
What Does This Mean?
NT
10,000 usersLess than 10,000 usersuse NT
?
20,000 users
?
30,000 users10k-30K users
consider UNIXrequire high availability
=> clusters (scale out)
NT*
NT*
40,000 users
NT*
50,000 users30k-1M users require cluster (scale out)require high availability (clusters)
NT*NT*NT*NT*NT*
NT*NT*NT*NT*
NT*NT*NT*
23
What Does This Mean?
Less than 10,000 usersuse NT
10k-30K usersconsider UNIXrequire high availability
=> clusters (scale out)
30k-1M band require cluster (scale out)require high availability (clusters)
Need h
igh-a
vaila
bility
clu
ster
Need h
igh-a
vaila
bility
clu
ster
even
at 1
0,00
0 use
rs
even
at 1
0,00
0 use
rs
10,000 users
20,000 users
30,000 users
40,000 users
50,000 users
24
OutlineOutline
• Scalability: What & Why?• Scale UP: NT SMP scalability• Scale OUT: NT Cluster scalability• Key Message:
– NT can do the most demanding apps today.
– Tomorrow will be even better.
Scale OutScale Out
Scale Up
Scale DownDown
25
Clusters Have Advantages• High Availability: Fault tolerance:
– Spare modules mask failures
• Modular growth– Grow by adding small modules
• Parallel data search– Use multiple processors and disks
• Clients and servers made from the same stuff– Inexpensive: built with commodity CyberBricks
26
How scaleable is NT??The Cluster Story
• 16-node Tandem Cluster– 64 cpus– 2 TB of disk– Decision support
• 45-node Compaq Cluster– 140 cpus– 14 GB DRAM– 4 TB RAID disk– OLTP (Debit Credit)
• 1 B tpd (14 k tps)
27
microsoft.com
• Production
– Windows NT V4 and IIS V3• 20 HTTP, • 3 download, • 3 FTP• 5 SQL 6.5• Index Server + 3 search
• Stagers– Site Server for content– DCOM Publishing wizard
• Network– 4 100MB FDDI rings– 6 DS3– 4 TB/day download capacity
• Replicas in UK and Japan
• 90m hits/day– 16.5m page views– #4 site on Internet
• 900k visitors per day• Not cheap
– Data Centers– Bandwidth– 27 people on content – 22 people on systems
28
Tandem 2 Ton
• 2 TB SQL database • 1.2 TB user data• 16 node cluster• 64 cpus, 480 disks• Decision support
parallel data-mining
• Will be Wolf Pack aware• Demoed at DB Expo in• ServerNet™ interconnect
29
Billion Transactions per Day Demo
• Built a 45-node Windows NT Cluster (with help from Intel & Compaq) > 900 disks, 3 TB
• All off-the-shelf parts• Using SQL Server &
DTC distributed transactionsDCOM & ODBC clientson 20 front-end nodes
• DebitCredit Transaction• Each server node has 1/20 th of the DB • Each server node does 1/20 th of the work• 15% of the transactions are “distributed”
30
Cluster Architecture
VIPDTC1 VIPDTC2 VIPDTC3 VIPDTC4 VIPDTC5
ControlDriver
DatabaseSwitchDTC
VIPDC42 VIPDC43 VIPDC44 VIPDC45 VIPDC46 VIPDC47 VIPDC48 VIPDC49 VIPDC50 VIPDC51
VIPDC2 VIPDC3 VIPDC4 VIPDC5 VIPDC6 VIPDC7 VIPDC8 VIPDC9 VIPDC10 VIPDC11
VIPDC12 VIPDC13 VIPDC14 VIPDC15 VIPDC16 VIPDC17 VIPDC18 VIPDC19 VIPDC20 VIPDC21
31
1.2 B tpd• 1 B tpd ran for 24 hrs.• Out-of-the-box software• Off-the-shelf hardware• AMAZING!
• Sized for 30 days• Linear growth• 5 micro-dollars per transaction
32
Millions of Transactions Per Day
0.1
1.
10.
100.
1,000.
1 Btpd Visa ATT BofA NYSE
Mtp
d
Millions of Transactions Per Day
0.100.200.300.400.500.600.700.800.900.
1,000.
1 Btpd Visa ATT BofA NYSE
Mtp
d
How Much Is 1 Billion Tpd?• 1 billion tpd = 11,574 tps
~ 700,000 tpm (transactions/minute)• ATT
– 185 million calls per peak day (worldwide)
• Visa ~20 million tpd– 400 million customers– 250K ATMs worldwide– 7 billion transactions
(card+cheque) in 1994 • New York Stock Exchange
– 600,000 tpd• Bank of America
– 20 million tpd checks cleared (more than any other bank)– 1.4 million tpd ATM transactions
• Worldwide Airlines Reservations: 250 Mtpd
33
Cluster Server: High Availability• Multiple servers form one system• Industry standard APIs and hardware• Server application and tools support
– IIS web server– File and Print servers– IP and NetName failover– Transaction and Queue Server failover– SQL Server, Enterprise edition
• Two-Node clusters now (2 to 20 cpus)• 16 node soon (2 to 192 cpus).
34
Web Web sitesite
DatabaseDatabase
Web site filesWeb site files
Database filesDatabase files
BrowserBrowser
HP-WolfPack ClusterIIS & SQL High Availability
Web Web sitesite
DatabaseDatabase
AliceAlice BettyBetty
35
SQL Server and IISSymmetric Virtual Servers
Web Web sitesite
DatabaseDatabase
Web site filesWeb site files
Database filesDatabase files
BrowserBrowser
Web Web sitesite
DatabaseDatabase
AliceAlice BettyBetty
Web site filesWeb site files
Database filesDatabase files
• Both Servers deliver service.• Almost no overhead during normal operation.• Each protects the other.• High availability:
– Planned outages for maintenance
– Unplanned outages (failures)
36
3 Tier WebWindows Windows TerminalTerminal BrowserBrowser NetPCNetPC WindowsWindows OtherOther
Data Stores: SQL, Messaging, ISAM, Host, Unstructured StorageData Stores: SQL, Messaging, ISAM, Host, Unstructured Storage
A consistent programming model for integrating all data, services and componentsA consistent programming model for integrating all data, services and components
““K2”K2”Services: Transactions, Security, Message QueuingServices: Transactions, Security, Message Queuing
COM and other standard protocolsCOM and other standard protocols
COM and other standard protocolsCOM and other standard protocols
ASP
ASP
ASP
ObjectObject
Use ASPs for presentation
Use COM for Business
Rules
37
Data Access Within NTWindows Windows TerminalTerminal BrowserBrowser NetPCNetPC WindowsWindows OtherOther
TransactionTransaction
ODBCODBCResourceResource
Dispenser Dispenser
SQLSQLServerServer
SQLSQLServerServer
Object
Object
38
Cedar: Accessing Legacy DataWindows Windows TerminalTerminal BrowserBrowser NetPCNetPC WindowsWindows OtherOther
TransactionTransaction
ODBCODBC
SQLSQLServerServer
SQLSQLServerServer
CedarCedar
IMSIMS CICSCICS
SNA ServerSNA Server
LU 6.2LU 6.2
Object
Object
39
MSMQ: Queued Processing Windows Windows TerminalTerminal BrowserBrowser NetPCNetPC WindowsWindows OtherOther
TransactionTransaction
ODBCODBC
SQLSQLServerServer
SQLSQLServerServer
CedarCedar
IMSIMS CICSCICS
SNA ServerSNA Server
MSMQMSMQ
MSMQMSMQ
LU 6.2LU 6.2
Object
Object
40
Accessing UNIX DBMS’sWindows Windows TerminalTerminal BrowserBrowser NetPCNetPC WindowsWindows OtherOther
TransactionTransaction
ODBCODBC
SQLSQLServerServer
SQLSQLServerServer
CedarCedar
IMSIMS CICSCICS
SNA ServerSNA Server
MSMQMSMQ
MSMQMSMQ DB2, OracleDB2, OracleInformixInformix
XAXALU 6.2LU 6.2
Object
Object
Third parties coming
41
DCOM Completes the PictureWindows Windows TerminalTerminal BrowserBrowser NetPCNetPC WindowsWindows OtherOther
Data Stores: SQL, Messaging, ISAM, Host, Unstructured StorageData Stores: SQL, Messaging, ISAM, Host, Unstructured Storage
A consistent programming model for integrating all data, services and componentsA consistent programming model for integrating all data, services and components
““K2”K2”Services: Transactions, Security, Message QueuingServices: Transactions, Security, Message Queuing
COM and other standard protocolsCOM and other standard protocols
COM and other standard protocolsCOM and other standard protocols
ASP
ASP
ASP
ObjectObject
Object
Object
Object
42
•20% Performance Advantage, Cluster Aware20% Performance Advantage, Cluster Aware•OLAP Support, Natural Language InterfaceOLAP Support, Natural Language Interface•Available: 97Q3 (US), Loc: F, G, Japanese Available: 97Q3 (US), Loc: F, G, Japanese
•Merchant ServerMerchant Server•Usage analyst Usage analyst •Available: 97Q3 (US)Available: 97Q3 (US)
•All Enterprise EditionsAll Enterprise Editions•Multi-ServerMulti-Server•Available: 97Q4Available: 97Q4
•Designed for Terabyte message stores Designed for Terabyte message stores •Cluster AwareCluster Aware•Available 97Q4 (US)Available 97Q4 (US)
2.0/E2.0/E
2.5/E2.5/E
xx /Exx /E
6.5/E6.5/E
BackOffice Enterprise Road Map
43
SummarySummary
• Scalability: What & Why?• Scale UP: NT SMP scalability• Scale OUT: NT Cluster scalability• Key Message:
– NT can do the most demanding apps today.
– Tomorrow will be even better.
Scale OutScale Out
Scale Up
Scale DownDown
44
Questions?
• Slides at:
– http://www.Research.Microsoft.com/~Gray
• I will be on show floor: 11:00: Microsoft booth
Mail to [email protected]
45
Cluster-Wide Load Balance
Package A Package BPackage B Package CPackage C Package A Package A Package A
Static Load BalancingStatic Load Balancing
IISPackage APackage A IIS Package APackage A
DNS Round RobinDNS Round Robin Dynamic Load BalancingDynamic Load Balancing
Package A Package A Package A