Upload
cassandra-mckee
View
47
Download
5
Tags:
Embed Size (px)
DESCRIPTION
I/O Interfaces, A Little Queueing Theory RAID. Vincent H. Berk November 11, 2005 Reading for Today: Sections 7.1 – 7.4 Reading for Monday: Sections 7.5 – 7.9, 7.14 Homework for Friday Nov 18: 5.4, 5.17, 6.4, 6.10, 7.3, 7.21, 8.9/8.10, 8.17. Common Bus Standards. ISA PCI AGP PCMCIA - PowerPoint PPT Presentation
Citation preview
ENGS 116 Lecture 18 1
I/O Interfaces,A Little Queueing Theory
RAID
Vincent H. Berk
November 11, 2005
Reading for Today: Sections 7.1 – 7.4
Reading for Monday: Sections 7.5 – 7.9, 7.14
Homework for Friday Nov 18: 5.4, 5.17,
6.4, 6.10, 7.3, 7.21, 8.9/8.10, 8.17
ENGS 116 Lecture 18 2
Common Bus Standards
• ISA
• PCI
• AGP
• PCMCIA
• USB
• FireWire/IEEE 1394
• IDE
• SCSI
3ENGS 116 Lecture 18
Programmed I/O (Polling)
CPU
IOC
Device
Memory
Is thedata
ready?
readdata
storedata
yesno
done? no
yes
Busy wait loopnot an efficientway to use the CPUunless the deviceis very fast!
However, checks for I/O completion can be dispersed among computationallyintensive code.
4ENGS 116 Lecture 18
User program progress only halted during actual transfer
1000 transfers at 1 ms each:1000 interrupts @ 2 µ sec per interrupt1000 interrupt service @ 98 µ sec each = 0.1 CPU seconds
Device transfer rate = 10 MBytes/sec 0.1 x 10-6 sec/byte 0.1 µsec/byte 1000 bytes = 100 µsec 1000 transfers 100 µsecs = 100 ms = 0.1 CPU seconds
Still far from device transfer rate! 1/2 time in interrupt overhead
Interrupt-Driven Data TransferCPU
IOC
device
Memory
addsubandornop
readstore...rti
memory
userprogram
(1) I/Ointerrupt
(2) save PC
(3) interruptservice addr
interruptserviceroutine(4)
5ENGS 116 Lecture 18
Direct Memory Access (DMA)Time to do 1000 transfers at 1 msec each:
1 DMA set-up sequence @ 50 µ sec1 interrupt @ 2 µ sec1 interrupt service sequence @ 48 µ sec
.0001 second of CPU time
CPU sends a starting address, direction, and length count to DMAC. Then issues "start".
DMAC provides handshake signals for peripheralcontroller and memory addresses and handshakesignals for memory.
CPU
IOC
device
Memory DMAC
0ROM
RAM
Peripherals
DMACn
Memory Mapped I/O
6ENGS 116 Lecture 18
Input/Output Processors
CPU IOP
Mem
D1
D2
Dn
. . .main memory
bus
I/Obus
CPU
IOP
issues instruction to IOP
interrupts when done(1)
memory
(2)
(3)
(4)
Device to/from memorytransfers are controlledby the IOP directly.
IOP steals memory cycles.
OP Device Address
target devicewhere commands are
looks in memory for commands
OP Addr Cnt Other
whatto do
whereto putdata
howmuch
specialrequests
ENGS 116 Lecture 18 7
Summary
• Disk industry growing rapidly, improves bandwidth and areal density
• Time = queue + controller + seek + rotate + transfer
• Advertised average seek time much greater than average seek time in practice
• Response time vs. bandwidth tradeoffs
• Processor interface: today peripheral processors, DMA, I/O bus, interrupts
ENGS 116 Lecture 18 8
A Little Queueing Theory
• More interested in long-term, steady-state behavior
Arrivals = Departures
• Little’s Law: mean number of tasks in system
= arrival rate mean response time
– Observed by many, Little was first to prove
– Applies to any system in equilibrium as long as nothing inside the system (black box) is creating or destroying tasks
• Queueing models assume state of equilibrium:
input rate = output rate
Arrivals
System
Departures
ENGS 116 Lecture 18 9
A Little Queueing Theory
• Avg. arrival rate
• Avg. service rate • Avg. number in system N
• Avg. system time per customer T = avg. waiting time + avg. service time
• Little’s Law: N = T
• Service utilization = /
Departures
ServerQueue
Arrivals
ENGS 116 Lecture 18 10
A Little Queueing Theory
• Server spends a variable amount of time with customers
– Service distribution characterized by mean, variance, squared coefficient of variance (C)
– Squared coefficient of variance: C = variance/mean2, unitless measure
• Exponential distribution: C = 1; most short relative to average
• Hypoexponential distribution: C < 1; most close to average
• Hyperexponential distribution: C > 1; most further from average
• Disk response times: C ≈ 1.5, but usually pick C = 1 for simplicity
ENGS 116 Lecture 18 11
Average Residual Service Time
How long does a new customer wait for the current customer to finish service?
Average residual time =
If C = 0, average residual service time = 1/2 mean service time
1
2Mean service time 1 C
ENGS 116 Lecture 18 12
Average Wait Time in Queue
• If something at server, it takes average residual service time to complete
• Probability that server is busy is • All customers in line must complete, each averaging Tservice
• If exponential distribution, C = 1 and
Tq Tservice 1 C
2 1
Tq Tservice
1
ENGS 116 Lecture 18 13
M/M/1 and M/G/1
• Assumptions so far:
– System in equilibrium
– Time between two successive arrivals in line are random
– Server can start on next customer immediately after prior customer finishes
– No limit to the queue, FIFO service
– All customers in line must complete, each averaging Tservice
• Exponential distribution (C = 1) is “memoryless” or Markovian, denoted by M
• Queueing system with exponential arrivals, exponential service times, and 1 server: M/M/1
• General distribution is denoted by G: can have M/G/1 queue
ENGS 116 Lecture 18 14
An Example
Processor sends 10 8-KB disk I/Os per second, requests and service times exponentially distributed, avg. disk service = 20 ms.
ENGS 116 Lecture 18 15
Another Example
Processor sends 20 8-KB disk I/Os per second, requests and service times exponentially distributed, avg. disk service = 12 ms.
ENGS 116 Lecture 18 16
Yet Another Example
Processor sends 10 8-KB disk I/Os per second, C = 1.5, avg. disk service = 20 ms.
ENGS 116 Lecture 18 17
Manufacturing Advantages of Disk Arrays
3.5”
Disk Array: 1 disk design
14”10”5.25”3.5”
Conventional: 4 disk designs
Low End High End
Disk Product Families14”
18ENGS 116 Lecture 18
Replace Small # of Large Disks withLarge # of Small Disks!
Data Capacity
Volume
Power
Data Rate
I/O Rate
MTBF
Cost
IBM 3390 (K)
20 GBytes
97 cu. ft.
3 KW
15 MB/s
600 I/Os/s
250 KHrs
$250K
IBM 3.5" 0061
320 MBytes
0.1 cu. ft.
11 W
1.5 MB/s
55 I/Os/s
50 KHrs
$2K
x70
23 GBytes
11 cu. ft.
1 KW
120 MB/s
3900 IOs/s
??? Hrs
$150K
Disk Arrays have potential for
large data and I/O rates
high MB per cu. ft., high MB per KW
awful reliability
19ENGS 116 Lecture 18
Array Reliability
• Reliability of N disks = Reliability of 1 Disk ÷ N
50,000 Hours ÷ 70 disks = 700 hours
Disk system MTTF: Drops from 6 years to 1 month!
• Arrays without redundancy too unreliable to be useful!
Hot spares support reconstruction in parallel with access: very high media availability can be achieved
20ENGS 116 Lecture 18
Mirroring/Shadowing (high capacity cost)
Horizontal Hamming Codes (overkill)
Parity & Reed-Solomon Codes
Failure Prediction (no capacity overhead!)VaxSimPlus — Technique is controversial
Techniques:
Redundant Arrays of Disks
• Files are "striped" across multiple spindles
• Redundancy yields high data availabilityDisks will failContents reconstructed from data redundantly stored in the
array Capacity penalty to store it Bandwidth penalty to update
21ENGS 116 Lecture 18
Redundant Arrays of Disks (RAID) Techniques
• Disk Mirroring, ShadowingEach disk is fully duplicated onto its "shadow" Logical write = two physical writes100% capacity overhead
• Parity Data Bandwidth ArrayParity computed horizontallyLogically a single high data bandwidth disk
• High I/O Rate Parity ArrayInterleaved parity blocksIndependent reads and writesLogical write = 2 reads + 2 writesParity + Reed-Solomon codes
10010011
11001101
10010011
00110010
10010011
10010011
Parity disk
22ENGS 116 Lecture 18
RAID 0: Disk Striping
• Data is distributed over disks
• Improved bandwidth and seek time on read and write• Larger virtual disk
• No redundancy
23ENGS 116 Lecture 18
RAID 1: Disk Mirroring/Shadowing
• Each disk is fully duplicated onto its "shadow" Very high availability can be achieved
• Bandwidth sacrifice on write: Logical write = two physical writes• Half seek time on reads
• Reads may be optimized
• Most expensive solution: 100% capacity overhead
Targeted for high I/O rate, high availability environments
recoverygroup
24ENGS 116 Lecture 18
RAID 3: Parity Disk
P
100100111100110110010011
. . .logical record 1
0010011
11001101
10010011
00110000
Striped physicalrecords
• Parity computed across recovery group to protect against hard disk failures 33% capacity cost for parity in this configuration wider arrays reduce capacity costs, decrease expected availability, increase reconstruction time
• Arms logically synchronized, spindles rotationally synchronized logically a single high capacity, high transfer rate disk
Targeted for high bandwidth applications
25ENGS 116 Lecture 18
RAID 4 & 5: Block-Interleaved Parity and Distributed Block-Interleaved Parity
• Similar to RAID 3, requiring same number of disks.• Parity is computed over blocks and stored in blocks.• RAID 4 places parity on last disk• RAID 5 places parity blocks distributed over all disks:
Advantage: parity block is always accesses on read/writes• Parity is updated by reading the old-block and parity-block, and writing
the new-block and the new parity-block. (2 reads, 2 writes)
26ENGS 116 Lecture 18
Disk access advantage RAID 4/5 over RAID 3
27ENGS 116 Lecture 18
Problems of Disk Arrays: Small Writes
D0 D1 D2 D3 PD0'
+
+
D0' D1 D2 D3 P'
newdata
olddata
old parity
XOR
XOR
(1. Read) (2. Read)
(3. Write) (4. Write)
RAID-5: Small Write Algorithm1 Logical Write = 2 Physical Reads + 2 Physical Writes
ENGS 116 Lecture 18 28
I/O Benchmarks: Transaction Processing
• Transaction Processing (TP) (or On-line TP = OLTP)– Changes to a large body of shared information from many terminals, with
the TP system guaranteeing proper behavior on a failure
– If a bank’s computer fails when a customer withdraws money, the TP system would guarantee that the account is debited if the customer received the money and that the account is unchanged if the money was not received
– Airline reservation systems & banks use TP
• Atomic transactions makes this work
• Each transaction 2 to 10 disk I/Os and 5,000 to 20,000 CPU instructions per disk I/O
– Efficient TP software, avoid disks accesses by keeping information in main memory
• Classic metric is Transactions Per Second (TPS) – Under what workload? How is machine configured?
ENGS 116 Lecture 18 29
I/O Benchmarks: Transaction Processing
• Early 1980s great interest in OLTP– Expecting demand for high TPS (e.g., ATM machines, credit cards)
– Tandem’s success implied medium range OLTP expands
– Each vendor picked own conditions for TPS claims, reported only CPU times with widely different I/O
– Conflicting claims led to disbelief of all benchmarks chaos
• 1984 Jim Gray of Tandem distributed paper to Tandem employees and 19 in other industries to propose standard benchmark
• Published “A measure of transaction processing power,” Datamation, 1985 by Anonymous et. al
– To indicate that this was effort of large group
– To avoid delays of legal department of each author’s firm
– Author still gets mail at Tandem
ENGS 116 Lecture 18 30
I/O Benchmarks: TP by Anon et. al
• Proposed 3 standard tests to characterize commercial OLTP– TP1: OLTP test, DebitCredit, simulates ATMs (TP1)
– Batch sort
– Batch scan
• Debit/Credit:– One type of transaction: 100 bytes each
– Recorded 3 places: account file, branch file, teller file; events recorded in history file (90 days)
• 15% requests for different branches
– Under what conditions, how to report results?
ENGS 116 Lecture 18 31
I/O Benchmarks: TP1 by Anon et. al
• DebitCredit Scalability: size of account, branch, teller, history function of throughput TPS Number of ATMs Account-file size
10 1,000 0.1 GB
100 10,000 1.0 GB
1,000 100,000 10.0 GB
10,000 1,000,000 100.0 GB
– Each input TPS 100,000 account records, 10 branches, 100 ATMs
– Accounts must grow since a person is not likely to use the bank more frequently just because the bank has a faster computer!
• Response time: 95% transactions take ≤ 1 second
• Configuration control: just report price (initial purchase price + 5 year maintenance = cost of ownership)
• By publishing, in public domain