View
216
Download
1
Tags:
Embed Size (px)
Citation preview
Computer Science
Scalability of Linux Event-Dispatch Mechanisms
Abhishek Chandra
University of Massachusetts
Amherst
David Mosberger
Hewlett Packard Labs
Palo Alto
Computer Science
Motivation
Large Web and Internet traffic Heavily Loaded/Accessed Web Servers
• cnn.com, britneyspearsfans.com, …• Starr Report, Napster ruling, ...
Challenge: Make Web Servers Scalable
ClientsWANWeb
Server
Computer Science
Server Scalability Issues
Large number of concurrent/idle connections Last-mile problem: Slow end-connections High latency WAN traffic HTTP/1.1 Persistent Connections
Heavy Request Loads Need for high throughput
Pure Thread-based vs. Event-based servers
Focus: Scalability of Event-based servers on Linux
Computer Science
Outline
Motivation
Event-based Servers
Linux Event-Dispatch Mechanisms
Evaluation: Handling concurrent connections
RT signals and Signal-per-fd enhancement
Evaluation: Handling request load
Concluding Remarks
Computer Science
1. Interest SetSpecification
Interest Set
Event-Based Servers
Server specifies Interest Set to Kernel Kernel notifies Server of Event on a connection Server handles I/O on the connection
Kernel
Server
Connections
2. Network Event
3. Event Notification
4. I/O Handling
Computer Science
Linux Event-Dispatch Mechanisms
select() system call
poll() system call
/dev/poll interface
POSIX.4 Real-Time Signals
Computer Science
Interest Set
select() system call
Kernel
Server
Connections
Ready Set
Scan
Interest Set specified on each call Notification requires scan of interest set
Computer Science
poll() and /dev/poll
Interest Set: List of pollfd structures
• Better for sparse interest sets, worse for dense sets
Notification Requires scan of Interest Set
/dev/poll: Interest Set specified incrementally More compact ready set
Computer Science
POSIX.4 Real Time Signals
RT signals are queued Multiple signals of same type can be delivered
RT signals carry a data payload (siginfo) Provides the context of the signal
sigwaitinfo() system call: Dequeues signals Avoids overhead of calling signal handler Signal can be blocked
Computer Science
1. AssociateRT Signal
Interest Set
Using Real Time Signals for Network I/O
Interest Set specified incrementally No scanning of Interest Set required
Kernel
Server
Sockets
2. Network Event
4.sigwaitinfo()
3. EnQueueSignal
5. DeQueueSignalSignal
Queue
Computer Science
Outline
Motivation
Event-based Servers
Linux Event-Dispatch Mechanisms
Evaluation: Handling concurrent connections
RT signals and Signal-per-fd enhancement
Evaluation: Handling request load
Concluding Remarks
Computer Science
Evaluation: Handling Concurrent Connections
Dispatch overhead and latency as a function of number of concurrent connections
Experimental Setup 400 MHz P3 Linux 2.4.0-test7 server μ -server using select(), /dev/poll or RT signals 10 clients running httperf Fixed request rate, increasing number of connections
Computer Science
Server CPU Usage
0
10
20
30
40
50
60
70
80
90
100
0 500 1000 1500 2000 2500 3000No. of concurrent connections
CP
U u
sa
ge
(%
)
select
/dev/poll
RTsignals
RT signal overhead independent of no. of concurrent connections
500 req/s
Computer Science
Response Time
RT signal response time independent of no. of concurrent connections
0
2
4
6
8
10
12
14
16
18
20
0 500 1000 1500 2000 2500 3000
No. of concurrent connections
Re
sp
on
se
tim
e (
ms
)select
/dev/poll
RTsignals
500 req/s
Computer Science
Limitations of Real Time Signals
Signal Queue Overflow:• New events lost
• Can lead to “hung server”
Unfair Allocation of Signal Queue
Kernel
Server
Interest Set
Sockets
Network Event
3 2
SignalQueue
1 2 3 4
3
Network Event
Drop
Network Event
3
Computer Science
Handling Signal Queue Overflow
Fallback mechanism select(), poll(), etc. Reconstruct current state
Issues Server complexity Overhead of maintaining explicit interest sets Potential performance penalty
Computer Science
RT Signal Enhancement: Signal-per-fd
Goals: Avoid signal queue overflows Fair Allocation of signal queue
Solution: Enqueue only one signal per socket
Kernel
Server
Interest Set
Sockets
Network Event
2
SignalQueue
1 2 3 4
3
Network Event
Discard
Network Event
1
Computer Science
Signal-per-fd
Idea: Signal queue length same as fdset size Bitmap used to efficiently determine
presence/absence of signal in queue Advantages:
Simpler Server Implementation• No signal queue overflows• No need for fallback mechanisms
Fair Allocation of Signal Queue Resource Avoids too fine-grained event notification
• Coalesce multiple events for a socket
Computer Science
Outline
Motivation
Event-based Servers
Linux Event-Dispatch Mechanisms
Evaluation: Handling concurrent connections
RT signals and Signal-per-fd enhancement
Evaluation: Handling request load
Concluding Remarks
Computer Science
Server Throughput
Linear scaling of RT signals, signal-per-fd
0
500
1000
1500
2000
2500
3000
3500
0 1000 2000 3000 4000 5000
Request Rate (req/s)
Th
rou
gh
pu
t (r
ep
/s)
selectRTsignalssignal-per-fd
6000 idle connections
Computer Science
Server CPU Usage
Linear Scaling of RT signals, signal-per-fd
0
10
20
30
40
50
60
70
80
90
100
0 1000 2000 3000 4000 5000Request Rate (req/s)
CP
U U
sa
ge
(%
)
select
RTsignals
signal-per-fd
6000 idle connections
Computer Science
Related Work
Event-Delivery API [BMD99] Performance studies:
select() [BM98], /dev/poll [PL00] RT signals [PLT00]
Web Servers: Event-based: Flash [PDZ99], phhttpd [Brown99] In-kernel: TUX, khttpd, AFPA [JKNRT01]
Future: Linux 2.5 Asynchronous I/O?
Computer Science
Summary Scalability issues with Linux Event-dispatch mechanisms Real Time Signals are scalable
• Performance independent of number of concurrent connections• Signal Queue Overflow Problems
Signal-per-fd enhancement• potentially improves performance • reduces server complexity• provides fairness
Patch available at http://www.netli.com/links