Upload
xen-project
View
890
Download
3
Embed Size (px)
DESCRIPTION
Event Channels are Xen's mechanism for paravirtualized interrupts. These were limited to only 4096 which then limits the number of guests that a host may support to around 300 to 500. This presentation will give a brief introduction to event channels, a detailed look at the new, innovative FIFO-based event channel ABI that increases the limit to over 100,000 as well as having several other useful features (e.g., multiple priorties). Some of the key performance measurments of the new ABI will be shown.
Citation preview
“Unlimited” Event Channels
David Vrabel
24 October 2013
What are Event Channels?
I Paravirtual interrupts
I Edge triggered
I Bidirectional
I Directed at a single VCPU
24 October 2013 “Unlimited” Event Channels 2 / 13
How do Event Channels Work?
Domain BDomain A
Shared MemoryNotify
Set Pending Upcall
1
2 3
4 Call Handlers
Xen
24 October 2013 “Unlimited” Event Channels 3 / 13
Original (2-Level) Design
Masked
Pending ...
... ...
... ...
...
...Selector
Shared Memory
Per-VCPU Shared Memory
4096 bits
64 bits
64 bits
1
0 1 0 0 0 1 0
24 October 2013 “Unlimited” Event Channels 4 / 13
Problems
I Too few
I No priorities
I Unfair
24 October 2013 “Unlimited” Event Channels 5 / 13
Additional Requirements
I Identical ABI between 32-bit and 64-bit guests
I Reasonable memory usage
I Easily extensible
24 October 2013 “Unlimited” Event Channels 6 / 13
New (FIFO-based) Design
P M L LINK
1
1
1
1
1
1
6
0
3
Up to 132,072 words
1
...
HEAD
0
0
0
READY
Per-VCPU shared memory Shared memory
Port 0(Reserved)
1
2
3
4
5
6
7
Bit 15 01
Priority 0
1
15
1Bit 31 030 29 16
...
24 October 2013 “Unlimited” Event Channels 7 / 13
Raising an Event
function raise(q, p)
E[p].pending = 1 if not E[p].masked and not E[p].linked E[p].linked = 1 if T[q] != p linked = link(T[q], p) else linked = false if not linked C.head[q] = p T[q] = p
1. Set pending
2. Unmasked and not in list? Add to list
3. Set LINK of tail or Set HEAD in control block
4. Advance tail
24 October 2013 “Unlimited” Event Channels 8 / 13
Handling an Event
function handle_one_event(q)
p = H[q] if p == 0 p = C.head[q] link = unlink(p) H[q] = link if E[p].pending and not E[p].masked handle(p)
1. Get local head
2. List was empty? Get new HEAD from control block
3. Clear LINKED and LINK
4. Advance local head
5. Pending and unmasked? Handle IRQ
24 October 2013 “Unlimited” Event Channels 9 / 13
Event Channel Fairness
0
500
1000
1500
2000
2500
3000
75 80 85 90
Late
ncy
(ns)
Event Channel Port
2-Level
FIFO-based
24 October 2013 “Unlimited” Event Channels 10 / 13
Current Status
I Xen support in 4.4
I Linux support in 3.14
24 October 2013 “Unlimited” Event Channels 11 / 13
Further Work
I Investigate how priorities may be used
I Add support for limiting number of event channels to other toolstacks (XAPI,libvirt)
I Add support in other OSes
I Xen event lock scalability
I Linux PIRQ setup/teardown refactoring
24 October 2013 “Unlimited” Event Channels 12 / 13
Questions?
I Design Documenthttp://xenbits.xenproject.org/people/dvrabel/event-channels-F.pdf
24 October 2013 “Unlimited” Event Channels 13 / 13