Upload
arich
View
28
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Brief Overview of Networking Evaluation Methods and Tools. Outline. Tools Simulation ns2 Cluster based network emulation Emulab Live Distributed Testbed PlanetLab Questions What is the right tool(s) for my research? What does it take to get up and running?. Outline. - PowerPoint PPT Presentation
Citation preview
winter 2008 Evaluation Tools 1
Brief Overview ofNetworking Evaluation
Methods and Tools
winter 2008 Evaluation Tools 2
Outline• Tools
– Simulation• ns2
– Cluster based network emulation• Emulab
– Live Distributed Testbed • PlanetLab
• Questions– What is the right tool(s) for my research?– What does it take to get up and running?
winter 2008 Evaluation Tools 3
Outline• For each tool/testbed
– How does it work?– A “hello world” in ___– What kind of research is it good for?
• Tradeoffs between tools
winter 2008 Evaluation Tools 4
Simulation(ns2)
winter 2008 Evaluation Tools 5
What is ns?• Network simulator• a discrete event simulator• focused on modeling network protocols
– wired, wireless, satellite– TCP, UDP, multicast, unicast– Web, telnet, ftp– Ad hoc routing; sensor networks– Infrastructure: stats, tracing, error models etc.
winter 2008 Evaluation Tools 6
ns --- what is it good for?
• Evaluate performance of existing network protocols.• Prototyping and evaluation of new protocols.• Large-scale simulations not possible in real experiments.
Used to:
winter 2008 Evaluation Tools 7
ns
• Event-driven simulator– Model world as events– Simulator has list of events– Process: take next one, run it, until done– Each event happens in instant of virtual time, but takes arbitrary real time
• Single thread of control• Packet level
How does it work:
winter 2008 Evaluation Tools 8
Ns models
• Traffic/applications– CBR, FTP, telnet, web
• Routing/Queuing– Drop-tail, FQ, SFQ, RED, DRR– Wired routing, adhoc routing etc
• Transport– TCP (variants), UDP, multicast (SRM)
winter 2008 Evaluation Tools 9
ns - software structure• Object oriented (C++, OTcl) – code reuse• Scalability + Extensibility
– Control/”data” separation– Split C++/OTcl object
• C++ for packet-processing (fast to run)• OTcl for control - (fast to write)
– Simulation setup and configuration
winter 2008 Evaluation Tools 10
otcl and C++: The Duality
OTcl C++
Pure OTclobjects
Pure C++objects
C++/OTcl split objects
ns
Your ns-script
winter 2008 Evaluation Tools 11
Outline• Overview• Tcl, OTcl basics• ns basicsns basics• Extending ns• ns internals
winter 2008 Evaluation Tools 12
Basic structure of ns-scripts• Creating the event scheduler• [Tracing]• Creating network topology• Creating Transport Layer - Agents• Creating Applications - Applications• Events!
winter 2008 Evaluation Tools 13
Creating Event Scheduler• Create scheduler
– set ns [new Simulator]
• Schedule event– $ns at <time> <event>– <event>: any legitimate ns/tcl commands
• Start scheduler– $ns run
winter 2008 Evaluation Tools 14
“Hello World” in nssimple.tcl
set ns [new Simulator]$ns at 1 “puts \“Hello World!\””$ns at 1.5 “exit”$ns run
bovik@gs19% ns simple.tclHello World!bovik@gs19%
winter 2008 Evaluation Tools 15
Creating Network• Nodes
– set n0 [$ns node]– set n1 [$ns node]
• Links & Queuing– $ns duplex-link $n0 $n1 <bandwidth> <delay>
<queue_type>– Queue type: DropTail, RED, CBQ, FQ, SFQ, DRR
winter 2008 Evaluation Tools 16
Routing + traffic• Unicast
– $ns rtproto <type> – <type>: Static, Session, DV
• Multicast support also.• Traffic
– Simple two layers: transport and application.– Transport: TCP, UDP etc.– Applications: web, ftp, telnet etc.
winter 2008 Evaluation Tools 17
The transport layer: UDP• UDP
– set udp [new Agent/UDP]– set null [new Agent/NULL]
– $ns attach-agent $n0 $udp– $ns attach-agent $n1 $null
– $ns connect $udp $null
winter 2008 Evaluation Tools 18
The transport layer: TCP• TCP
– set tcp [new Agent/TCP]– set tcpsink [new Agent/TCPSink]
– $ns attach-agent $n0 $tcp– $ns attach-agent $n1 $tcpsink
– $ns connect $tcp $tcpsink
winter 2008 Evaluation Tools 19
Creating Traffic: On Top of TCPFTP
– set ftp [new Application/FTP]– $ftp attach-agent $tcp– $ns at <time> “$ftp start”
Telnet– set telnet [new Application/Telnet]– $telnet attach-agent $tcp
winter 2008 Evaluation Tools 20
Creating Traffic: On Top of UDP
• CBR– set src [new Application/Traffic/CBR]
• Exponential or Pareto on-off– set src [new Application/Traffic/Exponential]– set src [new Application/Traffic/Pareto]
• Trace driven traffic– Inter-packet time and packet-size
winter 2008 Evaluation Tools 21
Attaching a traffic source
• set cbr [new Application/Traffic/CBR]• $cbr attach-agent $udp
• $ns at <time> “$cbr start”
winter 2008 Evaluation Tools 22
TracingTrace packets on all links:
– set f[open out.tr w]– $ns trace-all $f
– $ns flush-trace– close $f
<event><time><from><to><type><size>--<flags>--<flow id><src><dst><seqno> <pckt id>+ 1 0 2 cbr 210 ------- 0 0.0 3.1 0 0- 1 0 2 cbr 210 ------- 0 0.0 3.1 0 0r 1.00234 0 2 cbr 210 ------- 0 0.0 3.1 0 0
Is tracing all links always the best thing to do?
winter 2008 Evaluation Tools 23
More Tracing• Tracing specific links
– $ns trace-queue $n0 $n1 $f
• Tracing variables – set cwnd_chan_ [open all.cwnd w]– $tcp trace cwnd_– $tcp attach $cwnd_chan_
winter 2008 Evaluation Tools 24
Controlling object parameters
• Almost all ns objects have parameters– ex. Application/Traffic/Exponential has rate and
packetSize– set parameters in OTcl
• set etraf [new Application/Traffic/Exponential]• $etraf set rate_ 1Mb• $etraf set packetSize_ 1024
winter 2008 Evaluation Tools 25
Putting it all togetherset ns [new Simulator]set n0 [$ns node]set n1 [$ns node]$ns duplex-link $n0 $n1 1.5Mb 10ms DropTail Creating Topology
set tcp [$ns create-connection TCP $n0 TCPSink $n1 0] Creating Transport layer
set ftp [new Application/FTP]$ftp attach-agent $tcp Creating Applications
$ns trace-queue $n0 $n1 $f
Schedule Events$ns at 0.2 "$ftp start“
$ns at 1.2 ”exit“
$ns run
n0 n11.5Mb 10msFTP/TCP
winter 2008 Evaluation Tools 26
Example: XCP vs CSFQ
• Compare utilization of links
• Compare throughput of flows
winter 2008 Evaluation Tools 27
nam – the network animator
n0 n1
set nf [open out.nam w]$ns namtrace-all $nf
exec nam out.nam &…
winter 2008 Evaluation Tools 28
ns “components”• ns,ns, the simulator itself• namnam, the Network AniMator
– Visualize ns output– GUI input simple ns scenarios
• Pre-processing:– Traffic and topology generators
• Post-processing:– Simple trace analysis, often in Awk, Perl, or Tcl
winter 2008 Evaluation Tools 29
Network Dynamics: Link failures
• $ns rtmodel-at <time> <up|down> $n0 $n1
• $ns rtmodel Trace <config_file> $n0 $n1
• $ns rtmodel <model> <params> $n0 $n1 <model>: Deterministic, Exponential
winter 2008 Evaluation Tools 30
Issues in Simulations• Suppose you want to study the way TCP
sources share a bottleneck link…Agent/FTP
Agent/FTPAgent/FTPAgent/FTP
Which topology?
When to start sources?
What else affects results?
Which traffic sources? Background Traffic?
winter 2008 Evaluation Tools 31
Cluster Based Emulation(Emulab)
368 nodes3 big Ciscos
Slides based on SOSP poster by Jay Lepreau et. al. University of Utah www.emulab.net
winter 2008 Evaluation Tools 32
Why?• “We evaluated our system on five nodes.”
-job talk from university with 300-node cluster• “We evaluated our Web proxy design with 10 clients on 100Mbit
ethernet.”• “Simulation results indicate ...”• “Memory and CPU demands on the individual nodes were not
measured, but we believe will be modest.”• “The authors ignore interrupt handling overhead in their evaluation,
which likely dominates all other costs.”• “You have to know the right people to use the cluster.”• “The cluster is hard to use.”• “<Experimental network X> runs FreeBSD 2.2.x.”• “February’s schedule for <Testbed Y> is…”• “<Network Z> is tunneled through the Internet”
winter 2008 Evaluation Tools 33
What?• An instrument for experimental CS research• A completely configurable “Internet emulator” in a
room– At its core, it’s bare hardware, with…– … complete remote access and control
• But, also simple to use– Lots of fast tools for common case
• Automatic topology, node and link configuration• Automatic traffic generation
– Universally available• Universities, research labs, companies
– Zero-penalty for remote research
winter 2008 Evaluation Tools 34
Key Design Aspects• Allow experimenter complete control
– Configurable network properties• link bandwidth, Latency, and loss rates via transparently
interposed “traffic shaping” nodes that provide WAN emulation
– Configurable OS image• Linux, FreeBSD, Windows
• Virtualization– of all experimenter-visible resources– node names, network interface names, network addrs
• e.g., node-0.esmexp1.esm.emulab.net
winter 2008 Evaluation Tools 35
Emulab Architecture
Programmable “Patch Panel”
PCPC
Web/DB/SNMPSwitch MgmtUsers
Internet
Control Switch/Router
Serial
Sharks Sharks
160168
PowerCntl
winter 2008 Evaluation Tools 36
Experiment Creation Process
winter 2008 Evaluation Tools 37
Using Emulab• Submit ns script via web form
– Specify number of nodes– Specify link properties
• Relax while emulab …– Generates config from script & stores in DB– Maps specified virtual topology to physical nodes– Provides user accounts for node access– Assigns IP addresses and host names– Configures VLANs– Loads disks, reboots nodes, configures OSs
• Two ways to run experiments– Interactive
• Works only if there are enough free nodes right now– Batch
• Runs experiment when enough free nodes available
winter 2008 Evaluation Tools 38
Using Emulab• Time sharing model for nodes
– If you check out 50 nodes, they are exclusively yours until you release them (or forced to release them!)
winter 2008 Evaluation Tools 39
Using Emulab
set ns [new Simulator]source tb_compat.tcl
##setup the core nodesfor {set i 0} {$i <= 4} {incr i} { set node($i) [$ns node] tb-set-node-os $node($i) FC4-STD}## setup the core linksfor {set i 0} {$i < 4} {incr i} { set j [expr $i + 1] $ns duplex-link $node($i) $node($j) 100Mb 0ms DropTail}$ns duplex-link $node(4) $node(0) 100Mb 0ms DropTail
## setup the edge nodes and linksfor {set i 5} {$i <= 10} {incr i} { set node($i) [$ns node] tb-set-node-os $node($i) FC4-STD set j [expr $i % 5] set link($i) [$ns duplex-link $node($i) $node($j) 1000Kb 0ms
DropTail] tb-set-link-simplex-params $link($i) $node($i) 0ms 500Kb 0.0}
# Go!$ns rtproto Static$ns run
D
D
D
D
D D
winter 2008 Evaluation Tools 40
What is Emulab Good For?(creators words)
• Simulation– Fast prototyping, easy to use, but less realistic
• Small static testbeds– Real hardware and software, but hard to configure and
maintain, and lacks scale• Live networks
– Realistic, but hard to control, measure, or reproduce results
Emulab complements and also helps validate these environments
winter 2008 Evaluation Tools 41
What is Emulab Good For?• Studying routing protocols
– Difficult to create richly connected nodes– Experiments that need access to the network core
• Studying peer-to-peer protocols– But, lacks scale
• Scaling experiments with “raw” machines with no link emulation
• Good controlled environment for testing before going to PlanetLab
winter 2008 Evaluation Tools 42
Live Distributed Testbed(PlanetLab)
644 nodes at 304 siteswww.planet-lab.org
winter 2008 Evaluation Tools 43
Resource Sharing• Statistical sharing of resources
– Each application has a “slice” of the overlay resources
• Bandwidth, CPU, memory– No guarantees on bandwidth, CPU or memory
fairness– Slices guaranteed 1Mbps of bandwidth, rest shared
by all slices
winter 2008 Evaluation Tools 44
Resource SharingEmulab PlanetLab
winter 2008 Evaluation Tools 45
Slices
winter 2008 Evaluation Tools 46
Slices
winter 2008 Evaluation Tools 47
Slices
winter 2008 Evaluation Tools 48
Per-Node View
Virtual Machine Monitor (VMM)
NodeMgr
LocalAdmin VM1 VM2 VMn…
winter 2008 Evaluation Tools 49
PlanetLab Virtualization: VServers
• Kernel patch to mainstream OS (Linux)• Gives appearance of separate kernel for
each virtual machine– Root privileges restricted to activities that do not
affect other vservers• Some modification: resource control
(e.g., File handles, port numbers) and protection facilities added
winter 2008 Evaluation Tools 50
How to Use PlanetLab• Get a slice
– Request local PI– Setup SSH keys (no passwords on PlanetLab)
• Add nodes to your slice through the web interface
• Write (or borrow) scripts to deploy and run your system on PlanetLab and gather data from PlanetLab– Pssh/pscp (parallel versions of ssh/scp)
• Remember, you’re running on a live network!– Test your code before you deploy
winter 2008 Evaluation Tools 51
Not Just a Testing Infrastructure
• PlanetLab started by Intel …
“…to support seamless migration of an application from an early prototype,through multiple design iterations,
to a popular service that continues to evolve.”
winter 2008 Evaluation Tools 52
Testbed Comparison•ns2 •Emulab •PlanetLab
•Event-driven simulated network•Scale of 100s depending of configuration•Simulated topology•Controlled environment
•Emulated network•Scale of 10s most of the time•Real controllable topology•Controlled environment
•Real overlay network•Scale of 100s (goal is over 1000)•Real fixed topology•Live network, uncontrolled environment