RINSE: the Real-time Immersive Network Simulation Environment for NetworkSecurity Exercises (Extended Version)

Michael Liljenstam1, Jason Liu2, David Nicol1, Yougu Yuan1, Guanhua Yan1, Chris Grier1

1University of Illinois at Urbana-ChampaignCoordinated Science Laborabory

1308 W. Main St., Urbana, IL 61801{mili,nicol,yuanyg,ghyan,grier}@crhc.uiuc.edu

2Colorado School of MinesMathematical and Computer Sciences

Golden, CO 80401-1887xliu@mines.edu

Abstract

The RINSE simulator is being developed to supportlarge-scale network security preparedness and training ex-ercises, involving hundreds of players and a modeled net-work composed of hundreds of LANs. The simulator mustbe able to present a realistic rendering of network behavioras attacks are launched and players diagnose events and trycounter measures to keep network services operating. Wedescribe the architecture and function of RINSE and out-line how techniques like multiresolution traffic modeling,multiresolution attack models, and new routing simulationmethods are used to address the scalability challenges ofthis application. We also describe in more detail new workon CPU/memory models necessary for the exercise scenar-ios and a latency absorption technique that will help whenextending the range of client tools usable by the players.

1. Introduction

The climate on the Internet is growing increasingly hos-tile while organizations are increasingly relying on the In-ternet for at least some aspects of day-to-day operations.They are thus being forced to plan and prepare for networkfailures or outright attacks—how it might affect them andwhat actions to take. With current system complexity, toolsto assist in preparedness evaluation and training are likelyto become more and more important.The October 2003 Livewire cyber war exercise [2] con-

ducted by the Department of Homeland Security, is oneparticular instance of preparedness evaluation and trainingthat involved companies across industrial sectors as well asgovernment agencies. More exercises of this type are cur-rently being planned, and based on experiences from thefirst event, there was a desire for improved tools to auto-matically determine the impact on the network from attacksand defensive actions and the extent to which the network

is capable of delivering the services needed. Providing net-work simulation tool support for exercises such as Livewireis particularly challenging because of their scale. Future ex-ercises are expected to involve as many as a couple of hun-dred participating organizations, and will thus involve many“players” and a network of significant size.We are currently developing the Real-time Immer-

sive Network Simulation Environment for networksecurity exercises (RINSE) to meet this need and ad-dress the challenges inherent in this type of applica-tion. Hence, the goal for RINSE is to manage large-scalereal-time human/machine-in-the-loop network simula-tion with a focus on security for exercises and train-ing. It needs to be extensible so that it can evolve overtime, and it needs to be designed with an eye towards secu-rity and resilience to hardware faults since these exercisesinvolve many people and last for several days.The spectrum of approaches to general large-scale net-

work modeling being explored in the literature range fromhardware emulation testbeds like Emulab [38], network em-ulators like ModelNet [37], to network simulators like IP-TNE [34], GTNetS [9], PDNS [9], and MAYA [40]. Hard-ware emulation excels in application code realism (runningthe real thing), while simulations tend to be more flexibleand have an advantage in terms of scalability. However, themiddle ground is increasingly being explored; for instance,through increasing emulation support in simulators. For se-curity exercises we like the flexiblity and scalability of sim-ulation, and the safety of unleashing attacks in a simulatedenvironment rather than on a real network.Several simulators offer similar capabilities, including

parallel execution, real-time/emulation support, and dis-crete event/analytic models. However, we believe RINSEis unique in the way it brings together human/machine-in-the-loop real-time simulation support with multiresolutionnetwork traffic models, attack models that leverage the ef-ficiency of the multiresolution traffic representations, novel

Data Server SimulatorDatabaseManager

iSSFNet Network Simulator

database

Network ViewerClients

Internet

RINSE backup instance

Figure 1: RINSE architecture

models of host/router resources such as CPU and memory,and novel routing simulation techniques. In this position pa-per we provide an overview of RINSE to show how thesetechniques are being brought to bear on the problem at hand.We also detail some specific new contributions: i) a tech-nique for absorbing outside network latency into the simu-lation model and ii)models for including CPU and memoryeffects into network simulations.The remainder of this paper is organized as follows. Sec-

tion 2 describes the architecture of RINSE and outlines asimple example scenario that introduces the salient featuresof RINSE, described further in Sections 3 to 7. Finally, Sec-tion 8 summarizes and outlines future work.

2. RINSE Architecture

RINSE consists of five components: the iSSFNet net-work simulator, the Simulator Database Manager, adatabase, the Data Server, and client-side Network View-ers, as shown in Figure 1. The iSSFNet network simula-tor, formerly called DaSSFNet, is the latest incarnation ofthe C++ network simulator based on the Scalable Sim-ulation Framework (SSF), an Application ProgrammingInterface (API) for parallel simulations of large-scale net-works [5]. iSSFNet runs on top of the iSSF simulationkernel, which handles synchronization and support func-tions. iSSF uses a composite synchronous/asynchronousconservative synchronization mechanism for paral-lel and distributed execution support [23], and has re-cently been augmented to include real-time interactionand network emulation support. iSSFNet runs on paral-lel machines to support real-time simulation of large-scalenetworks.Each simulation node connects independently to

the Simulator Database Manager, which delivers datafrom the simulator to the database and delivers con-trol input from the database to the simulator. On theuser/player side, the Data Server interfaces with client ap-plications, such as the Java-based application “Network

Viewer”, which allows the user to monitor and control thesimulated network. In the future, we plan to evolve the ar-chitecture towards using the emulation capabilities tosupport direct SNMP interaction with the simulated net-work devices, thus having regular networking utilities andnetwork management tools as clients. In the current de-sign, the Data Server performs authentication for eachuser, distributes definitions of the client’s view of the net-work (using the Domain Modeling Language), and pro-vides a simple way for the client applications to accessnew data in the database through XML-based remote pro-cedure calls. The Network Viewer clients, a screen shotshown in Figure 2, provide the users with their local viewof the network (usually only their organization’s net-work) and periodically poll the Data Server for data. TheData Server responds with new data for each client, ex-tracted from the database. The game managers, function-ing as superusers of an entire exercise, also use NetworkViewer clients, but can have a more global view of the net-work.The Network Viewer clients have a simple command

prompt where the user can issue commands to influence themodel. User commands are sent in the opposite direction ofthe output data path and injected into the simulator. We cur-rently divide the commands into five categories:

Attack– the game managers can launch attacks against net-works or specific servers. RINSE focuses on Denial-of-Service effects on networks and devices, so attacksinclude DDoS and worms.

Defense– attacks can be blocked or mitigated, for instanceby installing packet filters.

Diagnostic Networking Tools– functionality simi-lar to some commonly used networking utilities,such as ping, are supported for the player to diag-nose the network.

Device Control– individual devices, such as hosts androuters, can be shutdown or rebooted.

Simulator Data– commands can be issued to the simula-tor to control the output, turn on or off trace flow froma particular host, etc.

Depending on the type of a command, it may be address-ing the whole simulator, a particular host or router, a par-ticular interface, or a particular protocol or application ona host or router. A command handling infrastructure in thesimulator passes the commands from the clients to the ap-propriate components of the simulation model.Next, we illustrate the salient features of RINSE and dis-

cuss user commands in more detail through a simple exam-ple scenario. We point to descriptions of important aspectsof the simulator as we go through the example.

Figure 2: Network Viewer client screen shot

2.1. Example Scenario

Consider a simple scenario where a player is responsiblefor a subnetwork, partially shown in Figure 2, containingamong other things a server. Multiple clients are requestinginformation from the server through some form of trans-actions. By transaction we simply mean a request-responseexchange between the client and the server. Sections 4 and 6outline RINSE’s models for efficient representation of traf-fic flows and route computation.The player can select data to monitor, such as the transac-

tion request and responses at the server and switches on theflow of output data from the simulator by issuing the com-mand:report server on

here server is simply a symbolic name for the server ad-

dress.A game manager attempts to disrupt the operations of

the server by launching a DDoS attack against open ser-vices on the server, and the player responsible for the net-work will need to diagnose what is going on and try to takeremedial actions. The game manager launches the attack byissuing an attack command at the Network Viewer client:

ddos attack attacker server 100 2000

Both attacker and server are symbolic names for theattacker’s host and the targeted host, respectively. Upon re-ceiving the command (via the command handling infras-tructure), the attacker’s host uses a simulated intermedi-ate communication channel (e.g., Internet Relay Chat) to

send attack signals to zombie hosts—hosts under the at-tacker’s control. These zombie hosts then initiate the denial-of-service attack against the targeted victim. The attack is tolast for 100 seconds and each zombie emits traffic at a rateof 2000 kbits/s. RINSE attack models leverage efficienciesin its high volume traffic representations, as is further de-scribed in Section 5.We will assume here that the DDoS traffic simply loads

the open service daemons and thus induces a large CPU loadon the server. This load disrupts the processing of legiti-mate transactions. As shown in the screen shot in Figure 2,the player managing the server can monitor the CPU uti-lization on the server and observe an abnormally high load.Models of host and router resources like CPU and memoryare described in more detail in Section 7. After determiningthat the load likely stems from abnormal traffic, the playerattempts to block traffic on a certain port that has been in-advertently left open by issuing the command:filter server add 0 deny all all * all * 23

to install a filter on the server to deny packets coming in onall interfaces, using all protocols, from all source IP ad-dresses (“*”), and all source ports, to all destination IPaddresses (“*”) and destination port 23. Successful filter-ing blocks packets from reaching the open service daemonsand thus alleviates the load on the server at the expense ofsome processing cost for filtering.We now proceed to describe aspects of the system men-

tioned here in more detail, starting with the real-time simu-lation support.

3. Real-time Simulation Support

In addition to supporting the RINSE Network Viewerclient, we are currently developing support for the Sim-ple Network Management Protocol (SNMP) to allow us tomonitor and control the simulated network devices throughindustry-standard network management tools. For that rea-son, our real-time simulation support must be simple, flex-ible, and be able to accommodate real-time interactions atvarying degrees of intensity, including both human-in-the-loop and machine-in-the-loop simulations. In this section,we describe the real-time support both in the iSSF parallelsimulation kernel and in the network simulator supportedby iSSF.

3.1. Kernel Support

Over the years, we have seen many network emulators,ranging from single-link traffic modulators to full-scale net-work emulation tools, e.g. [32, 37]. Most network emulatorsare time-driven. For example, ModelNet [37] stores packetsin “pipes” sorted by the earliest deadline. A scheduler ex-ecutes periodically (once every 100 µseconds) to emulate

packet moving through the pipes. There are two main draw-backs associated with the time-driven approach: i) the ac-curacy of the emulation depends on the time granularity ofthe scheduler, which largely depends on the target machineor the target operating system, and ii) there has not been agood model used by network emulators to accurately char-acterize the background traffic and its impact on the fore-ground transactions (i.e., traffic connecting real-time appli-cations). Simulation-based emulation (also referred to asreal-time network simulation), on the other hand, providesa common framework for real application traffic to inter-act with simulated traffic, and therefore allows us to studyboth network and application behaviors with more realism.Examples of existing real-time network simulators includeNSE [8], IP-TNE [34], MaSSF [21], and Maya [40]. IP-TNE is the first simulator we know that adopts parallel sim-ulation techniques for emulating large-scale networks. Thereal-time support in iSSF inherits many features of theseprevious simulation-based emulators. Our approach, how-ever, is unique in several ways, which we elaborate next.

Extending SSF API. The real-time support is designed asan extension to the SSF API, thus making an easy transi-tion for other SSF models that require real-time support. InSSF, an inChannel (or outChannel) object is definedas a communication port in an entity to receive (send) mes-sages from (to) other entities. We extended the concept ofthe in-channel using it as the conduit for the simulator toreceive events from outside the simulator (e.g., acceptinguser commands arrived at a TCP socket). We extended theAPI so that a newly created in-channel object can be associ-ated with a reader thread. The reader thread converts (exter-nal) physical events into (internal) virtual events and injectsthem into the simulator using the putVirtualEventmethod. A virtual event is created to represent the corre-sponding physical event and is assigned with a simulationtimestamp calculated as a function of i) the wall-clock timeat which the event is inserted into the simulator’s event-list, and ii) the current emulation throttling speed (whichwe will elaborate momentarily). The SSF entities receiveevents from the in-channel objects as before, regardless ofwhether they represent special devices that accept exter-nal events. From a modeling perspective, there is no dis-tinction between processing a simulation event and a real-time event. Similarly, we also extended the concept of theout-channel using it as a device to export events (for ex-ample, reporting the network state to a client applicationover a TCP connection). In this case, a writer thread canbe associated with the special outChannel object. Thewriter thread invokes the getRealEvent method to re-trieve events designated for the external device and con-verts the virtual events into physical events. Each of theseevents is assigned with a real-time deadline indicating thewall-clock time at which the event is supposed to happen.

The real-time deadline is calculated from the virtual timeand again the emulation throttling speed. The writer threadis responsible for delivering the event upon the deadline.

Throttling Emulation Speed. The system can dynamicallythrottle the emulation speed (either by accelerating or decel-erating the simulation execution with respect to real time).This feature is important for supporting fault tolerance. Forexample, if a simulator fails over a hardware problem, afterfixing the problem, the simulator should be able to restartfrom a previously checkpointed state and quickly catch upwith rest of the system. We can accelerate the emulationspeed and use the same user input logged at the databaseserver to restore the state. In order to regulate the timeadvancement, we modified the startAll method in theEntity class (which is used to start the simulation in SSF),adding an optional argument to allow the user to specify theemulation speed as the ratio between virtual time and wall-clock time—for example, a ratio of one means simulationin real-time, “infinity” means simulation as fast as possible,and zero means halting the simulation. An Entity classmethod throttle is also added to make it possible to dy-namically change the ratio during the simulation.

Prioritizing Emulation Events: We use a priority-basedscheduling algorithm in the parallel simulation kernel tobetter service events with real-time constraints. In SSF, theuser can cluster entities together as timelines, i.e., logicalprocesses, that maintain their own event-lists. Events on thetimelines are scheduled according to a conservative syn-chronization protocol [23]. In a “pure” simulation scenario,where the simulation is set to run as fast as possible, a time-line can be scheduled to run as long as it has events readyto be processed safely without causing causality problems.For that reason, during the event processing session, the ker-nel executes all safe events of a timeline uninterrupted toreduce the context switching cost. When we enable emu-lation, however, the timelines that contain real-time eventsmust be scheduled promptly.To promptly process the events with real-time deadlines

in the system, we adopted a greedy algorithm in iSSF as-signing a high priority to emulation timelines. These time-lines contain real-time objects—special in-channels andout-channels that are used for connecting to the physicalworld. Whenever a real-time event is posted and ready to bescheduled for execution on these emulation timelines, thesystem interrupts the normal event processing session of anon-emulation timeline and makes a quick context switch toload and process the real-time events in the emulation time-line. This priority-based scheduling policy allows the eventsthat carry real-time deadlines to be processed ahead of regu-lar simulation events. Note that, however, since normal sim-ulation events may be on a critical path that affects a real-time event, this method is not an optimal solution. We are

currently investigating other more efficient scheduling algo-rithms that can promptly process emulation events as wellas events on the critical path, so that the real-time require-ment can be satisfied in a resource-constrained situation.

3.2. Latency Absorption

We realize that the real-time demand not only puts a tightconstraint on how we process events to reduce the chanceof missed deadlines, but also on the connectivity betweenthe simulator and the real applications. For example, con-sider a scenario in which a path is established between aclient machine running the ping application and the ma-chine running the network simulator, as shown in Figure 3.The client machine, which assumes the role of a host inthe simulated network (with a virtual IP address 10.5.0.12),pings another host at 10.0.1.19. The ping application at theclient machine generates a sequence of ICMP ECHO pack-ets targeting 10.0.1.19. These packets are immediately cap-tured by a kernel packet filtering facility [22] and then sentto the machine running the simulator. A reader thread re-ceives these packets, and converts them to the correspond-ing simulation events. The simulator carries out the simu-lation by first putting the ICMP ECHO packets in the out-put queue of the simulated host 10.5.0.12. The packets arethen forwarded over the simulated network to the desig-nated host 10.0.1.19, which responds with ECHO REPLYpackets. Once the packets return to the host 10.5.0.12, thesimulator exports the events to a writer thread, which sendsthem to the client machine running the ping application. Theclient ping application finally receives the ECHO REPLYpackets and prints out the result. Note that the segment ofthe path between the client application and the simulatedhost does not exist in the model. The problem is that the la-tencies of the physical connection can contribute a signifi-cant portion of the total round-trip delay. Simply on the for-warding path (from the client to the simulator), it may takehundreds of microseconds even on a high-speed local areanetwork, before the emulation packet is eventually insertedinto the simulator’s event-list.1 It can tremendously affectapplications that are sensitive to such latencies.Our solution to this problem is to hide the latencies due

to the physical connection inside the simulated network.Since delays are imposed upon network packets transmit-ted from one router to another in simulation, we can mod-ify the link layer model to absorb the latencies by send-ing the packet ahead of its due time. The simulator mod-els the link-layer delay of a packet in two parts: the queu-

1 The delay includes the time for the sender’s operating system to cap-ture and send the packet, the transmission time of the packet, the timefor the reader thread to receive the packet, and the time for the simu-lator to finally accept the event and insert it into the appropriate event-list.

% ping 10.0.1.19PING 10.0.1.19: 56 data bytes

64 bytes from 10.5.0.12: icmp_seq=0

ttl=64 time=0.54 ms

64 bytes from 10.5.0.12: icmp_seq=1

ttl=64 time=0.28 ms…

10.5.0.12

10.0.1.19

physical connections

virtual connections

Simulated

Network

Reader

Thread

Writer

Thread

Figure 3: Emulation of a Ping Application

ing time—the time to send all packets that are ahead of thepacket in question, and the transmission time—the time forthe packet to occupy the link before it can be successfullydelivered, which we model as the sum of the link latencyand the transmission delay—the latter is calculated by di-viding the packet size by the link’s transmission rate. As-suming that packets are sent in first-in-first-out (FIFO) or-der, the time required to transmit a packet is known as soonas the packet enters the queue at the link layer. Note that, ifthe FIFO ordering is not observed (e.g., packets are priori-tized according to their types), one cannot predict the packetqueuing time precisely. Furthermore, if we need to provide amore detailed model on lower protocol layers, the link statelayer may play a significant role in determining the packettransmission time as well. In either case, we can still use alower bound of the packet delays in our scheme. In the dis-cussions to follow, we assume the delays are precise for bet-ter exposition.We use a list to store the packets in the queue together

with their precalculated transmission times. Let Tnow be thecurrent simulation time and P0 be the last packet transmit-ted over the link. T0 is the simulation time that P0 startstransmission (T0 ! Tnow). Let Pi be the ith packet inthe queue, where 0 < i ! N and N is the total num-ber of packets currently in the queue. The time to transmitpacket Pi is therefore Ti = T0 +

!i!1j=0(! + "j), where

! is the link latency and "j is the transmission delay ofpacket Pj . Suppose that an ICMP ECHO packet is cre-ated externally at wall-clock time tR, and the correspond-ing simulation packet Pd is injected into the simulator attime t"R. As a result, the packet carries a virtual time deficitof #d = (t"R " tR)/R, where R is the proportionality con-stant that indicates the emulation speed (i.e., the ratio of vir-tual time to real time). Rather than appending the packet tothe end of the queue, we insert the packet right before packet

Pk, where k = max{i|i # 0 and #d <!N

j=i(!+"j)}.2 Af-ter inserting the packet in the queue, we reduce deficit of thepacket by the total transmission times of all packets behindthe packet in the queue:

!Nj=k(! + "j). Further improve-

ment can be made to transmit the emulation packets evenearlier. When a packet with a deficit becomes the head ofthe queue, we can simulate the packet transmission in zerosimulation time. That is, we can further reduce the deficitby the packet’s transmission time. Note that in iSSF the de-lay of the link that connects hosts belonging to two separatetimelines is used to calculate the lookahead for the conser-vative parallel synchronization protocol. It is required thatthe link latency ! for cross-timeline links must be largerthan zero. In this case, we can only reduce the deficit by asmuch as the expected packet transmission delay.

It is reasonable to insert an event with a time deficitahead of others in the queue. After all, were the physicalconnection latencies not present, the event would have en-tered the queue much earlier. However, in cases where thedeficit is larger than the sum of transmission time of allpackets in the queue (the packet is therefore inserted at thehead of the queue), we can only allow the packet to con-tinue carrying the remainder of the deficit to the next hop,and therefore preempt events at the next hop. The processcontinues until the deficit is reduced to zero, or the packetreaches its destination. Since we do not “unsend” packetsthat have been sent before the emulation packet with thedeficit arrives, this scheme is simply an approximation oncethe deficit is carried to the next hop.

Another issue concerns accommodating the physi-cal connection latencies in the reverse path (from thesimulator to the client application). A simple solu-tion is to assume such latencies in the reverse path tobe the same as in the forwarding path, and use a deficitof the same amount for all packets traveling in that di-rection. The problem with this approach is that the sim-ulated network always tries to make up for the deficitwithin the first few hops, while in fact such a deficit is ex-pected at the last segment of the path from the simulatorto the application client. This means the interactions be-tween the packets with deficits and other packets in sim-ulation do not represent reality. We expect that, since inlarge-network simulations there are much fewer emula-tion packets than simulation packets, the effect of such adistortion may not be significant at all. We plan to quan-tify such effect in our future work.

2 We do this by scanning the list from the packet at the tail of the list.k=0 means that the packet is inserted at the front of the list.

4. Multi-resolution Traffic Models

A key technique employed in iSSFNet to make real-time simulation of large networks feasible is to use multi-resolution representations of traffic. The idea is to adjust thelevel of detail with which a traffic flow is simulated depend-ing on how interested we are in the detailed dynamics ofthe flow. Traffic that is “in focus”, what we call foregroundtraffic is simulated with high fidelity at packet-level detail.Traffic that represents “other things” going on in the net-work, i.e., background traffic, is abstracted using fluid mod-eling, either using fine grained per-flow models, or coarsetime-scale periodic fixed point solutions.Fluid modeling of network traffic is a technique with

some history [14, 29, 3, 20, 19], and is being exploredalso in other network simulators, such as MAYA [40],IP-TN [15], and HDCF-NS/pdns [31]. The models usedin iSSFNet are based on our previous work to developdiscrete-event fluid modeling of TCP and hybrid traffic in-teraction models such that the packet and fluid representa-tions can coexist in the same simulation [26]. Very recentwork has addressed coarser models using fixed point solu-tion techniques [27] to calculate the effects of flows com-peting as they pass through a large network. As we havepreviously shown [24], order of magnitude speedups of net-work simulations are possible through the type of multi-resolution modeling outlined here, which makes it possibleto represent larger networks and more flows in real time.

5. Attack Models

Attack models in RINSE focus on assets at a networkresource level, i.e., things like network bandwidth, con-trol over hosts, or computational or memory resources inhosts. Attack models include DoS attacks, worms, and sim-ilar large-scale attacks typically involving large numbers ofhosts and high intensity traffic flows.

5.1. Distributed Denial-of-Service

The DDoSmodel in RINSE assumes that the attacker hasalready established a network of zombie hosts that he/shecan control through some (semi-)anonymous channel, e.g.Internet Relay Chat (IRC). Thus, to launch the attack the at-tacker sends a signal to an intermediate agent (host) that dis-seminates the signal to the zombies. The zombies then pro-ceed to blast the target with traffic.It is worth noting that we are thus often only interested

in the coarse behavior of the attack traffic (a large volumeof traffic) rather than the detailed traffic dynamics (minorvariability). Consequently, we leverage the multi-resolutiontraffic models to provide coarse fluid models of attack trafficthat is significantly more efficient for simulating large-scale

attacks than all out packet simulation. The following exam-ple illustrates the point.Figure 4 compares total (kernel) event counts of fluid-

and packet-based DDoS attack models for a scenario us-ing the NMS baseline model [1]. The network consists ofa ring of LANs with a single target and a fixed number ofzombies (240) in each LAN. Let Z be the number of zom-bies, R be the traffic rate injected by each zombie, T bethe attack duration, P be the average packet size, and Hbe the number of average hops. Focusing on the zombieattack traffic, in the absence of congestion the total num-ber of packet transmission events for the packet model isNp = Z ·(R·T/P )·H and the number of fluid update eventsis Nmin

f = Z · c · H for the fluid, where c = 2 (flow setupand teardown) for a constant rate flow. In the experimentsT = 50 seconds, P = 1000 bytes, the number of cam-puses and thus Z is varied between 240 and 1440, and Ris varied R $ {1200, 2400, 4800} kbps. Adding networksslightly increases H , but high traffic rates can decrease Hthrough congestion loss, with observedH $ [7.1, 9.6]. Con-gestion loss for fluids lead to flow interactions. In the worstcase each new flow interacts with each previous flow suchthat Nmax

f = %Z/2&(Z + 1) · H · c.In the experiments event counts for fluid modeling is es-

sentially constant for different R (not shown), and ordersof magnitude lower than the packet model. Due to conges-tion interaction Nf lies closer to Nmax

f , and thus the esti-mated ratio Np/Nmax

f ! RT/P (Z + 1)c is in the orderof 102. Events other than packet transmission events con-tribute only marginally. Figure 5 shows how execution times(2.6 GHz CPU running Linux) essentially correspond to theevent counts, i.e., order of magnitude reductions for fluid.Adding other flows to the model can increase the numberof fluid rate adjustments from congestion and thus increasethe number of events necessary for the fluid model. Theseresults simply illustrate the indication for a significant ad-vantage for fluid representations of coarse high-volume at-tack traffic.

5.2. Scanning Worms

RINSE also has Internet scanning worm models [18, 28,17] that allow for abstraction of parts of the topology and/orabstraction of the traffic flows by utilizing the fluid modelsto represent scan flows through the network. By modelingthe scanning traffic as it traverses the network, it capturesinteractions between worm propagation and the infrastruc-ture, such as bandwidth constraints slowing down propaga-tion of the worm.The current scanning worm model is the Estimate-Next-

Infection (ENI) model [39]. In ENI model the networktopology can be abstracted using the concept of a worm-net. A wormnet is a subnet that has susceptible hosts in-

100000

1e+06

1e+07

1e+08

1e+09

1e+10

200 400 600 800 1000 1200 1400 1600

Tota

l Ker

nel E

vent

Cou

nt

Number of Zombies

1200kbps pkt level ddos2400kbps pkt level ddos4800kbps pkt level ddos

fluid level ddos

Figure 4: DDoS event count comparison

0.1

1

10

100

1000

10000

200 400 600 800 1000 1200 1400 1600

Wal

l Clo

ck E

xecu

tion

Tim

e [s

econ

ds]

Number of Zombies

1200kbps pkt level ddos2400kbps pkt level ddos4800kbps pkt level ddos

fluid level ddos

Figure 5: DDoS execution time comparison

side and is represented by a single (gateway) router, as il-lustrated in Figure 6. This router announces an IP prefix andkeeps the states of the worm propagation inside this subnet.The network topology is composed of backbone routers andwormnets. The worm propagation inside each single worm-net is computed individually, and the wormnets affect eachother through scanning traffic they inject into the backbonerouters. Optionally, the internal topology inside the worm-net may be retained, in which case the packet traffic to andfrom the individual hosts get fluidized and defluidized bythe wormnet gateway.The scanning flow rates get updated using discrete time

steps of size !t. The model considers scans inside thewormnet and scans hitting it from the outside separately, es-sentially through separate time steps. The worm model re-lies on the fixed point fluid traffic model to compute scanflows through the backbone network every !t time units,

wormnetwormnet

wormnet

wormnetgateway

backbonenetwork

fluidize

defluidize

Figure 6: Wormnets can abstract large numbers of vulnera-ble hosts or model them in detail using fluidizing and deflu-idizing of traffic, i.e., conversion to packet-level traffic.

while adaptive time steps are used to handle the internalscans. This approach improves precision for preferentialscanning strategies, using localized scanning, while limit-ing the work to update traffic rates through the backbonenetwork. Let $ be the scan rate of one worm instance. Atthe beginning of each time step, starting at t, for a givenwormnet j, two things are computed:

1. New infections that will happen in this wormnet in thiscoming time step.

2. Scans that will be sent out to the other wormnets in thistime step.

The number of scans sent out by this wormnet during thetime step [t, t + !t] is sj(t) = Ij(t) · $ · !t, where Ij(t) isthe number of infected hosts initially at time t. The receivedscanning rate rj is the sum of scan rates from either localsources rjL or external sources rjE , i.e., rj = rjL + rjE.rjE is assumed to be constant during this time step. Assumethat the received scans arrive at this wormnet follow a Pois-son process, then the infections that will happen also followa Poisson process with a rate of !j(t) = rj(t) · Sj(t)/Cj ,where Sj(t) is the number of infected hosts at time t in-side wormnet j, and Cj is the size of the IP space of worm-net j. Thus the time for the next infection can be sampledusing an exponential process with the mean of 1/!j(t),i.e., %t $ Exp(!j(t)). If the sampled time is outside thistime step t + %t > t + !t, then we finish computing theworm propagation for this wormnet for this time step. If %tis within this time step, the status of the wormnet is updatedto consider the scans sent out by this newly infected hostfrom its infection time. Then the time is advanced to thesampled infection time, the scans from local sources rjL isupdated as rjL(t + %t) = rjL(t) + %r, and the scans sent

0

50000

100000

150000

200000

250000

300000

350000

400000

0 20000 40000 60000 80000 100000 120000

Infe

ctio

ns

Time [seconds]

Worm Propagation in Code Red v2 Scenario

ENI modelCode Red v2 data trace

Figure 7: Worm ENI model output with parameters set forthe Code Red v2 attack in July, 2001 with collected dataduring the attack. Received scans at the Chemical AbstractService were used to estimate the actual number of infectedhosts in the Internet.

out by this wormnet is updated as sj = sj + %s. After theupdate, we repeat the above steps through an iterative pro-cess. Note that the computing of %t and %s needs to takeinto account whether the worms have a preference for lo-cal address when choosing scanning addresses, the detailsof which are omitted here.Figure 7 shows results for a validation experiment us-

ing Code Red v2 parameters, comparing infections in themodel with collected data during the actual attack in July2001. The data trace was been collected in a /16 network atthe Chemical Abstract Service (http://www.cas.org). Usingthe unique source IP addresses from incoming scan packetsto the network, the actual number of infected hosts in the In-ternet was estimated using the method described in [41]. Inthe experiment we used dataset from the Rocketfuel projectat the University of Washington [35] to generate a backbonenetwork topology and attached 244 wormnets to it. Fromthe processed real-world data trace, we assumed that the to-tal number of susceptibles is 374,500. There is no prefer-ential scanning involved in Code Red v2 incident, the scan-ning rate is set to 5 scans/sec, and the experiment was rundeterministically. In this experiment we are not attemptingto capture host repair and patching, and the growth phase iscaptured very well by the model. 3

6. Routing

Memory and computational demands for routing of traf-fic have been identified as significant obstacles for large-scale network simulation and emulation, and has been ad-

3 Note that the network bandwidth was not a limiting factor in the prop-agation of this particular worm.

dressed in several studies [30, 12, 11, 4, 7]. A naive repre-sentation of routing information requiresO(n) in each nodefor n nodes, for a total of O(n2) storage. Hierarchical ad-dressing in the Internet improves upon this to O(p) in eachnode, where p is the number of IP prefixes, each represent-ing a network. p varies from 1–2 in end hosts (one defaultroute) to more than 130,000 in core Internet routers. Thus,for large-scale network models, the amount of memoryneeded to store all the routing information will still quicklybecome unwieldy. Some studies [30, 12, 4] start from thepremise of shortest path routes and try to reduce compu-tational and representational complexity through spanningtree approximations [12, 4] or lazy evaluation [30]. Othershave achieved memory reductions in detailed protocol mod-els, such as BGP (policy based routing) through implemen-tation improvements [11, 7].In iSSFNet we have developed a method for on-demand

(lazy) computation of policy based routes, as computed byBGP [16]. For efficiency reasons and to ensure that traffic(attack traffic in particular) can address and reach a desti-nation network even if the destination is missing, we needhierarchical addressing. Hence, our routing model is cur-rently being extended to handle route aggregation. We arethus able to preload partial (precomputed) forwarding ta-bles based on a priori known traffic patterns in the model,such as scripted background traffic, and compute routes forother flows as needed.

7. Modeling Device Resources

Based on experiences with earlier exercises of the typeRINSE is targeting, it became apparent that the networkmodel will need to capture not only the effect of limited net-work resources, like bandwidth, but also some aspects ofconstraints on computational resources in hosts and routers.Partly because they may be targeted by Denial-of-Serviceattacks, but also for realism in terms of feasible defenses.For instance, if there is no cost for packet filtering, a de-fender might employ packet filters and let the number of fil-ters go towards infinity without observing any ill effects inthe model. Consequently, we need models of computationalresources (CPU) and memory in RINSE.The problem of modeling processing constraints in net-

work simulations has been given only limited attention todate. Indeed, in most cases a fairly simple model will suf-fice. However, in the case of RINSE, a fair amount of detailis necessary to be able to capture, at least coarsely, inter-actions between different tasks and traffic flows in terms ofprocessing. This results in significant implementation hur-dles, as will be described, and the situation is also compli-cated by the fact that the multi-resolution representation oftraffic necessitates a multi-resolution representation of com-

putational workload (i.e. both discrete and fluid representa-tions coexisting).Examples of network simulators that include mod-

els of computational resources include the follow-ing. Models of the Border Gateway Protocol, such asSSFNet.OS.BGP [10], which has been used to study rout-ing convergence, and BGP++ [6] have been fitted withsimple models of computational delays. The modelsin SSFNet.OS.BGP and BGP++ both use random uni-formly distributed processing delays, while BGP++ also of-fers the choice of measuring the computation delay in theembedded routing code. The simple model for route pro-cessing delays in SSFNet.OS.BGP was thus one of theparameters considered in [10] to study route conver-gence time. In another study, a model of Secure-BGP(derived from SSFNet.OS.BGP) was used to study the im-pact of cryptographic overheads on the performance ofthe protocol [25]. Similarly to the original SSFNet.BGPmodel, costs were associated with each BGP update mes-sage.The sensor networking community, being very conscious

of the constraints imposed by tiny sensors, are particularlyinterested in modeling the power consumption of differentcomponents. Thus, simulators such as SensorSim (an ex-tension to ns-2) include a CPU model that appears primar-ily focused on coarsely modeling the power consumption ofthe CPU [33].However, in the case of RINSE, a fair amount of detail

is necessary to be able to capture, at least coarsely, inter-actions between different tasks and traffic flows in terms ofprocessing. This results in significant implementation hur-dles, as will be described, and the situation is also compli-cated by the fact that the multi-resolution representation oftraffic necessitates a multi-resolution representation of com-putational workload (i.e. hybrid discrete and fluid represen-tations).

7.1. CPU Model in RINSE

In RINSE we want to model CPU and memory re-sources, where the specifics will depend on the scenarioin question (i.e., which resources could potentially be ex-hausted). We identified the following requirements on ourCPU model:

• Interference between different CPU intensive tasks.

• Traffic delay could result from high CPU load–e.g., asa result of reduced server responsiveness.

• Possibility of packet loss due to sustained high load.

• Observable CPU load: the user should be able to mon-itor CPU load to diagnose the system.

task 1

task 2

task N

!1f "t #$!1

p"t #

!2f "t #$!2

p"t #

!Nf "t #$!N

p "t #

%1"t #

FCFS processorsharing

%

&1"t #

Figure 8: Processing work model where work is handledFCFS by tasks that are allocated “cycles” on the CPU

• Light weight: we must strive for the simplest possiblemodels that can at least approximately represent thedesired effects.

Thus, we require more behavior detail than many other ap-plications do to be able to capture, at least coarsely, inter-actions between different tasks and traffic flows in terms ofprocessing. This results in significant implementation hur-dles, as will be described, and the situation is also compli-cated by the fact that the multi-resolution representation oftraffic necessitates a multi-resolution representation of com-putational workload (i.e. hybrid discrete and fluid represen-tations). However, given the complexity of operating sys-tems and hardware layers, we must strive for the simplestpossible models that can at least approximately representthe effects we have identified.Interference: to observe interference between different

tasks, we need to model how processing cycles are allo-cated. The generic UNIX process scheduling mechanism4[36] is based on priority scheduling, where process priori-ties are continuously recomputed to try to achieve good re-sponsiveness and latency hiding for I/O bound tasks.We do not want to get into the details of the schedul-

ing mechanism, but be able to observe competition for re-sources. Within the CPU, a set of tasks are defined, wherea task can be thought of as a process or thread. For in-stance, these could be application layer processes like webclients/servers, a database server, or lower layer functional-ity like a firewall process doing packet filtering on incom-ing packets. Figure 8 illustrates how each task services thework it has to do in FCFS order, but cycles are allocatedamong tasks using processor sharing. In this first model wesimplify the problem by assuming that the tasks we con-sider have roughly the same priority (same range), so thatthey are treated equally. The requests (incoming traffic) toeach task may be a mixture of packets and fluid traffic flows.

4 It varies somewhat between different flavors. Linux has a slightly dif-ferent mechanism, but for the purposes of this discussion it’s essen-tially the same.

As in the hybrid packet/fluid traffic model in [26], we forma hybrid queue by fluidizing the packet load through esti-mating the packet rate. However, the service model inter-leaving the tasks actually make things even more compli-cated here than most hybrid traffic models since service isnot FIFO. Assume there are N tasks. Let !f

i (t) be the in-coming fluid workload rate for task i (in cycles per second)at time t, and and µ is CPU service rate (i.e. its speed). Apacket has an associated workload, wt in cycles. By esti-mating the the packet arrival rate over a time window [t", t],we get the estimated packet workload rate !p

i (t). Let the to-tal arriving workload for task i be !i(t) = !f

i (t) + !pi (t).

We need to allocate a service rate to each task µi(t), deter-mine backlog "i(t) and possibly lost work &i(t). A discreteworkload arrival (packet workload) at t is always added tobacklog on arrival "i(t) ' "i(t) + wt. Note, however, thatif no discrete arrivals preceded it in [t", t], then !p

i (t) = 0.We consider two cases:

Non-overload, the total incoming workload rate over alltasks is less than or equal to the workload servicerate the CPU can handle, i.e.

!i !f

i (t) + !pi (t) ! µ.

In this case each task is first assigned the fluid ser-vice rate it requires µi(t) = !f

i (t) + !pi (t). Tasks that

have any backlog ("i(t) > 0), and this applies to anytasks processing packets, are marked as greedy. Let gbe the number of greedy tasks. Any left-over cycles'(t) = µ "

!i µi(t) are allocated equally to greedy

tasks µi(t) ' µi(t) + '(t)/g. This ensures that thebacklog gets drained as quickly as possible and thuspackets are processed as quickly as possible. Conse-quently, fluid workload results in a processor utiliza-tion in proportion to the incoming rate, while discreteworkload results in bursts of full utilization.

Overload, the sum of the incoming fluid workload ratesand the averaged packet workload rates is greater thanthe service rate of the CPU. That is, there is a sustainedoverload condition. In this case the tasks are deniedcycles in proportion to their fraction of the total work-load, and what cannot be handled accumulates as back-log.

µi(t) =!i(t) · µ!

i !i(t)(1)

An arriving discrete workload (packet) that does notyet have an average rate estimate poses a problem inthis case. It is given µi(t) = 1 (full utilization) with-out affecting other flows. This is unrealistic in that thetotal CPU service rate is now briefly more than µ, butis a reasonable approximation for occasional packets.If the packet is the first in a series with high averageworkload rate, then the service rates will be correctedthe moment the first arrival rate estimate is calculated.

filtering

CPU

forwardingfluid traffic

f 1 f 2 f 3

A B!1

load

"1serviced

Figure 9: Control feedback for fluid flows and tasks usingthe CPU

When a task is defined, a buffer space size bi can be as-signed to it to limit the backlog and introduce the possi-bility of loss of work if the task cannot keep up. Packetsoccupy buffer space according to their size until serviced.Fluid flows are assumed to have a simple linear relation-ship between the workload rate (cycles/second) and mem-ory used for backlog rate (bytes/second).

Modeling loss in hybrid queues is a delicate matter, aspointed out in [26]. If a discrete workload (packet) arrivesto a back-logged task queue such that there is not enoughspace to fit it in the buffer we consider the state of the queue.If it is draining, the average arrival rate is less than the ser-vice rate, and we assume that it will fit (replacing fluidbuffer space with the packet). If the queue is filling, we givethe packet a probability of fitting into the queue equal to it’sproportion of the total task load p = !p

i (t)/(!fi (t)+!p

i (t)).

An unexpected complication resulting from the introduc-tion of the CPU model that has a fluid representation wasthe possibility of feedback loops within a host/router. In anoverload condition, tasks become coupled through compe-tition for CPU but may also be coupled through the traf-fic flow. This may lead to a cyclic dependency of traffic andCPU work. Assume, for instance, that we consider the costof filtering and traffic forwarding in a firewall router. Asillustrated in Figure 9, protocol layers induce load on theCPU. If the CPU gets overloaded it needs to report back tothe protocol layers so that they can reduce the traffic rateemitted. However, since the traffic flow passes first throughfiltering (A) and then forwarding (B) there is a feedbackloop in terms of rate adjustments. When B changes its loadto the CPU, it must update the serviced load for A. A mustthen update the traffic rate emitted to B, which must thenperform another load update to the CPU. For n tandemtasks, where work is proportional to flow rate, the principleof proportional loss (equation 1) limits the feedback. Con-sider the i:th task. Let fi be the inflow, !i = ki · fi be the(offered) workload, and cn

i be the cycles allocated for taski. Initially, flow rate f1 is sent through all tasks, so equa-tion 1 implies we allocate cycles as cn

i = ki/!n

j=1 kj . Tan-dem dependencies means that fi = (cn

i!1/!i!1) · fi!1, and

thus

!i = kicni!1

!i!1fi!1 =

ki · ki!1fi!1 · fi!1!nj=1 kjfi!1ki!1fi!1

=ki!n

j=1 kj

That is, the required cycles !i to handle the adjusted in-flow fi equals the fraction of cycles assigned cn

i , so the allo-cation stabilizes immediately. But completely avoiding thisfeedback loop does not appear possible, so we rate limit thefeedback from the CPU to the protocol layers. Through thisrate limiting, we mimic the control delay imposed by thescheduling mechanism and bound the computational costsin the model.Traffic delay: one difficult issue was how to implement

delays within the protocol stack without incurring signifi-cant overheads. Firstly, we assume that most tasks requireinsignificant overheads so that they do not need to be de-layed or accounted for. Thus, our implementation should beefficient for the frequent case of not modeling processingused for a packet or fluid traffic flow. Moreover, we wantto avoid incurring additional code complexity and eventscheduling as much as possible.iSSFNet uses a protocol model inspired by the x-kernel

design [13], where each host or router contains a proto-col graph containing protocol sessions. The composition ofprotocol session is configurable, as are many parameters foreach protocol session. The key idea is to have a well definedcommon interface through which protocol sessions can beplugged together. These are the push and pop methods. Fig-ure 10 illustrates the position of the push/pop interfaces thatare used as exchange points between the protocol sessions.Packets are pushed to lower protocol sessions and poppedupwards. The iSSF simulation kernel, which iSSFNet isbuilt on top of, supports process-oriented simulation. How-ever, for maximum efficiency, the programming patternsused in the protocol stack are based on event-orientationthrough timer objects and continuations. Supporting arbi-trary suspension points for processing delays in the protocolstack would require switching to a process-oriented modeland instrumenting the code to state save local variables forprocess suspension. (This is done by annotating the C++code to indicate simulation procedures and state variables,after which the iSSF system performs a source-to-sourcetranslation before compilation.) Instead we opted to limitthe possible suspension points, i.e. points in the stack wherediscrete packets can get delayed, to the push/pop entry in-terfaces to the protocol sessions. Thus, multiple delays on apacket within one protocol session will be merged into onedelay that is not incurred until the point where the packet en-ters the next protocol session. The push/pop API’s are goodcandidate suspension points because the state of process-ing of a packet (or a fluid flow) is passed in the packet it-self along with a small number of additional parameters.Hence, we can safely assume that there are no additional

POP

PUSH

POP

PUSH

send()

Memory

CPU

Protocol Graph

HOST/ROUTER

ProtocolSession

ProtocolSession

Socket LayerProtocolSession

ProtocolSessionApplication Layer

sharedresources

suspensionpoints

packet "path"

packet processinguses CPU

packet delayed

Figure 10: Requests for shared resources, like CPU andmemory, can occur anywhere in the protocol graph (stack).But computation delays are only imposed at certain “sus-pension points”, as packets pass from one protocol sessionto the next.

state variables earlier in the execution stack that need sav-ing. So, upon return we continue processing from the pushor pop call without reconstructing the process stack. Otherdata structures in the protocol sessions, such as queues ofpackets that have been delayed pending some condition,evolve through the passage of time, i.e. while the suspendedpacket undergoes processing, and thus do not require sav-ing.The accumulated delay for a packet within a protocol

session is stored in the packet and thus detected as thepacket reaches the next push/pop suspension point. Suspen-sion points can be enabled or disabled through the DMLconfiguration; the idea being to make it easy to aggregatedelays, and thus aggregate events, by having fewer enabledsuspension points.The application layer is different from the rest of the pro-

tocol sessions in that it interfaces with the rest of the proto-col stack through the socket interface (designed to be simi-lar to the BSD socket API). Hence, no packets exist at thislayer. Instead the socket is used as the suspension point andto store delays. If the socket suspend point is disabled, thepacket is marked with the delay which follows it down thestack to the first enabled suspension point.Displacing the suspension point from the point in the

code where the delay should take place alters the causal or-dering of state modifications in the model, i.e. the interleav-ing of updates in simulation time will be slightly altered.We believe this will not be a significant issue for the proto-cols under consideration here, but more experience with themodel will be needed to bear this out.

0

10

20

30

40

50

60

70

80

90

100

0 5 10 15 20 25 30

Per

cent

Util

izat

ion

Time [s]

measuredsim., discrete

sim., fluid

Figure 11: Example: scp transfer

7.2. Example

We illustrate the CPU model through a very simple ex-ample. In an experiment a 41.6 MB file was downloadedfrom a Linux laptop (acting as the server) using scp (se-cure copy). The CPU load on the data source (server) wasmonitored using vmstat. Figure 11 shows the CPU uti-lization during the transfer as “measured”. This scenariowas modeled in iSSFNet using its packet level TCP model.A client host is connected directly to a server host through a100 Mb/s link. When modeling the CPU load, we have twochoices: use a fluid representation of the load on the CPU, oruse discrete chunks of work. The advantage of the fluid rep-resentation is that it is simple to use and has very low simu-lation cost. The drawback is that it is coarse and will not im-pose any delay on the sending of the packets. Discrete workis more expensive to simulate, but is more fine-grained anddelays packets.Using fluid work, we simply call a setFluid method

on the CPU, as the transfer starts, to set the instruction rateduring the transfer (we simply match the observed utiliza-tion). When the transfer is completed the instruction rateis set back to zero. The result, shown as “fluid load”, indi-cates a shorter transfer time than what was measured. Al-ternatively, we can use discrete workloads. Examining theOpenSSH scp implementation indicates that it transfers datathrough a 2 KB buffer, so we write data to the socket in 2 KBblocks and impose a compute delay on each block for datatransfer and encryption. The computation cost is registeredthrough a call to cpu.use(...) with the number of in-structions used and a pointer to the socket being used. Thesocket send() code hides a call to cpu.delay(...)causing the socket processing to be suspended and delayed.We also use a timer to add a small idle delay between eachblock to model latencies. After tuning these delays, the re-

sult shown as “discrete load”, can be made to match realityfairly well.There is a significant difference in simulation cost be-

tween these two approaches. Using fluid CPU load, no ex-tra events are added by the CPU model, but with the dis-crete workload model, each block requires a resource de-parture event and results in an event for drained backlog.Thus, the total event count increases by a factor of about2.4 and the execution time by a factor of 4. It is up to themodeler to determine when the additional cost is justified.

7.3. Discussion

Linear regression on results from a small benchmarkmodel (a sequence of forwarding routers) indicated thatthe event cost for discrete (per-packet) CPU work delaysis more than twice that of packet forwarding events, dueto complex updates and statistics bookkeeping. We are cur-rently looking into ways to reduce this cost, but it appearslikely that there will still be a significant overhead associ-ated with resource updates for discrete work. Hence, it ap-pears necessary to be selective in what costs to model and/oruse coarser fluid representations of the workload imposedon the CPU also for packet-level traffic.Aside from approximations arising from implementation

decisions, the current CPU resource model represents manysimplifications. The principle of proportional loss is fre-quently used for fluid traffic and alleviates the allocationfeedback issue mentioned previously. But we see the needfor more emphasis on distinction of task priorities to bettermimic prioritization of processes and threads. For instance,kernel level processes should be largely insulated from de-mands at the user level. We are looking into new allocationpolicies that can prioritize demands.

8. Summary and Future Work

RINSE incorporates recent work on i) real-time inter-action/emulation support, ii) multi-resolution traffic model-ing, iii) efficient attack models, iv) efficient routing simula-tion, and v) CPU/memory resource models, to target large-scale preparedness and training exercises. Described herewere efficient CPU/memory models necessary for the sce-nario exercises, and a latency absorption technique that willhelp when extending the range of client tools usable bythe players. We also provided empirical results that pointto the significant performance improvements that are possi-ble in simulations of Distributed Denial-of-Service attacksby leveraging off the fluid based traffic models. The net-work worm models incorporated in RINSE permit model-ing the interaction with network infrastructure and the prop-agation dynamics have been validated against collected dataduring real worm attacks.

Aside from model refinements, our ongoing and futurework includes more fundamental issues such as supportingfault tolerance and efficient real-time scheduling of com-pute intensive tasks like background traffic calculations andmajor routing changes. For example, we would like our sim-ulation framework to permit certain background tasks, suchas background traffic calculations, to be adaptively sched-uled based on higher priority load.Acknowledgements This research was supported in partby DARPA Contract N66001-96-C-8530, NSF Grant CCR-0209144, and DHSOffice for Domestic Preparedness award2000-DT-CX-K001. Thus, the U.S. Government retains anon-exclusive, royalty-free license to publish or reproducethis contribution. Points of view expressed are those of theauthors and do not necessarily represent the official posi-tion of the U.S. Department of Homeland Security.

References

[1] DARPA NMS baseline challenge network model.http://www.ssfnet.org/Exchange/gallery/-baseline/index.html, 2002.

[2] T. Bridis. Gov’t simulates terrorist cyberattack. Asso-ciated Press, http://www.zone-h.org/en/news/-read/id=3728, November 2003.

[3] T. Bu and D. Towsley. Fixed point approximations for TCPbehavior in an AQM network. In Proceedings of ACM SIG-METRICS 2001, Cambridge, Massachusetts, June 2001.

[4] J. Chen, D. Gupta, K. Vishwanath, A. Snoeren, and A. Vah-dat. Routing in an Internet-scale network emulator. InSymposium on Modeling, Analysis and Simulation of Com-puter and Telecommunication Systems (MASCOTS), October2004.

[5] J. Cowie, D. Nicol, and A. Ogielski. Modeling the global In-ternet. Computing in Science and Engineering, 1(1):42–50,January 1999.

[6] X. Dimitropoulos and G. Riley. Creating realistic BGP mod-els. In Symposium on Modeling, Analysis and Simulationof Computer and Telecommunication Systems (MASCOTS),October 2003.

[7] X. Dimitropoulos and G. Riley. Large-scale simulation mod-els of BGP. In Symposium on Modeling, Analysis and Sim-ulation of Computer and Telecommunication Systems (MAS-COTS), October 2004.

[8] K. Fall. Network emulation in the Vint/NS simulator. In4th IEEE Symposium on Computers and Communications(ISCC’99), pages 244–250, July 1999.

[9] R. Fujimoto, K. Perumalla, A. Park, H. Wu, M. Ammar, andG. Riley. Large-scale network simulation – how big? howfast? In Symposium on Modeling, Analysis and Simulationof Computer Telecommunication Systems (MASCOTS), Oc-tober 2003.

[10] T. Griffin and B. Premore. An experimental analysis of bgpconvergence time. In 9th International Conference on Net-work Protocols (ICNP), November 2001.

[11] F. Hao and P. Koppol. An Internet scale simulation setup forBGP. ACM SIGCOMM Computer Communication Review,33(3):43–57, July 2003.

[12] P. Huang and J. Heidemann. Minimizing routing state forlight-weight network simulation. In Symposium on Model-ing, Analysis and Simulation on Computer and Telecommu-nication Systems (MASCOTS), August 2001.

[13] N. C. Hutchinson and L. L. Peterson. The x-kernel: An archi-tecture for implementing network protocols. IEEE Transac-tions on Software Engineering, 17(1):64–76, January 1991.

[14] G. Kesidis, A. Singh, D. Cheung, and W. W. Kwok. Feasibil-ity of fluid-driven simulation for atm network. In Proceed-ings of IEEE Globecom, November 1996.

[15] C. Kiddle, R. Simmonds, C. Williamson, and B. Unger. Hy-brid packet/fluid flow network simulation. In 17th Workshopon Parallel and Distributed Simulation, June 2003.

[16] M. Liljenstam and D. Nicol. On-demand computation of pol-icy based routes for large-scale network simulation. In 2004Winter Simulation Conference (WSC), December 2004.

[17] M. Liljenstam, D. Nicol, V. Berk, and R. Gray. Simulatingrealistic network worm traffic for worm warning system de-sign and testing. In 2003 ACM Workshop on Rapid Malcode(WORM), October 2003.

[18] M. Liljenstam, Y. Yuan, B.J. Premore, and D. Nicol. Amixed abstraction level simulation model of large-scale in-ternet worm infestations. In Symposium on Modeling, Analy-sis and Simulation of Computer and Telecommunication Sys-tems (MASCOTS), October 2002.

[19] B. Liu, D. R. Figueiredo, Y. Guo, J. Kurose, and D. Towsley.A study of networks simulation efficiency: Fluid simulationvs. packet-level simulation. In IEEE Infocom, Anchorage,Alaska, April 2001.

[20] B. Liu, Y. Guo, J. Kurose, D. Towsley, and W. Gong. Fluidsimulation of large scale networks: Issues and tradeoffs. InProceedings of the International Conference on Parallel andDistributed Processing Techniques and Applications, LasVegas, Nevada, June 1999.

[21] X. Liu, H. Xia, and A. Chien. Network emulation toolsfor modeling grid behavior. In 3rd IEEE/ACM Interna-tional Symposium on Cluster Computing and the Grid (CC-Grid’03), May 2003.

[22] S. McCanne and Van Jacobson. The BSD packet filter: a newarchitecture for user-level packet capture. InWinter USENIXConference, pages 259–269, January 1993.

[23] D. Nicol and J. Liu. Composite synchronization in paral-lel discrete-event simulation. IEEE Transactions on Paralleland Distributed Systems, 13(5):433–446, May 2002.

[24] D. Nicol, J. Liu, M. Liljenstam, and G. Yan. Simulationof Large-Scale Networks Using SSF. In Winter SimulationConference (WSC), December 2003.

[25] D. Nicol, S. Smith, and M. Zhao. Evaluation of efficient se-curity for BGP route announcements using parallel simula-tion. Simulation Pratice and Theory Journal, special issueon Modeling and Simulation of Distributed Systems and Net-works, June 2004.

[26] D. Nicol and G. Yan. Discrete event fluid modeling of back-ground TCP traffic. ACM Transactions on Modeling andComputer Simulation, 14:1–39, July 2004.

[27] D. Nicol and G. Yan. Simulation of network traffic at coarsetime-scales. In Workshop on Principles of Advanced andDistributed Simulation (PADS), 2005.

[28] David M. Nicol, Michael Liljenstam, and Jason Liu. Mul-tiscale modeling and simulation of worm effects on the in-ternet routing infrastructure. In Proceedings of 13th Inter-national Conference on Modeling Techniques and Tools forComputer Performance Evaluation (Performance TOOLS),September 2003.

[29] J. Padhye, V. Firoiu, D. Towsley, and J. Kurose. ModelingTCP throughput: A simple model and its empirical valida-tion. In Proceedings of ACM SIGCOMM’98, Vancouver, CA,September 1998.

[30] G. Riley, M. Ammar, and R. Fujimoto. Stateless routing innetwork simulations. In Symposium on Modelling, Analysis,and Simulation of Computer and Telecommunication Sys-tems (MASCOTS), May 2000.

[31] G. Riley, T. Jaafar, and R. Fujimoto. Integrated fluid andpacket network simulations. In Symposium on Modeling,Analysis, and Simulation of Computer and Telecommunica-tion Systems (MASCOTS), October 2002.

[32] L. Rizzo. Dummynet: a simple approach to the evaulation ofnetwork protocols. ACM SIGCOMM Computer Communi-cation Review, 27(1):31–41, January 1997.

[33] A. Savvides, S. Park, and M. B. Srivastava. On modelingnetworks of wireless micro sensors. In SIGMETRICS, June2001.

[34] R. Simmonds and B. Unger. Towards scalable network em-ulation. Computer Communications, 26(3):264–277, Febru-ary 2003.

[35] N. Spring, R. Mahajan, and D. Wetherall. Measuring ISPTopologies with Rocketfuel. ACM SIGCOMM ComputerCommunication Review, 32(4):133–145, 2002.

[36] A. Tanenbaum. Modern Operating Systems, 2nd ed. PrenticeHall, Upper Saddle River, NJ, 2001.

[37] A. Vahdat, K. Yocum, K. Walsh, P. Mahadevan, D. Kos-tic, J. Chase, and D. Becker. Scalability and accuracy in alarge scale network emulator. In Proceedings of the 5th Sym-posium on Operating Systems Design and Implementation(OSDI’02), December 2002.

[38] B. White, J. Lepreau, L. Stoller, R. Ricci, S. Guruprasad,M. Newbold, M. Hibler, C. Barb, and A. Joglekar. An inte-grated experimental environment for distributed systems andnetworks. In Fifth Symposium on Operating Systems Designand Implementation, pages 255–270, December 2002.

[39] Yougu Yuan. On the design of an immersive environment forsecurity-related studies. Technical Report 2005-552, Dart-mouth College, Hanover, NH, August 2005.

[40] J. Zhou, Z. Ji, M. Takai, and R. Bagrodia. MAYA: integratinghybrid network modeling to the physical world. ACM Trans-actions on Modeling and Computer Simulation (TOMACS),14(2):149–169, April 2004.

[41] C. Zou, L. Gao, W. Gong, and D. Towsley. Monitoring andearly warning for internet worms. In CCS ’03: Proceedingsof the 10th ACM conference on Computer and communica-tions security, pages 190–199, New York, NY, USA, 2003.ACM Press.