OMNINet

Joe Mambretti, Director, (j-mambretti@nwu.edu)

International Center for Advanced Internet Research (www.icair.org)Director, Metropolitan Research and Education Network (www.mren.org)

Partner, StarLight/STAR TAP (www.startap.net/starlight)PI-OMNINet (www.icair.org/omninet)

JETNew Port News, Va

April, 2004

• Creation and Early Implementation of Advanced Networking Technologies - The Next Generation Internet All Optical Networks, Terascale Networks

• Advanced Applications, Middleware, Large-Scale Infrastructure, NG Optical Networks and Testbeds, Public Policy Studies and Forums Related to NG Networks

Accelerating Leading Edge Innovation and Enhanced Global Communications through Advanced Internet Technologies, in Partnership with the Global Community

Introduction to iCAIR:

CONTROL

PLANE

Clusters

DynamicallyAllocatedLightpaths

Switch Fabrics

PhysicalMonitoring

Apps

Multi-leveled Architecture

NEW!

Multiwavelength Optical Amplifier

Multiwavelength Fiber

CSWASW

ASW

λ1λ2λ3λ4

10GE Links

GE LinksΝ*N*N

ΙΕΕΕ 802.3 LAN PHYInterface, eg, 15xx nm 10GE serial

10GE Links

GE Links

Optical, λ Monitors, for Wavelength Precision, etc.

Power Spectral Density Processor, Source + Measured PSD

DWDM Links

Multiple λ Per Fiber

Computer Clusters Each Node = 1GEMulti 10s, 100s, 1000s of Nodes

Multiple Optical Impairment Issues, Including Accumulations

Grid Clusters

Grid Clusters

Controller

Client Device

Client Controller

ControllerControllerController

Controller

Optical Layer Control PlaneClient Layer Control Plane Optical Layer Control Plane

Client Layer Traffic Plane

Optical Layer – Switched Traffic Plane

UNI

I-UNI

CICICI

+ Optical DynamicIntelligent Network (ODIN)for Dynamic Provisioning

Performance Monitoring & Control

IIIIIII

IIIIIII

IIIIIII

IIIIIII

IIIIIII

IIIIIIIIIIIIII

IIIIIII

IIIIIII

OMNInet Testbed Experiments (MREN Used as Out-of-Band Control Channel)

4x10GE

Northwestern U

OpticalSwitchingPlatform

Passport8600

ApplicationCluster

OMNInet Technology Trial: January 2002

• A four-site network in Chicago -- the first 10GE service trial!• A test bed for all-optical switching and advanced high-speed services• Partners: SBC, Nortel, iCAIR at Northwestern, EVL, CANARIE, ANL

ApplicationCluster

OpticalSwitchingPlatform

Passport8600

4x10GE

StarLight

OPTera Metro5200

ApplicationCluster

OpticalSwitchingPlatform

Passport8600

4x10GE8x1GE

UIC

(1st @CA*net3/4—Chicago)

OpticalSwitchingPlatform

Passport8600

ApplicationCluster

4x10GE8x1GE

8x1GE

8x1GE

600 South Federal

Testbed Configuration

StarLight

Ca*net4

iCAIR EVL UIC

OFA

OFA

OFA

NWUEN-1

NWUEN-2

NWUEN-3

NWUEN-4

NWUEN-5

NWUEN-6

NWUEN-7

NWUEN-8 NWUEN-9

ADM OC-12 to MREN

Fiber

SW

10/100/GE

SW

10/100/GE

10/100/GE

OFA

Up to 8 λPer fiber

8 λ AWG

• Scalable photonic switch• Trunk side – 10 G WDM• Lambda specific interface

λ1 λ2 λ3

To CA*Net4, SURFnetand Other

International Nets

10 GbE WANSW

OFA

10/100/GE

λ1 λ2 λ3

To Optical MREN

OMNInet Network Configuration

10 GE10 GE

To Ca*Net 4

Lake Shore

Photonic Node

S. FederalPhotonic

Node

W Taylor SheridanPhotonic

Node10/100/GIGE

10/100/GIGE

10/100/GIGE

10/100/GIGE

10 GEOptera5200

10Gb/sTSPR

Photonic Node

λ4

PP8600

10 GEPP

8600

PP8600

2

3

4

1

λλλ

λ

λλ

λ2

3

1

Optera5200

10Gb/sTSPR

10 GE10 GE

Optera5200

10Gb/sTSPR

2

3

4

1

λλλ

λ

Optera5200

10Gb/sTSPR

2

3

4

1

λλλ

λ

1310 nm 10 GbEWAN PHY interfaces

10 GE10 GE

PP8600

Fiber

KM MI1* 35.3 22.02 10.3 6.43* 12.4 7.74 7.2 4.55 24.1 15.06 24.1 15.07* 24.9 15.58 6.7 4.29 5.3 3.3

NWUEN Link

Span Length

…CAMPUSFIBER (16)

EVL/UICOM5200

LAC/UICOM5200

CAMPUSFIBER (4)

INITIALCONFIG:10 LAMBDA(all GIGE)

StarLightInterconnect

with otherresearchnetworks

10GE LAN PHY (Dec 03)

• 8x8x8λ Scalable photonic switch• Trunk side – 10 G WDM

• OFA on all trunks

TECH/NU-EOM5200

CAMPUSFIBER (4)

INITIALCONFIG:10 LAMBDAS(ALL GIGE)

Optera Metro 5200 OFA

NWUEN-1

NWUEN-5

NWUEN-6NWUEN-2

NWUEN-3

NWUEN-4

NWUEN-8 NWUEN-9NWUEN-7

Fiber in useFiber not in use

5200 OFA

5200 OFA

Optera 5200 OFA

5200 OFA

DOTClusters

Overlay Management network (Typical Site & Inter-site Network)

Photonic Switch

OPTera 5200 OFA & OTR

Passport 8600

Local Management

Station (Lakeshore only)

10/100 BT

10/100 BTInternet Connection

(Lakeshore only)

Firewall

Centilian swor Baystack

350T Hub

100BT/1000BT

• Hub and spoke network from 710 Lakeshore Dr.

Evanstan(Sheridan node)

710 N. Lakeshore

Dr. (Starlight)

600 S. Federal (LGN CO-LO)

UIC(W.Taylor

node)

192.26.85.0/27

Overlay Management Network

5/18/2004

Ethernet VPN via OMNInet

& PP8600

Ethernet VPN via I-WIRE &

PP8600

Ethernet VPN via NU Inter-campus LAN

Photonic H/W

Photonic Performance SubsystemR

elativeFiber pow

erR

elativeλ

power

Tone code

A/D

PPS Control Middleware

tapOFA

D/A

Management & OSC Routing

VOA

D/A

Power measurement

switch

SwitchControl

AWG Temp.Control alg.

D/AA/D

AWG

Heater

+-

Setpoint

DSP Algorithms & Measurement

tap

PhotoDetectorPhotoDetector

Drivers/data translation

Connectionverification

Path ID Corr.

Fault isolation

Gain Controllerλ Leveling

Transientcompensator

Power Corr.

+-

LOS

+-

PhotonicsDatabase

100FXPHY/MAC

Splitter

OSC cct

FLIPRapidDetect

Source: Kent Felski, Nortel

Chicago

iCAIR Grid Clusters and OM5200 DWDM System

Leverone Hall Data Com Center

Up to 16xGE(SMF LX)

4-fibers~1km

DOT ClustersFor Telecom2003 Demo

Technological Institute

10GE WAN/LAN

PHY to OMNInet

OM5200 OM5200PP8600

~20m

DWDM Between Cluster Site and OMNInet Core Node in Evanston

• The implementation is lambdas (unprotected).• Installed shelf capacity and common equipment permits expansion of up to 16 lambdas

through deployment of additional OCLD, and OCI modules.• A fully expanded OM5200 system is capable of supporting 64 lambdas (unprotected) over

the same 4-fiber span.

DWDM on

Dedicated

Fiber~20m

GRID Distributed Optical Testbed Components

Enviromental Conditioning, Power

Cluster

Cluster

AdvancedPhotonics

DOT Sites, I-WIRE, and OMNInet

UIUC/NCSA

Starlight(NU-Chicago)Argonne

UChicagoIIT

UIC

Illinois Century NetworkJames R. Thompson CtrCity HallState of IL Bldg

4 pair

12 pair

4 pair

2 pair 2 pair

4 pair

18 pair

4 10 pair

12 pair

2 pair

Level(3)111 N. Canal

McLeodUSA151/155 N. MichiganDoral Plaza

Qwest455 N. Cityfront

UC Gleacher450 N. Cityfront

OMNInet

All DOT Links Here= GE

Not Yet Provisioned

NCSA ClusterNot Yet Implemented

Fiber being Deployed atUoC, IIT

Not Yet Part of Testbed

No Routing –L2 Switching Only!

Linux Traffic Gen

259M

Linux

DV

IPSLinux

WindowDV/Linux

Linux/IPS

NTSC/SDI

iCAIR

O3 O4

O1 O2

StarLightStarLight

Display

Netherlight

SX270M/540M

240M/480M

380M

100M

Chicago

Amsterdam

GigE TX400M/900M

Omninet

NU LeveroneEvanston, IL

UIC/EVLGigE

GigEGigE

GigEGigEGigE

GigE

GigEGigE

GigE

GigE

OM

NIN

ETO

MN

INET λ

λλλ

λ λ λλ

100M TX

iGrid 2002

100M TX

GigE SX270M540M

GigE TXLaptop

Linux TrafficGenerator

259M

SurfNet

100M TX30M/60M

400M/900M

240M/480M

Nelle BaconiCAIRAugust 21, 2002

2.5 Gbps 10 Gpbs

SARAnet

OMNINETOMNINET SURFnet

Photonic TeraStreamApplication

iGrid 2002United States and The Netherlands

Photonic TeraStream• International Center for Advanced Internet Research (iCAIR), Northwestern

University, USA • University of Amsterdam, The Netherlands

www.icair.org

This project showcases new technologies and techniques being developed on the StarLight/NetherLight testbed and on OMNInet in support of high-performance digital media and extremely-high-performance data transfer.

This 10Gbps capability is being used to develop a reference model for a next-generation core optical metro network infrastructure. This infrastructure will allow access to and dynamic interaction with very large amounts of data. This application can utilize multi-Gbps today; in the future, it may utilize multiple 10GigE streams.

iGrid 2002United States

Optical Dynamic Intelligent Network (ODIN)• International Center for Advanced Internet Research• Electronic Visualization Laboratory• Laboratory for Advanced Computing, UIC

ODIN is mechanism that allowsglobal applications to use intelligent signaling to provision their own international lightpaths to optimize network based resource discovery and performance.

www.icair.org, www.evl.uic.edu, www.lac.uic.edu

iGrid 2002United States

Terascale High-Performance Optical Resource Regulator (THOR)

• International Center for Advanced Internet Research• Electronic Visualization Laboratory• Laboratory for Advanced Computing, UIC

THOR is a method of provisioningand managing lightpaths on wavelength-switchednetworks

www.icair.org, www.evl.uic.edu, www.lac.uic.edu

iGrid 2002United States – Set New World Record for Data Transport

Photonic Data Services• Laboratory for Advanced Computing, UIC• National Center for Data Mining, UIC• International Center for Advanced Internet Research• SURFnet, NetherLight

www.lac.uic.edu, www.icair.org

At iGRID2002 the LAC, NCDM and iCAIR demonstrated Photonic Data Services (PDS) and set a new data transit record, based on several new protocols designed for high performance optical networks

Photonic Data Services Stack –Cooperative Project with LAC

1. Physical

4. Transport – TCP, UDP, SABUL,… (UIC/LAC)

5b. Data Web Serv. (UIC/LAC)

6. Data Intensive Applications

2. Photonic Path Serv. – ODIN, THOR,... (NW)

3. IP

5a. Storage (UCSD)

5c. Data Grid Services

Source: Bob Grossman

DARPA DWDM-RAM Large Scale Data+Dynamic Lambdas – Demonstrated at GGF9 & SC2003

Dynamic Lightpaths

Grid L3-L7 OGSA Compliant

Data Web Services

Data Intensive App2

Dynamic vLANs

Dynamic Path Services (ODIN, THOR, etc),OGSA Compliant, Soon WSRF

Grid Data Management

ServicesData Grid Services

Data Intensive App3HP-PPFS Data Intensive App4

Physical Processing Monitoring and Adjustment

NewControl Plane

And Management

Plane Processes

OptIPuter Open Source LambdaGrid Software for Distributed Virtual Computers

Source: Andrew Chien, UCSDOptIPuter Software Architect

European lambdas to US–8 GigEs Amsterdam—Chicago–8 GigEs London—Chicago

Canadian lambdas to US–8 GigEs Chicago-Canada-NYC–8 GigEs Chicago-Canada-Seattle

US lambdas to Europe–4 GigEs Chicago—Amsterdam–3 GigEs Chicago—CERN

European lambdas–8 GigEs Amsterdam—CERN –2 GigEs Prague—Amsterdam–2 GigEs Stockholm—Amsterdam–8 GigEs London—Amsterdam

TransPAC lambda (yellow)–1 GigE Chicago—Tokyo

IEEAF lambdas (blue)–8 GigEs NYC—Amsterdam–8 GigEs Seattle—Tokyo

TransLight Lambdas

NortelPassport 8600

NortelPassport 8600

Photonic Switch

NU-E (Leverone) MonitorServers

2 x GE

StarLight-Chicago

1 x GE

Photonic Switch

ONI

10GE

10GE

ONI

1 GE

1 x GE

MonitorServer?

McCormick School of Engineering, Tech Institute

10GE

10*1GElinks DOT

Clusters

DWDM onDedicated

Fiber

DWDM onDedicated

Fiber

DOTCluster

Switch

NCSA

CERN/iCAIR WSIS Demo Dynamic Wavelength Provisioning

And Medical imaging

NortelPassport 8600

NortelPassport 8600

Photonic Switch

NU-E (Leverone) FASTWorkstations

2 x GE

StarLight-Chicago

CalTechCisco 7609

2 x GE

Photonic Switch

Alcatel1670

10GE

10GE

Alcatel1670

2 x GE2 x GEOC-48

DataTAG

2 x GE

FASTWorkstations

CERN -Geneva

San Diego

FAST dispaly

10GE

CERNCisco 7609

10,000 km

J. Mambretti, F. Yeh (Northwestern)

A. Adriaanse, C. Jin, D. Wei (Caltech)

S. Ravot (Caltech/CERN)

CERN/iCAIR/CalTech FAST DemoCheng Jin, David Wei

Caltech