Giving users control

Designing the Future 2005Sydney, 6 April 2005

Rene.Hatem@canarie.caTel: +1.613.944.5604http://www.canarie.ca/

CANARIE Inc.- Overview

• Federal leadership: Concept born in 1990 out of Industry Canada discussions

• Founding: Incorporated in 1993 by industry and academia

• Funding: From Industry Canada: For networks and research applications from Canadian Heritage, HRDC, Health Canada

• Mission: To facilitate development and use of Canada’s advanced communications infrastructure

• Primary stakeholders: universities, Government Departments, provincial research networks, broader research community, colleges, carriers, IT sector, SMEs, broader education sector, broader health sector, provinces

Dark fiber: Businesses see the light

> http://news.com.com/Dark+fiber+Businesses+see+the+light/2100-1037_3-5557910.html?tag=sas.email

> According to publisher of USA Today, if you are spending more than $7000/mth on telecom, then dark fiber is for you

> Lighting up fiber used to be technically difficult, but CWDM has made it a no brainer

> Next generation CWDM will allow up to 80 Gbps for less than $5k with 10Gbps wavelength

Customer owned fiber for businesses

> Significant reduction in price for local loop costs> Ability to outsource LAN and web servers to distant location as

LAN speeds and performance can be maintained over dark fiber> Access to lower cost competitive service providers at carrier

neutral hotels– New entrants cannot afford high cost of building out their own fiber

networks> Reduce Internet transit costs via remote peering> Examples:

– Colgate-Palmolive build in Cincinnati– Bank of America– Bell Canada subsidiary CGI in Montreal– Lehman Brothers in NY– Ford in Detroit

Condominium Fiber Networks

> Several next generation carriers and fiber brokers are now arranging condominium fiber builds– Lexent, Urban Networks, America Fiber, Looking Glass, etc

> Organizations such as schools, hospitals, businesses, municipalities and universities become anchor tenants in the fiber build

> Each institution gets its own set of fibers on a point to point architecture, at cost, on a 20 year IRU

> Fiber is installed and maintained by 3rd party professional fiber contractors

> Institution lights up their own strands with whatever technology they want – Gigabit Ethernet, ATM, PBX, etc

> Cost – on average $25K US plus $1500 per year for maintenance

Municipal Condo Architecture

School

School board orCity Hall

School

Telco Central Office

Central OfficeFor Wireless

Company

VDSL, HFC or FTTH

Condominium Fiber with separate strands owned by school and by service providers

Carrier Owned Fiber

Cable head end

Average Fiber Penetration to 250-500

homes

ColoFacility 802.11b

Business

NYC Condo build with Lexent Inc

Canberra ICON project

> ICON provides ‘dark fibres’ to all government departments and agencies in Australian capital Canberra

> Several thousand fiber strands between sites> ICON does not mandate any speed nor protocol

– Gives agencies whatever they want– Gigabit Ethernet is very common

> One time cost of $1000 per strand of fiber from anywhere to anywhere– No charges for bandwidth– Each agency has annual maintenance cost of $15000 per

annum regardless of number of fibers

Halifax Condo Fiber Build

Private sector fibre optic network

12-15 km$350,000 build$150,000 engineering

Links all major universities, hospitals, research centers and some schools

Connects to CA*net4 at Nova Scotia GigaPOP

What’s next?

> As more and more institutions and users acquire fiber and wavelengths there will be a need to interconnect these islands of fiber

> Cost of national wavelengths less than $US 150K per year– Individual researchers and/or institutions can afford their own

local fiber and national wavelengths

> Two ways to interconnect these networks– Purchase a managed service from telcos; or– Develop a new peer to peer technology that allows direct

interconnection

Customer Controlled Lightpaths

> CANARIE’s CA*net 4 is world’s first customer controlled network

> Rather than building a traditional IP routed network with central management and control, CA*net 4 is made up of a number of separate customer controlled IP networks

– Articulated Private Networks (APNs) using UCLP technology> Examples

– Large enterprise wide area network for NRC institutes– Discipline specific IP network for high energy physics facilities– Distributed backplane for computational grids

> Provides most of the advantages of dark fiber– Customer can control bandwidth, routing, topology, add/drop, etc– Customer can partition network and offer it to third parties– Customer can do their own inter-domain connectivity

> Similar concepts in ITU Y.1312/Y.1313

What is UCLP?

> User Controlled LightPaths – a configuration and provisioning tool built around web services

> A proxy that sits in front of optical switches and SONET cross connects that allows control of a subset of the cross connects to be delegated to a third party

> Third party can concatenate cross connects together from various networks to produce a wide are network that is under their control

– Articulated Private Network (APN)> Uses Service Oriented Architecture (SOA) and so network can

be integrated with other web service applications> APN can also do routing or switching with logical routers or

switches represented as web services

UCLP general operation

CA*net 4 UCLPSwitch Agents

Signal Control Plane Agents

Standard CLI or TL1 interface

Customer A and sub- partition

Customer B

Customer C

DWDM EastboundDWDM Westbound

X

X

OSPF

GMPLS

ISIS

Customer A signaling plane

Subtended Lightpaths to User

Customer B signaling plane

Grooming agents

Customer C signaling plane

X X

Customer A UCLP Server

MonFox TL1 Proxy

OXC

X

Customer C signaling plane

Customer C

User Controlled LightPaths (UCLP): Objectives

> Wide Area Network for the enterprise– To integrate wavelengths and fiber from different suppliers within

institution’s network management domain– offer VPNs to users

> Create discipline specific re-configurable IP networks– Multi-homed network which bypasses firewalls with direct connect to

servers and routers– Crosses multiple domain and institutions

> User controlled traffic engineering for remote peering– Active replacement for Sockeye and Route Science– Alternative to MPLS

UCLP intended for projects like National LambdaRail

CAVEwave acquires a separate wavelength between Seattle and Chicago and wants to manage it as part of its network including add/drop, routing, partition etc

NLR Condominium lambda network

OriginalCAVEwave

Today’s hierarchical IP network

University

Regional

National or Pan-Nationl IP Network

Other national networks

Regional A Regional B Regional C Regional D

Remote peering for ISPs and enterprise

World

UniversityRegional

Server

World World

National DWDM Network

Regional A Regional BRegional C Regional D

ChildLightpaths

Child Lightpaths

Creation of specialty IP networks

CommodityInternet

Bio-informaticsNetwork

University

University

University

CERN

University

University

Automobile Parts Network

Business Supply Chain Network

Dept

Research Network

CANARIE provides APN to TRIUMF

Amsterdam

New York

Toronto

Vancouver

Victoria

Edmonton

Ottawa

Geneva

10G Lightpath WS

1G Interface WS

5G Interface WS

1. Note: An incoming lightpath (STS) can be assigned to an outgoing STS or a specific interface

2. TIUMF UCLP GUI would only see this APN3. CANARIE UCLP GUI can this APN or

underlying network or other APN

1G Lightpath WS

Montreal

To Fermi

To Brookhaven

URI: http://canarie_apns/triumf_apn.ws

TRIUMF GUI harvests other APNs from UoVic, UoT, etc

UoToronto PhysicsTier 2

UoVictoria PhysicsTier 2

TRIUMFTier 1

CERNTier 0

Amsterdam

New York

Chicago

Toronto

Vancouver

Victoria

FERMITier 1

BrookhavenTier 1

UBC Physics

UA Physics

UoT Physics

Carleton Physics

UdM Physics

CA*net 4

Edmonton

Ottawa

Geneav

10G Lightpath WS

TRIUMFAPN

UoTAPN

UoVAPN

1G Interface WS

5G Interface WS

External links or APNs

Note: Typical View on TRIUMF UCLP GUI

Montreal

1 Gbe 5 Gbe

GUI display using a workflow tool

2 Gbe Vic-Van

10 Gbe Van-Edm

10 GbeEdm-Tor

10 Gbe Tor-NYC

1 GbeTor-Ott

10 Gbe NYC-Ams

10 Gbe Ams-Gen

1 Gbe NYC-MTl

1 Gbe

1 Gbe

1 Gbe

1 Gbe 1 Gbe

1 Gbe

5 Gbe

5 Gbe

5 Gbe

1 Gbe

1 Gbe

TRIUMF

FERMI

Brookhaven CERN

UoVic Tier 2

UoT Tier 2

Harvested APNs

1. http://TRIUMF_APN/triumf.ca2. http:/UoVic-APN/uvic.ca3. http://UoT_APN/uot.ca

Interface web service

Lightpath web service

External web service

ATLASserver

TRIUMFVLAN

TRIUMFCWDM

Tier 2Server

Note: External APN may be represented as a single web service

“drag and drop”

1G HEPnet daisy chainrouted

TRIUMF partitions APN and establishes cross connects with 3rd parties APNs

UoToronto PhysicsTier 2

UoVictoria PhysicsTier 2

TRIUMFTier 1

CERNTier 0

Amsterdam

New York

Chicago

Toronto

Vancouver

Victoria

FERMITier 1 Brookhaven

Tier 1

UBC Physics

UA Physics

UoT Physics

Carleton Physics

UdM Physics

CA*net 4

Edmonton

Ottawa

To other physics users at smaller universities Geneav

CWDMCWDM

5G Tier 1 data

2G Tier 2 data

Optionalinterfaces

Note: Typical View on TRIUMF UCLP GUI

TRIUMF creates child APN for HEPnet

Toronto

Vancouver

Victoria

UBC Physics

UA Physics

UoT Physics

Carleton Physics

UdM Physics

CA*net 4

Edmonton Ottawa

UoVAPN

1G Interface WS

Montreal

Note: TRIUMF has created this child APN from elementsfrom the original CANARIE APN and the APNs provided by UoVictoria, TRIUMF, UoT, etc

Note: View seen by HEPnet UCLP GUI

CERN

HEPnet APNcannot see switches in Amsterdam or NY

Resultant HEPnet routed network

UBC Physics

UA Physics

UoT Physics

Carleton Physics

UdM Physics

CA*net 4

UoVAPN

1G Interface WS

Montreal

CERN

To smaller physics depts through university router

CANARIE provides APN to NRC

New York

ChicagoSeattle

Victoria

Vancouver

Edmonton

CalgaryRegina

Saskatoon

Winnipeg

Toronto

Ottawa

Montreal

Fredericton

HalifaxCA*net 4 router2G Lightpath WSGbE interface WS

NRC partitions APN

New York

ChicagoSeattle

Victoria

Vancouver

Edmonton

CalgaryRegina

Saskatoon

Winnipeg

Toronto

Ottawa

Montreal

Fredericton

Halifax

NRC logical view of APN

New York

ChicagoSeattle

Victoria

Vancouver

Edmonton

Regina

Saskatoon

Winnipeg

Toronto

Ottawa

Montreal

Fredericton

Halifax

Integrating network into SOA

> All hardware (sensors -wireless and wired), software processes (Data processing and HPC) and network elements (ORAN, CA*net 4) expressed as WSDL web services – Web services may be instantiations of orchestrations

> Hardware, software and network web services linked together by science user with BPEL– WSDL and BPEL provide for generic and open control plane

> Elimination of network made up of layers– Every layer a web service that can communicate with other WS

> Hence all “science” processes use network data recursive architectures– Re use and replication of same modules for software, hardware

and network for each science project

Network Workflow graph

Conclusion - 1

> The concept of customer owned networks started with the same people who first brought you the Internet – our universities and research institutions

– Most major universities and research centers have acquired their own metro and regional fiber

> Customer owned networks are becoming increasingly affordable with the advent of companies specializing in dark fiber installation and availability of low cost optics such as CWDM

> Consolidation of telcos is forcing large enterprises and systems integrators to acquire their own networks

> Prices of dark fiber are still high and do not come close to reflecting actual costs

– These prices are expected to drop in the metro area as utilities focus on providing dark fiber rather than trying to be telcos

Conclusion -2

> The Canberra ICON project demonstrates how cheap fiber should be

> The cost of long distance wavelengths is dropping dramatically and is now affordable for most large enterprises and systems integrators

> UCLP and Y.1312/1313 provides customers ability to manage their own wide area optical network integrated with their LAN

> Allows the network to be integrated with SOA architectures> Security, remote peering, supply chain management are the big

drivers for customer owned networks