SuperJANET5, an overview - London South Bank University · SuperJANET5, an overview Roland Trice ... Ciena OADM Operation ... Ciena OADM Operation Deploy Ciena Metro CN4200 with

February 2006 SuperJANET5 Overview 1

SuperJANET5, an overview

Roland Trice

SuperJANET 5 Project

[email protected]


Introduction

• What we said we wanted

• What we’ve procured

• Infrastructure

• Routers

• What are we going to do now?

• Deployment

• Migration


Past and Present Challenges

• SuperJANET4

• Network capacity increased by two orders of magnitude

• Strengthened design and management to meet increase in user base

• Still a “one size fits all” network

• SuperJANET5

• End-to-end network delivery and reliability to the end user

• Meeting the needs of an increasingly diverse user base that has a very broad spectrum of requirements

• Development vs production stability


Requirements Analysis

• Q2/3 2003

• Identified a number of areas for study

• Architecture and Services at Multiple Network Layers

• Network Infrastructure

• Network Equipment

• Reliability

• Wider consultation

• Requirements analysis paper produced & issued

• Community workshops

• International comparisons


Consultation Carried Out

• Requirements analysis paper

• Requirements analysis launch event

• RNO workshop

• Research sector workshop

• Reliability questionnaire

• Articles in:

• UKERNA News; ALT Newsletter; JISC Inform

• Discussions with key stakeholder groups:

• UCISA; e-Science; FERL; Regional Networks


Issues raised

• Network reliability

• Latent bandwidth demand

• Protection of interests of the teaching & learning and research communities

• Participation of Regional Networks in delivery to institutions

• Network monitoring services to applications


Translate Simply to….

• RELIABILITY

• SCALABILITY

• SEPARABILITY

• FLEXIBILITY

• VISIBILITY

END-TO-END


An architecture that responds to the requirements

Flexible transmission platform

IP production network

Special purposebandwidth

Testbed(s)

Commodity e-ScienceNetwork R&D

Service Dev.

Testbed(s)

Requirements to be served


Multiple Services

• IP Production Network• Ubiquitous network

• High availability and reliability

• Simple engineering for reliability

• Special Purpose Bandwidth• User requirements not compatible today with the IP

production service

• Point-to-point circuits

• Reliable, Possibly to IP production standards

• Testbeds• ‘Breakable’ networks


Why multiple services

• Reliability

• Stability

• Strict change control

• Don’t fiddle

• Don’t introduce bugs

• Development & Flexibility

• Change

• New software (free bugs)

• Takes a very long time if you want to minimise disruption


“You’re never alone with Schizophrenia”

• Very high levels of reliability and the development of advanced services are mutually exclusive over a one-size-fits-all IP service

• Multiple service streams are essential


Flexible Transmission PlatformDesign characteristics

• Transmission channels configured and managed at our control

• Allows multiple service offerings from a single coherent network infrastructure

• Scalable to higher bandwidth at a predictable and manageable cost


Routers “The Next Generation”

• Carrier class reliability

• “Five Nines” availability (99.999% )

• Less than 5.5 minutes of down time a year

• Building block for a better overall service

• Does NOT mean that JANET becomes 99.999% available

• Other problems will almost certainly ensure this


How to achieve “Five Nines”

• Modular h/w and s/w architecture

• Reduced chance of adding bugs

• Non-intrusive upgrades

• Automatic failure detection and rapid recovery

• Eliminate single points of failure in the router

• Partitioned system offering multiple “virtual”routers per platform

• May enable multiple IP service streams in a single chassis


What we’ve procured


Infrastructure

• Negotiated Procedure under public procurement regulations

• We know what we want but not how to get it

• Pre-qualification

• Q4 2004

• Six suppliers short-listed

• Negotiate form of ITT with all six

• Q1 and 2 of 2005


Infrastructure

• Produce and launch ITT

• June 2005

• Only to short-listed suppliers

• Award Contract December 2005

• MCI (now Verizon) the preferred supplier


Routers

• RFI process

• Informal talks with suppliers in Q3/4 2003

• RFI document issues in early 2004

• Evaluating features of carrier class routers

• NOT a procurement

• Hands-on testing of

• Chiaro Enstara, Cisco CRS-1, Juniper T-640, Procket PRO 8000


Routers

• Open procurement launched in Sept 2005

• Contract signed with Lucent Technology

• This morning

• Juniper T-640 routers

• Carrier Class

• >1000 in service world wide

• 40 Gbit/s demonstrated by Juniper and Verizon

• Available as a product now

• Very good deal, Driving down the cost of ownership


Juniper T-640


What does this mean in practice?

• Fibre infrastructure dedicated to SuperJANET5

• Single pair of fibres

• Unprotected

• Dedicated transmission infrastructure

• Operated by Verizon initially

• UKERNA may manage wavelength provision later in the project

• WDM technology

• Deployed in the core and out to the Region


Core Network Design

• Core Terminal sites – Six blue locations

• Flexible OADM sites (Optical Add Drop) – Eight white locations

• Optical Amplification sites –Eleven locations, not shown

• Core Network consists of three basic components:

• Let’s examine this span further….


Ciena CoreStream Regional


Ciena CoreStream Regional

NetworkManagement

Bristol

(Terminal)London(Terminal)

Devizes(Optical

Amplifier)

OpticalAmplifier

OpticalAmplifier

BasingstokeOptical Add/Drop

Multiplexer

OADM: C-OADMOADM: C-OADMIntegrated Line Amplifier: ILAIntegrated Line Amplifier: ILA


Core Traffic Provision

• 10G Wavelengths are provisioned using two transponders only

• No Regeneration needed

• Each wavelength has to be regenerated separately

• Optical amplification still needed every 80 KM or so



• 10G Wavelengths are provisioned using two transponders only

• IP Production Traffic: 1 X 10G between Terminal locations



• 10G Wavelengths are provisioned using two transponders only (no Regen)


• Diverse 1 x 10G mesh for London Principal External Interconnects (PEI)






• Development traffic – identical to IP network with 1 x 10G between Terminal locations, except…..






• Development traffic – identical to IP network with 1 x 10G between Terminal locations, except…..

• …. different configuration of 2x10G to London PEI


Ciena OADM Operation

• Ciena supports 80 x 10G wavelengths between Terminals

• OADM drops/adds one Band of 8 x 10G wavelengths

• 80 Wavelengths grouped into 10 Bands of 8 Channels per Band

• Example: Preston has access to 8 Wavelengths North and 8 Wavelengths South

Require Optics compatible with CoreStream product


Ciena OADM Operation

Deploy Ciena Metro CN4200 with compatible Optics with Backbone


Ciena CN4200 Platform


Regional Network Design

• Linear chain configurations

• Networks are either carried over fibre, or connected into Core Network at OADM sites

Regional Network Carried over CoreStream OADM System


RA1

RA2

RA3/1 (74km)

RA3/2 (81km)RA4

RA5

RB1/1 (83km)

RB6/1(46km)

RC1

RC2

RD1

RD2

RD3 RD4

RD5

RE1/1

RE1/2

RK1

RK2

4. EastMAN

11.MidMAN

6. EMMAN

Nottingham

Edinburgh Uni

13. NNW

ManchesterLeeds

M1 L1

19.YHMAN

P2

C2

L1

3. C&NL MAN

14. NorMAN

MCI

Newcastle

MCI Carlisle

MCI Preston

MCI

Middlesbrough

Newcastle

Glasgow Uni

Carlisle Uni

Lancaster Uni

G2

G1

N2

N1

N1

N2

Preston

Newcastle

HEANET

AMP Sheffield

AMP Thirsk

AMPChesterfield

AMPNorton Bridge

AMP Oxenholme

AMP Sanquar

AMP

Falkirk L1L2 E1E2

D1

AMP

AMP Greenlaw

Alnwick

AMPAMP

SwintonLytham

MCI Dublin

AMP

Newry

Renfew

Belfast Shore Road

Belfast Kennedy Way

Warrington

C-PoP

Leeds

C-PoP

Glasgow

C-PoP C1

C2/1

C3

C4/1

C6

C7/1

C8/1

C5

C9/1

RN1/2

C2/3

C2/2

C4/2

C7/2

C8/2

C9/2

C15/2

AMP Glenrothes

RB6/2 (58km)

L2

RL1

RL2MCI

Edinburgh

Scolocate

Edinburgh

RN1/1

RN1/3

RN1/4

RN2/2

RN2/3

RN2/5RN2/4 RN2/6

RN1/5


Regional Network Design

Regional Network Carried over collector arc


RA1

RA5

RB6/1(46km)

RC1

RC2

RD1

RD2

RD3 RD4

RD5

RE1/1

RE1/2

RE2

RK1

RK2

11.MidMAN

6. EMMAN

B1

Nottingham

Edinburgh Uni

Birmingham Uni

13. NNW

ManchesterLeeds

M1 L1

19.YHMAN

P2

C2

L1

3. C&NL MAN

14. NorMAN

MCI

Newcastle

MCI Carlisle

MCI Preston

MCI

Middlesbrough

Newcastle

Carlisle Uni

Lancaster Uni

G1

MCI Leicester

N2

N1

N1

N2

Preston

Newcastle

AMP Sheffield

AMP Thirsk

AMPChesterfield

AMPNorton Bridge

AMP Oxenholme

AMP Sanquar

Falkirk L1L2 E1E2

AMP

AMP Greenlaw

Alnwick

AMPAMP

SwintonLytham

MCI Dublin

AMP

Newry

Renfew

Belfast Shore Road

Belfast Kennedy Way

Warrington

C-PoP

Leeds

C-PoP

Glasgow

C-PoP C1

C2/1

C3

C4/1

C6

C7/1

C8/1

C5

C9/1

C15/1

RN1/2

C2/3

C2/2

C4/2

C7/2

C8/2

C9/2

C15/2

AMP Glenrothes

RB6/2 (58km)

L2

RL1

RL2MCI

Edinburgh

Scolocate

Edinburgh

RN1/1

RN2/5RN2/4 RN2/6


RA1

RA2

RA3/1 (74km)

RA3/2 (81km)RA4

RA5

RB1/1 (83km)

RB2/1 (40km)

RB2/3 (38km)

RB3

(30km)

RB4/2 (60km)

RB4/1 (67km)

RB5 (1km)

RB6/1(46km)

RC1

RC2

RD1

RD2

RD3 RD4

RD5

RE1/1

RE1/2

RE2

RK1

RK2

17. UHI

12. NIRAN

Perth

4. EastMAN

11.MidMAN

6. EMMAN

Nottingham

Edinburgh Uni

Birmingham Uni

13. NNW

ManchesterLeeds

M1 L1

19.YHMAN

P2

C2

L1

2. ClydeNET

3. C&NL MAN

14. NorMAN

MCI

Newcastle

MCI Carlisle

MCI Preston

MCI

Middlesbrough

Newcastle

Glasgow Uni

Carlisle Uni

Lancaster Uni

G2

G1

N2

N1

N1

Inverness

N2

Preston

I1

Newcastle

HEANET

STM-1

Key

Core Network

Regional Network

AMP Sheffield

AMP Thirsk

AMPChesterfield

AMPNorton Bridge

AMP Oxenholme

AMP Sanquar

AMP

Falkirk

AMP

1. AbMAN

A2 A1

7. FatMANDundee Uni

L1L2 E1E2

D1

D2

AMP

AMP

AMP Greenlaw

Alnwick

Brechin

AMPAMP

SwintonLytham

MCI Dublin

AMP

Newry

Renfew

Belfast Shore Road

Belfast Kennedy Way

Warrington

C-PoP

Leeds

C-PoP

Glasgow

C-PoP

Charleston

C1

C2/1

C3

C4/1

C6

C7/1

C8/1

C5

C9/1

RN1/2

C2/3

C2/2

C4/2

C7/2

C8/2

C9/2

C15/2

AMP

Braco

P2

AMP

Fiddes

AMP Glenrothes

RB6/2 (58km)

RB2/2 (80km)

RB1/2 (47km)

L2

RL1

RL2MCI

Edinburgh

ScolocateEdinburgh

RN1/1

RN1/3

RN1/4

RN2/2

RN2/3

RN2/5RN2/4 RN2/6

RN1/5

RN2/1

Belfast Uni

B1

B2

HEA

HEANET

AMP

Dublin


RD1

RD2

RD3 RD4

RD5

RE1/1

RE1/2

RE2

RF1/1 RF1/2

RG1

RG2

RH2/1

RH2/2

RH2/3

RH3

RH1

RI1 RI3 RI4

RJ1RJ2

RJ3

RJ4

RJ5/2

RJ5/1

RJ5/3

RJ5/4

RK1

RM2/1

RM2/2

Warwick Uni

8. KentishMAN

Chatham

W2

11.MidMAN

6. EMMAN

10. LMN

B1

Nottingham

Birmingham Uni

13. NNW

ManchesterLeeds

M1 L1

19.YHMAN

P2

L1

3. C&NL MAN

15. SWERN

18. WalesMAN

16. TVN

MCI Preston

Bristol

Cardiff

C1

C2

5. EastNet

Cambridge

MCI Leicester

N2

C1

MCI Portsmouth

MCI

Basingstoke

S1

9. LeNSE

Southampton

FarnhamF2

MCI Birmingham

Chelmsford

C1

N1

C2

Canterbury

TVN2 in Reading MCI

Preston

HEANET

Telecity

Harbour

Exchange

AMP

AMP

AMP

Newport

Gloucester

Evesham

AMPClifton

AMP Newbury

AMP

Devizes

AMP

AMP

Croydon

Brighton

AMP

Farnborough

AMP

Maidstone

AMP

Sevenoaks

Milton Keynes

AMP Sheffield

AMP Thirsk

AMP

Peterborough

AMPChesterfield

AMPNorton Bridge

AMPAMP

SwintonLytham

MCI Dublin

Cardiff

Bristol

C-PoP

Reading

C-PoP

Warrington

C-PoP

Leeds

C-PoP

Project:

UKERNA SJ5 Network (Schematic)

Author:

Kevin Hayden

C4/1

C6

C5

C9/1

C10/1

C11/1

C12/1C13/1

C14/1

C15/1

C16

C17

C18

C4/2

C9/2

C10/2

C11/2

C12/2

C13/2

C14/2

C15/2

RF1

Version:

1.0 (06.02.06)

University College

LondonNeos

R1 L1L2

Reading

C PoP

Ch2

RM1/1

RM1/2

Telehouse

North

London

UK5

C PoPImperial College

London

AMP

B2 B1

L2

RN2/2

RN2/3

RN2/5RN2/4 RN2/6

RN2/1

HEA

HEANET

Dublin


Reliability

• Reduce single points of failure

• Duplication of critical equipment

• Diverse fibre routing in the core

• Dual connection of Regional Networks

• SDH technology

• Rapid failure detection and recovery

• Detect latent problems


Reliability

• Reduce complexity

• Simplify IP design

• Reduce number of backbone routers

• Carrier class equipment

• Ciena optical technology

• Juniper T-series routers

• High availability


Scalability

• Ability to expand capacity

• Where it’s needed

• When it’s needed

• 40 Gbit/s trial in early 2007

• Predictable cost

• Marginal cost for transmission channels

• Significant discount for router equipment

• Not clawed back in maintenance charges


Separability

• Hard multiplexing of bandwidth channels

• Production services completely protected

• Banding of channels could even allow GMPLS experiments over the core


Flexibility

• Deliver bandwidth quickly

• In weeks rather than months

• P6 is still necessary

• With a variety of interfaces and bandwidths

• 10 and 2.5 Gbit/s SDH

• 10 and 1 GE (LAN-PHY and WAN-PHY)

• Temporary arrangements may be practical


Visibility

• Requirement to monitor network in more detail

• Operational monitoring

• Research projects

• Optical taps


What are we going to do now?


SJ5 delivery Timetable

• Core links RFS by 30th April 2006

• Includes UKERNA acceptance testing

• Telehouse & TeleCity part of the core

• UKERNA Acceptance testing

• Agilent network testers

• Loops in the right places

• Links are ready to go when routers delivered



• Test router to ULCC in March

• Use to build initial configurations

• Familiarisation for the NOSC

• Enhances training courses March/April

• Core routers RFS by 31st May

• Pre-staging by Lucent to eliminate DOA hardware

• Pre-configured, plug-and-go



• All RN links RFS by 31st June

• Expect some before this date

• Acceptance using Agilents or Core Routers

• Connection of SJ4 to SJ5 at Docklands mid June

• Migration of RNs will begin ASAP

• Non regional services have to migrate as well


Arrangements for migrating

Cease resilient links between Docklands and SJ4 core

• Will be done in early June

• Connect SJ4 to SJ5 at Docklands

• Will be done in mid June



• Migrate sites connected to Cosham, Bristol, Glasgow and Edinburgh C-PoPs 1st

• Reduce ongoing router maintenance costs

• Free up 10 Gbit/s equipment for re-deployment

• Cross connect links to bypass old core routers while maintaining SJ4 resilience

• Will then move RNs from remaining C-PoPs



• Avoid migrating during clearing

• An ambition, NOT a promise

• EastNET will probably have to �

• EastNET must connect by end of September

• Existing ntl contract terminates on the 30th


How’s it going?

• Very well thank you……….

• University of Kent, Canterbury

• Longest SJ5 dig

• From the A2 into the campus

• Through the mating ground of…………..


The Great Crested Newt

• Verizon talking to DEFRA and the world expert

• Who happens to work at UKC


Gone for, and GOT!

• RELIABILITY

• SCALABILITY

• SEPARABILITY

• FLEXIBILITY

• VISIBILITY

END-TO-END


Questions?

Documents

SuperJANET5, an overview - London South Bank University · SuperJANET5, an overview Roland Trice ... Ciena OADM Operation ... Ciena OADM Operation Deploy Ciena Metro CN4200 with