13/11/14  

1  

3RD TERENA ARCHITECT WORKSHOP

Sharing of Spectrum and Alien Waves in and around SURFnet

Rob Smets – Architect Transport and Light Systems

Outline

•  London CBF and Brussels Photonic Exchange

•  Time and Frequency Transfer in SURFnet’s network

•  100G Alien waves for customers

•  Multi-domain alien-wave demand planning rules

•  Infinera waves on a Ciena light system •  Guy Roberts from GEANT Association

(Some of these slides have been presented at CEF2014 Prague)

13/11/14  

2  

London CBF / Brussels Photonic Exchange

Amsterdam – London CBF

•  Joint Collaboration between NORDUnet and SURFnet

•  Collaboration: NORDUnet leases the fiber, SURFnet installs and operates the light system. We start with two 100G waves.

•  CBF between ASD001A – ASD002A – LDN001A •  For SURFnet puts total number of CBFs to four (Geneva, Hamburg, London, Aachen) •  Puts total number of foreign PoPs to six.

•  Driven by desire to connect to services in London directly

•  Installation took place in September 2013 during ECOC 2013

•  Has been in service since December 2013. Expected doubling of 100G services in one year.

13/11/14  

3  

Amsterdam – London CBF in more detail

London (HEX) Leiston

Zandvoort

Wherstead Whickford

Amsterdam 2 Amsterdam 1

215km 40.6dB/20.3 dB

IL RAMAN (RX+TX) = 5.4dB Total IL = 46dB

Gain RAMAN: 25.7dB (Ppump=2.88W , Pseed=18mW)

78km 17dB/18.2dB

6500 WL3

Alien 79km 17.9dB/18.2dB

47km 10.3dB/13.5dB

53km 12.8dB/14.8dB

6500

W

L3

6500 WL3

Alien

10km + 10dB 2.4dB/13.5dB

Photonic Exchange in Brussels

Hamburg

London

Amsterdam

Brussels

Paris Geneva

PoP

PoP PoP

PoP

PoP

PoP

PoP

NORDUnet

SURFnet

13/11/14  

4  

Why and how?

Use cases: •  Restoration

•  Redundancy

•  For both customer and non-customer facing services

Requirements: •  Support >10Gbps

•  50GHz ITU grid / 88 channels (all DWDM channels in the C-band)

•  Switch from and to any direction

•  Local add/drop

•  All waves should be presented in a single fiber

Concept

SURFnet OSI_1

To ASD To GEN

To HB To LON

Datacenter NORDUnet

Datacenter SURFnet

WS

S

WS

S

WS

S

WS

S

Ca. 10 Km NDSF + patching: 7dB (max) Virtual BMD2 & CMD44s (Derived Adj.)

WSS-DIA

WSS-DIA

BMD2

SURFnet OSI_2

NORDUnet OSI_1

NORDUnet OSI_2

MLA

EDFA

MLA

MLA

MLA

MLA M

LA

Line fiber (0 km) + padding

CMD44

Local Add/Drop

Local Add/Drop

13/11/14  

5  

Restoration using OPS switches

6500 6500 TOADM ROADM (50 GHz)

TOADM Gr. 3

Gr. 7 Gr. 3

OTS add/drop

OTS

ROADM (50GHz)

OTS

6500/CPL

London

ROADM (50GHz)

OTS BRU001A_CPL1P

OTS DIA DIA OTS OTS

To/from Hamburg

To/from Geneva

Amsterdam 2

Brussels - NORDUnet Brussels - SURFnet

Ch. 68 OPS

OCLD

Gr. 3 OPS

OCLD

Ch. 28

Ch. 28

Ch. 28

Ch. 68

Ch. 68

Ch. 28

WSS

WSS

OTS

OPS

OCLD ASD-LON link

LON-BRU link

BRU-ASD link

OTS

add/drop

add/drop

Ch. 68

OCLD

OPS

Amsterdam 1

6500

OTS add/ drop

ROADM (50GHz)

add/ drop

Time and Frequency Transfer

13/11/14  

6  

Time and Frequency Transfer in SURFnet

•  Objective: Allow clocks to synchronize their time with an accuracy better than 500ps

•  Two approaches that allow the far end to be compensated for the offset to the middle of a loop: •  Maintain unidirectional traffic and calibrate the system to compensate for the

asymmetry due to different length of each fiber in the fiber pair •  Implement bi-directional transmission and amplification on a single fiber and only

calibrate the time difference occurring in the amplifier and in fiber due to dispersion.

•  SURFnet aims to make adjustments to the network that allows institutions to deploy White Rabbit systems beyond 10km.

Semiconductor Optical Amplifiers

LEDN001A_BIDIRAMP_01 (BDOA100B901)

1470nm

In: 1470nm Out: 1490nm

In: 1490nm Out: 1470nm

1490nm

LEDN001A_BIDIRAMP_02 (BDOA100B902)

In: 1490nm Out: 1470nm

In: 1470nm Out: 1490nm

1470nm

1490nm

13/11/14  

7  

Experimental setup over dark fiber

•  In 2013 we have started with the following set-up:

VSL-Delft (Dutch Metrology Institute) LEDN001A

5km M

S

M

S 0.5km

35.5km

14km

81km

Campus TU-Delft / DT001B

ASD002A (TC2)

ΔT

NIKHEF-Amsterdam (National Institute for Subatomic Physics)

ASD001A (SARA)

G.655

G.655 G.652

G.652 G.652

135km, mixed G.655/G.652 fiber 19.6dB

9.5dB

-24.0dBm

3.1dB

3.4dB

1.4dB

0.8dB 0.8dB

3.4dB

19dB

1dB

1dB

-17.3dBm 1.7dBm -1.0dBm

-1.0dBm

-26.7dBm -7.7dBm -24.0dBm

Findings:

•  BiDi amplifiers are placed asymmetrical in the link. •  Link engineering becomes significantly more complex in the absence of sim-tooling •  Gain of SOAs difficult to control by changing the electrical pump current •  Amplifiers produce about 19dB of gain on 1470nm and 1490nm •  Compatibility with fiber infrastructure is good.

•  There are two types of optical budget: too much and too little! •  Initial tests showed a positive budget of 0.1dB on worst link and wavelength combination •  After cleaning a positive budget of 2.0 to 3.5 dB exists •  Total budget equals: 20dB + PTX – SRX = 52 – 57 dB depending on transceiver

combinations. •  Total losses equal: 17.3dB between Delft and Leiden and 26.7dB between Amsterdam

and Leiden sites.

•  Both slaves lock •  Round trip delay of 668,981,165ps corresponds to 136km of fiber (272km round trip). •  Frequency offset of -0.3*10-13 Hz/Hz with a spread of 6*10-13 Hz/Hz over several hours. •  12 digit accuracy of the frequency already present. •  On-going work to calibrate the small difference between 1490nm and 1470nm waves. •  Measurements are ongoing. We expect to achieve time accuracy around 500ps.

13/11/14  

8  

Press release

100G Alien waves for customers

13/11/14  

9  

Extend DWDM waves into the customer’s domain requires a New Demarcation Box: Optical Gate

Institution A Institution B SURFnet

Router Router

DWDM TRX TRX TRX TRX OG OG

MGMT MGMT MGMT

Institution A Institution B SURFnet

Router Router

DWDM TRX TRX TRX TRX OG OG

MGMT Virtualization of the management function

Challenges

•  Optical plane: •  DWDM signals are transported over an infrastructure that is intended for un-

engineered links on campus •  A proper CFP!

•  Cross domain management: •  Fault Management •  Configuration Management •  Accounting •  Provisioning •  Security •  Controlled and accessible by both SURFnet and institutions

Optical Gating and Management Information Exchange Functionality may be required to:

•  Isolate and protect the DWDM network •  Monitoring •  Measure frequency •  Test photonic path between two gateways •  Exchange of management information and instructions (may be virtualized in a

datacenter)

13/11/14  

10  

Two Scenario’s

•  CPE equipment (router or switch) of both customers is the same: •  Vendor has DWDM blade in portfolio that interoperates with light system •  Vendor has DWDM blade in portfolio that does not interoperate with light system •  Vendor has no DWDM blade in portfolio but does support a CFP slot.

•  CPE equipment (router or switch) of both customers is different: •  Both CPEs have a CFP slot •  At least one of the CPEs does not have a CFP slot -> no solution

Can we find such a CFP that: •  Interoperates with SURFnet’s DWDM equipment •  Compatible with Topology of SURFnet7 and SURFnet8 •  Is affordable! (<$10,000)

ACACIA 100G Coherent CFP

What if you: •  optimize power of transmission impairment compensating ASICs; •  start using 28nm/20nm semiconductor process; •  start using Silicon Photonics Integrated Circuits and InP chips; •  pay extreme detail to power consumption and heat management?

Source: Acacia Inc.

•  DWDM single lambda solution •  50GHz C-band grid •  25ps PMD tolerance & 40 ns/nm CD tolerance •  Compatible with >1000km network solutions •  < 15dB OSNR for up to 2000km •  SD-FEC, HD-FEC

•  <24-26W •  <7.5W for DSP •  <7.5W for ADC/DAC/SERDES •  Ca. 5W for PIC •  < 10W for misc. funct.

•  24W for ZR range •  Path to smaller form factors like CFP2 •  Supported by hosts

Source: Flickr

13/11/14  

11  

Multi-domain alien-wave planning

Spectrum Sharing on Cross Border Fibers

Three important cross-border fibers

•  Amsterdam – London •  Amsterdam – Geneva

•  Amsterdam – Hamburg

Photonic Exchange in Brussels!

Photonic Exchanges in Hamburg/Geneva ?

13/11/14  

12  

SURFnet’s CBFs

•  In operation for several years using a variety of signals •  10Gbps with electronic dispersion compensation •  40Gbps waves •  40Gbps alien waves •  100Gbps waves •  100Gbps alien waves

•  GEN & HB CBFs are the CBFS we have the most knowledge on its performance and capabilities •  Non-linearity (is King!) •  OSNR performance (is King Kong!) •  Chromatic dispersion and PMD (piece of cake!) •  Operational issues (escalation matrix and testing of alarms)

•  Can we predict performance impact of an alien wave in order to validate demands end-to-end traversing multiple domains. •  With reasonable accuracy •  Without complex, time consuming and expensive high-fidelity simulations •  Using planning tooling as provided by vendor of light system

Fiber Non-Linearity & Dispersion

Non-Linearity:

•  No DWDM interfaces that use Back-Scatter-Propagation or other non-linearity compensation.

•  Stick to the linear Gaussian noise model •  SPM and XPM present themselves as additive Gaussian noise •  Neighboring channels with different modulations

•  Fixed penalty that adds to OSNR budget •  Guard channel

Dispersion:

•  Chromatic Dispersion: Linear additive

•  PMD: Sum of max. DGDs per domain.

13/11/14  

13  

OSNR validation using FoM

•  Each DWDM interface has a certain OSNR budget. •  EDFA amplification has matured.

•  Little difference in EDFA behavior of different vendors. •  Translate multiple-span link to single span, single EDFA link.

•  Each traversed domain is represented by a single span.

16dB 17dB 17dB 20dB 23dB 23dB 16dB 13dB 13dB

L=10*log10(40+50+50+100)=23.8dB

L=10*log10(200+200)=26dB

L=10*log10(40+20+20)=19dB

L=10*log10(240+400+80)=28.6dB

25.6dB 25.6dB L=10*log10(360+360)=28.6dB

Cannot validate this

Can validate this

TX

TX

TX

TX RX

RX

RX

RX

Anteneh Beshir, Roeland Nuijts, et.al. “Survivable Impairment-Aware Traffic Grooming”, Networks and Optical Communications (NOC), 2011 16th European Conference on Optical Communications Emmanuel Desurvire, “Erbium-Doped Fiber Amplifiers: Principles and Applications”

Example: ASD-GEN and ASD-HB

•  Amplifier nodes •  22 ASD-GEN •  8 ASD-HB

•  Total length •  1500km ASD-GEN •  600km ASD-HB

•  100Gbps PM-QPSK •  All demands pass!

•  1.5dB OSNR margin ASD-GEN •  6.2dB OSNR margin ASD-HB

•  ASD-GEN:

•  FoM(ASD-GEN)=1910 with 1.5dB margin: FoM(ASD-GEN,max)=2700 •  FoM(GEN-ASD)=2053 with 1.5dB margin: FoM(GEN-ASD,max)=2900

•  ASD-HB: •  FoM(ASD-HB)=720 with 6.2dB margin: FoM(ASD-HB)= 3001 •  FoM(HB-ASD)=775 with6.2dB margin: FoM(HB-ASD,max)=3230

FoM(avg)=2800

FoM(avg)=3115

Average FoM to calculate with is approx. 2960, Difference between GEN & HB CBF = 0.5dB (ROADM filters 2x)

13/11/14  

14  

Infinera waves on a Ciena light system