Upload
lilly
View
54
Download
0
Embed Size (px)
DESCRIPTION
Deutsches Forschungsnetz. Roll-out of a National Research Network - a Field Report. Spring Internet2 Member Meeting Martin Wilhelm German National Research Network DFN [email protected]. Why a new network for DFN ?. - PowerPoint PPT Presentation
Citation preview
Deutsches Forschungsnetz
Roll-out of a National Research Network -a Field Report
Spring Internet2 Member Meeting
Martin WilhelmGerman National Research Network DFN
Seite 3
• Contract with operator (TSI, subsidiary of Deutsche Telekom) ended at the end of 2005
• Market situation had changed significantly (availability of dark fiber, lower prices)
• Requirements of users had changed a lot. In particular increased demand for– high capacities between relatively few sites– improved reliability and availability– more capacity for same / lower price
Why a new network for DFN ?
Seite 4
The „Old“ Network G-WiN
G-WiN
Leipzig
Berlin
Frankfurt
Karlsruhe
Garching
Kiel
Dresden
Aachen
Regensburg
Kaiserslautern
Augsburg
Hannover
ErlangenHeidelberg
IlmenauWürzburg
Magdeburg
Oldenburg
Essen
St. Augustin
Core Nodes10 Gbit/s2,4 Gbit/s2,4 Gbit/s622 Mbit/s
Status: July 2005
Rostock
Global Upstream Hamburg
Stuttgart
Braunschweig
Marburg
Bielefeld
Göttingen
Seite 5
Pre-Tender: Market Survey
• Market Survey started in early 2004
• Constraint: No budget for investments, so:– asking for offers of DWDM-services (no
purchase of equipment)– asking for offers of rental of DF (no IRUs with
upfront payment)
• Results of Survey:– there is a market for DF in Germany (not the
case when tendering for the „old“ Network)– a number of competitors can offer DWDM
services at reasonable prices
Seite 6
Preparing Call for Tender
• The top 60 sites (by volume) were defined as potential end points for dark fiber (DF) as well as for Managed Wave Lengths (MWL), resulting in a 60 x 60 matrix
• Bidders were asked to offer options for DF and / or MWL between any of two sites
• In another lot bidders were asked to offer DWDM-services on DF provided by DFN
• In a 4th lot offers for network supervision were solicited (24 * 7 * 365)
Seite 7
Results of Tender (1)
• DFs were offered between a significant number of DFN sites at reasonable prices
• Comparing prices for MWL and cost for DF plus DWDM-services showed in many cases economic benefits for DF-solution from 2nd wavelength on (surprise, surprise ...)
• For a few areas there was either no offer for DF or offers were economically not reasonable (the German DF badlands ;-))
Seite 8
Results of Tender (2)
• As a nice side effect, offers of DF could be arranged to build a number of rings. In total X-WiN now spans across approx. 5,500 fiber kilometers (approx. 3440 miles)
• These rings then were complemented by a number of MWLs
• Increased number of nodes reduced number of access lines cost savings
Seite 9As of 18.01.2006
X-WiN-Topology(Fibers and Managed Wavelengths)
Fiber KPN
Fiber GL
Fiber GC
Fiber DFN
Managed WL
ESF46 Core Sites (+ 3 CBF)
•31 connected by Fiber
•10 connected by Managed WL
• 9 connected by fiber and MWL
KOP
ENS
STB
BAY
BIR
GSI
WUE
GOEKAS
MAR
GIE
GRE
GAR
ERL
MUE
FZJ
AAC
HAM
DES
POTTUB
FZK
DUI
BRE
HAN
BRA MAGBIE
FRA
HEI
STU
REG
DRE
CHE
ZIB
ILM
KIE
ROS
LEI
JEN
HUB
ADH
EWE
AUG
FFO
AWI
SAA
KEH
Richtung Basel
BAS
Seite 10
Another topological view
TUB
POT
ADHHUB
MAG
BRA
BAY
ILM
LEICHE
DRE
JEN
FRA
BIR
FZJ
AAC
DUI
MUE
BIE
HAN
BRE DESHAM
ROS
REG
GAR
STUFZKHEI
GSI
27
39
32
69.7
168.5
62.68
121.73
132.30
274.42155,2060.50
91.00
69.3
188.98
138.10
55.00
268.8
141.6
145.0
139.35
156.9
196.0010.00
249.00
380.11
107.0
76.88
179.70
102.2
120.20
144.9
104.3
300.24
16
138.00
230.4
KIE
B
THU
E
F
G
I
J
K
L
M
N
O
P
Q
R
S
T U V
W
X
Y Z
AA
AC
AD
AL
AM
AE
AFAG
AIAJ
AK
ZIB
ESFL-S1
79.71
A
CAH
ERL
D1
D2
265.58
158.30
H
FFO
AWI
operational
Under construction
AN AO115.00
132
Kehl-S2102
154.00AP
BAS
Seite 11
DWDM Equipment
• Contract (provision of DWDM-services) awarded to Huawei
• Includes installation, operation and maintenance of DWDM equipment (carrier grade DWDM) as well as amplification (OLA)
• Contract defines SLAs (and penalties) and obligation for pro-active 24*7 monitoring
• Access to performance data (read-only) via open non-proprietary interface is part of contract
Seite 12
Wavelength Planning
• Task 1: Substitute G-WiN by X-WiN as hassle-free as possible (minimize reconfigu-ration and relocation of existing routers)– done by (almost) 1:1 copy of existing SDH-links
to wavelengths (STM-16, STM-64)
• Task 2: Integrate new DFN sites– this required a number of new routers. Some
reconfiguration could not be avoided
• Task 3: Optimise wavelength plan after the dust has settled (current work)
Seite 13
Roll Out Stage 1
• Co-ordination of providers (DF and DWDM)– exchange of fiber info (lengths of fibre spans,
attenuation of spans, location and access regulations of amplifier huts, contact details as phone, fax, e-mail of hotlines, ...)
• Preparation of DFN-sites– creating space for additional equipment (DWDM)
at „old“ sites– preparing grounds (sometimes from scratch) in
„new“ sites: installation of cabinets, power, UPS, A/C, patch fields, out-of-band access, ...
Seite 14
Roll Out Stage 2
• Verification of WL routing– Optimal WL routes proposed by Huawei experts– Cross-checked by DFN and verified against IP
routing
• Building of the (in)famous last mile– Almost all DFN-sites are on university / research
lab campuses. All (but two) sites have dual connections. Cross-check that all fiber routes are built with full redundancy (sometimes difficult because of unadequate infrastructure or stubborn administrations)
Seite 15
Roll Out Stage 3
• Installation of Optical Line Amplifiers (OLA)– Early installation was possible, because all long
distance spans of fibers as well as amplifier huts were already in place
• Installation of DWDM equipment at DFN sites
Seite 16
Some Installation Impressions
Seite 17
More Impressions
Seite 18
Even more Impressions
Seite 19
Roll Out Stage 4
• Link Commissioning– five teams working in parallel to set up basic
interworking between adjacent DWDM locations. Since in most cases OLAs were required, this was done step by step: DWDM A -> OLA1 -> OLA2 -> DWDM B -> OLA3 -> DWDM C, then back to set up the other direction.
– Detailed planning was necessary to minimize travel time between sites (problem of traveling sales man ;-)).
– Average fuel consumption per working day: approx. 70 l per team
Seite 20
Roll Out Stage 5
• Service Commissioning– again five teams in parallel working to set up
end-to-end wavelengths including the digital interfaces (STM-64, STM-16, 10 GE, 1 GE), fine tuning of lasers etc.
– again quite a bit of travel was involved, although remote set up and calibration was possible at this stage
Seite 21
Roll Out Stage 6
• Switch over from „old“ G-WiN SDH-links to „new“ X-WiN wavelengths
• The first approx. 10 links could be set up in parallel to existing SDH-links. This worked well. Good for building confidence ;-)
• Since interfaces at all of our routers could not simply be doubled, most of the core links had to be „hot swopped“ (no old links in parallel standby). Some nervosity could be noticed ...
Seite 22
Timeframe was a bit tight ...
• The only thing we were sure of was the cut off date for G-WiN: 31 December 2005 ...
• Fortunately, remarkably few things went wrong:– 4 or 5 wrong patches, a few faulty patch cables,
three faulty boards, one weak laser etc etc
• Last G-WiN link switched off on 29/12/05
• A German proverb says: „Luck is with the competent“ :-)
• The following slides show how time went by
Seite 23
FFO
GIE
GAR
ERL
BAY
MUE
FZJ
AAC BIR
HAM
DES
POTTUB
FZK
GSI
DUI
BRE
HAN
BRA MAGBIE
FRA
HEI
STU
REG
DRE
CHE
ZIB
ILM
KIE
ROS
LEI
JEN
ESF
HUB
ADH
AWI
GOEKAS
MAR
GRE
WUE
AUG
SAA
DF delivered
DF ordered
MWL ordered
MWL delivered
DFN-Site
Site infrastructure ready
DWDM installed
Roll out X-WiN (15/09/05)
KEH
BAS
Seite 24
FFO
GIE
GAR
ERL
BAY
MUE
FZJ
AAC BIR
HAM
DES
POTTUB
FZK
GSI
DUI
BRE
HAN
BRA MAGBIE
FRA
HEI
STU
REG
DRE
CHE
ZIB
ILM
KIE
ROS
LEI
JEN
ESF
HUB
ADH
AWI
GOEKAS
MAR
GRE
WUE
AUG
SAA
DF delivered
DF ordered
MWL ordered
MWL delivered
DFN-Site
Site infrastructure ready
DWDM installed
Roll out X-WiN (26/09/05)
KEH
BAS
Seite 25
FFO
GIE
GAR
ERL
BAY
MUE
FZJ
AAC BIR
HAM
DES
POTTUB
FZK
GSI
DUI
BRE
HAN
BRA MAGBIE
FRA
HEI
STU
REG
DRE
CHE
ZIB
ILM
KIE
ROS
LEI
JEN
ESF
HUB
ADH
AWI
GOEKAS
MAR
GRE
WUE
AUG
SAA
KEH
BAS
Roll out X-WiN (29/09/05)
DF delivered
DF ordered
MWL ordered
MWL delivered
DFN-Site
Site infrastructure ready
DWDM installed
Seite 26
FFO
GIE
GAR
ERL
BAY
MUE
FZJ
AAC BIR
HAM
DES
POTTUB
FZK
GSI
DUI
BRE
HAN
BRA MAGBIE
FRA
HEI
STU
REG
DRE
CHE
ZIB
ILM
KIE
ROS
LEI
JEN
ESF
HUB
ADH
AWI
GOEKAS
MAR
GRE
WUE
AUG
SAA
KEH
BAS
Roll out X-WiN (02/10/05)
DF delivered
DF ordered
MWL ordered
MWL delivered
DFN-Site
Site infrastructure ready
DWDM installed
Seite 27
FFO
GIE
GAR
ERL
BAY
MUE
FZJ
AAC BIR
HAM
DES
POTTUB
FZK
GSI
DUI
BRE
HAN
BRA MAGBIE
FRA
HEI
STU
REG
DRE
CHE
ZIB
ILM
KIE
ROS
LEI
JEN
ESF
HUB
ADH
AWI
GOEKAS
MAR
GRE
WUE
AUG
SAA
KEH
BAS
Roll out X-WiN (13/10/05)
DF delivered
DF ordered
MWL ordered
MWL delivered
DFN-Site
Site infrastructure ready
DWDM installed
Seite 28
EWEFFO
GIE
GAR
ERL
BAY
MUE
FZJ
AAC BIR
HAM
DES
POTTUB
FZK
GSI
DUI
BRE
HAN
BRA MAGBIE
FRA
HEI
STU
REG
DRE
CHE
ZIB
ILM
KIE
ROS
LEI
JEN
ESF
HUB
ADH
AWI
GOEKAS
MAR
GRE
WUE
AUG
SAA
KEH
BAS
Roll out X-WiN (31/10/05)
DF delivered
DF ordered
MWL ordered
MWL delivered
DFN-Site
Site infrastructure ready
DWDM installed
Seite 29
Roll out X-WiN (07/11/05)
DF delivered
DF ordered
MWL ordered
MWL delivered
DFN-Site
Site infrastructure ready
DWDM installed
EWEFFO
GIE
GAR
ERL
BAY
MUE
FZJ
AAC BIR
HAM
DES
POTTUB
FZK
GSI
DUI
BRE
HAN
BRA MAGBIE
FRA
HEI
STU
REG
DRE
CHE
ZIB
ILM
KIE
ROS
LEI
JEN
ESF
HUB
ADH
AWI
GOEKAS
MAR
GRE
WUE
AUG
SAA
KEH
BAS
KOP
Seite 30
EWEFFO
GIE
GAR
ERL
BAY
MUE
FZJ
AAC BIR
HAM
DES
POTTUB
FZK
GSI
DUI
BRE
HAN
BRA MAGBIE
FRA
HEI
STU
REG
DRE
CHE
ZIB
ILM
KIE
ROS
LEI
JEN
ESF
HUB
ADH
AWI
GOEKAS
MAR
GRE
WUE
AUG
SAA
KEH
BAS
KOPRoll out X-WiN (18/11/05)
DF delivered
DF ordered
MWL ordered
MWL delivered
DFN-Site
Site infrastructure ready
DWDM installed
Seite 31
EWEFFO
GIE
GAR
ERL
BAY
MUE
FZJ
BIR
HAM
DES
POTTUB
FZK
GSI
DUI
BRE
HAN
BRA MAGBIE
FRA
HEI
STU
REG
DRE
CHE
ZIB
ILM
KIE
ROS
LEI
JEN
ESF
HUB
ADH
AWI
GOEKAS
MAR
GRE
WUE
AUG
SAA
KEH
BAS
KOPRoll out X-WiN (21/11/05)
DF delivered
DF ordered
MWL ordered
MWL delivered
DFN-Site
Site infrastructure ready
DWDM installed
Seite 32
Roll out X-WiN (30/11/05)
DF delivered
DF ordered
MWL ordered
MWL delivered
DFN-Site
Site infrastructure ready
DWDM installed
EWEFFO
GIE
GAR
ERL
BAY
MUE
FZJ
AAC BIR
HAM
DES
POTTUB
FZK
GSI
DUI
BRE
HAN
BRA MAGBIE
FRA
HEI
STU
REG
DRE
CHE
ZIB
ILM
KIE
ROS
LEI
JEN
ESF
HUB
ADH
AWI
GOEKAS
MAR
GRE
WUE
AUG
SAA
KEH
BAS
KOP
Seite 33
EWEFFO
GIE
GAR
ERL
BAY
MUE
FZJ
BIR
HAM
DES
POTTUB
FZK
GSI
DUI
BRE
HAN
BRA MAGBIE
FRA
HEI
STU
REG
DRE
CHE
ZIB
ILM
KIE
ROS
LEI
JEN
ESF
HUB
ADH
AWI
GOEKAS
MAR
GRE
WUE
AUG
SAA
KEH
BAS
KOPRoll out X-WiN (05/12/05)
DF delivered
DF ordered
MWL ordered
MWL delivered
DFN-Site
Site infrastructure ready
DWDM installed
Seite 34
EWEFFO
GIE
GAR
ERL
BAY
MUE
FZJ
BIR
HAM
DES
POTTUB
FZK
GSI
DUI
BRE
HAN
BRA MAGBIE
FRA
HEI
STU
REG
DRE
CHE
ZIB
ILM
KIE
ROS
LEI
JEN
ESF
HUB
ADH
AWI
GOEKAS
MAR
GRE
WUE
AUG
SAA
KEH
BAS
KOPRoll out X-WiN (14/12/05)
DF delivered
DF ordered
MWL ordered
MWL delivered
DFN-Site
Site infrastructure ready
DWDM installed
Seite 35
EWEFFO
GIE
GAR
ERL
BAY
MUE
FZJ
AAC BIR
HAM
DES
POTTUB
FZK
GSI
DUI
BRE
HAN
BRA MAGBIE
FRA
HEI
STU
REG
DRE
CHE
ZIB
ILM
KIE
ROS
LEI
JEN
ESF
HUB
ADH
AWI
GOEKAS
MAR
GRE
WUE
AUG
SAA
KEH
BAS
KOPRoll out X-WiN (19/12/05)
DF delivered
DF ordered
MWL ordered
MWL delivered
DFN-Site
Site infrastructure ready
DWDM installed
Seite 36
Today:
• Everything now on X-WiN
• First links of old network were cut off on 2 January 2006
• No operational issues so far ...
Seite 37
Cross Border Fibers
FFO
KIE
HUB
ADH
EWE
GIE
GAR
ERL
BAY
MUE
FZJ
AAC BIR
HAM
DES
POTTUB
FZK
GSI
DUI
BRE
HAN
BRA MAGBIE
FRA
HEI
STU
REG
DRE
CHE
ZIB
ILM
ROS
LEI
JEN
ESF
AWI
GOEKAS
MAR
GRE
WUE
AUG
SAA
KEH
PSNC
Switch/GARR
Renater
CPH
(PL)
(NL)
(FR)
CBFDFN PoPDark fibreWavelength
Surfnet
Seite 38
The Idea behind CBFs
• Connectivity between NRENs usually via GÉANT
• Cross Border Fibers complement existing GÉANT infrastructure and can add resilience
• CBFs have a big potential for savings
• Will be part of European infrastructure - not a testbed !
• Challenge: Operational issues. Requires definition of technical, operational, organisational and financial framework
Seite 39
Questions ...?
? ??