100 Gigabit Ethernet

© 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential100G LACNOG 2010

Igor Giangrossi

Consulting Systems Engineer

100G Deployment Considerations

© 2010 Cisco Systems, Inc. All rights reserved. Cisco Confidential100G LACNOG 2010 2

Agenda

The need for 100GE

Standards Work Status

100G and Optical Transport Systems

100GE Router Implementations


0

10

20

30

40

50

60

70

2009 2010 2011 2012 2013 2014

Ex

ab

yte

s/m

o

Business

Consumer

34% CAGR 2009–2014

87%

13%

Global IP Traffic Growth

IP Traffic will increase 4.3x from 2009 to 2014

Source: Cisco Visual Networking Index (VNI) Global Forecast, 2009–2014


Why Even Higher Speed Interfaces?

Current Interfaces:

10Gbps: Ethernet

40Gbps: SONET/SDH

Ethernet Link Aggregation

Max 8 links (Std)

Unequal Load Balance

Limits flow BW

Harder to manage

Inefficient fiber use


Different SDOs for 100Gb/s Interfaces

Defined Ethernet MAC and PHY

specifications

Defining 100Gb/s signals over 50GHz

optical channels on DWDM systems

(modulation, etc)

Defining how to map 100Gb/s signals

over OTN/G.709 encapsulation


IEEE 802.3ba Standard

Ratified on IEEE July 2010 Meeting

40 and 100 Gb/s Ethernet MAC and PHY

Cooper and fiber connectivity


Why 40G and 100G Ethernet?Addressing Needs of Core and Access Network

2000 2005 2010 2015 20201995

100

1000

10 000

100 000

1 000 000 Core Networking

Doubling ≈ 18 months

Server I/O

Doubling ≈ 24 months

Gigabit Ethernet

10G Ethernet

100G Ethernet

40G Ethernet

100G forecast to be needed in the core in 2010

Server I/O will not require 100G for several years

- Will require 40G much earlier

- Growth follows Moore’s Law

Bandw

idth

(M

b/s

)


IEEE 803.2ba Summary

40 GbE 100GbE Comments

1 m Backplane 40GBASE-KR4

7 m Copper 40GBASE-CR4 100GBASE-CR4 4/10 cooper @ 10Gb/s

100 m MM Fiber 40GBASE-SR4 100GBASE-SR10 4/10 fibers @ 10Gb/s

10 km SM Fiber 40GBASE-LR4 100GBASE-LR4 CWDM (4λ)

40 km SM Fiber 100GBASE-ER4 CWDM (4λ)

Additional Info:

- 802.3 Frame format is maintained

- 64B/66B Encoding: 40GBASE-R (40Gb/s) and 100GBASE-R (100Gb/s)

- Cooper and backplane PHYs have an optional FEC layer


ITU Work on 40/100G (SG15 – OTN)

Wide Area Network

OTU1

(2.7Gb/s)

OTU2

(10.7Gb/s)

OTU3

(43Gb/s)

G.709: OTN Network Node Interface (NNI)

• Client interfaces specified for SDH services (STM-16, STM-64 and STM-256)

• OTU3 supports STM-256; proposed for 40GbE

• OTU4 (>100Gb/s) proposed for 100GbE

OTU4

(100Gb/s)

OTU4


Optical Characteristics for 100G/40G

100G vs 10G 100G vs 40G

OSNR Requirement 10 dB higher 4 dB higher

CD Tolerance 100 X less 6.25 X less

DGD Tolerance 10 X less 2.5 X less

PMD Limited

Distance100 X less 6.25 X less

Optical Bandwidth 10 X 2.5 X

Implementing 100G and 40G is much harder than

10G in the Optical Domain


100G Optical Systems

Transmitter: Increasing speed means

Complex Optics = Complex Electronics = $$$$$$

More Optical Impairments

Receiver: Address impairments

Optical compensation vs. Electrical compensation

Coherent vs. Direct detection

Forward Error Correction (FEC)

Hard Decision FEC

Standard FEC – 6dB Coding Gain

Enhanced FEC – 8+dB Coding Gain

Newer FECs – 9+dB Coding Gains

Soft Decision FEC

Each of the above FECs can be used for any of the options


100G Work at the OIF

Several Projects for 100G DWDM Systems

100G Long Distance DWDM Transmission Framework

100G Long Distance DWDM Integrated Photonics Receiver

100G Long Distance DWDM Integrated Photonics Transmitter

Forward Error Correction (FEC) for 100G DP-QPSK Long Distance Communication

100G Long-Haul DWDM Transmission Module -Electromechanical

100G Long-Haul DWDM Transmission Module – MDI

Agreed on PM-QPSK* as the modulation scheme, using coherent receivers

* Polarization Multiplexing – Quadrature Phase Shift Keying


100G – The Transmitter

Need to go slower

Optical impairments are directly related to signaling rates

Need to increase modulation efficiency

Signaling speed decreases & Information Rate increases

NRZ to ODB to (D)PSK to (D)QPSK

Need to increase optical efficiency

Split signal over two polarizations (PM – Mod Scheme)

1 bit/symbol

NRZ

0 1

1 bit/symbol

PSK

1-1

2 bits/symbol

QPSK

00

1011

01


6 9 12 15 18 21 24 27 300.5

1

2

4

8

16

32

OSNR over 0.1 nm [dB]

Spectr

al effic

iency [

bit/s

/Hz]

PM-QPSK

PM-8QAM

PM-16QAM

PM-64QAM

PM-256QAM

PM-1024QAMShannon Bound

for PM formats

Requires extra 4 dB to

double spectral efficiency

Requires extra 8 dB to

triple spectral efficiency

OSNR Required at 111 Gb/s

Higher level modula-tion formats require increasingly higher OSNR even at constant bit rate

PM-16QAM requires 4 dB more than PM-QPSK

PM-64QAM requires 8 dB more than PM-QPSK

PM-256QAM would require 13 dB more than PM-QPSK!

BER=1E-3


100GE Interfaces on a Router

CFPM

A

C

FWD

Engine

Fabric

IF

CFPM

A

C

FWD

Engine

Fabric

IF

CFPM

A

C

FWD

Engine

Fabric

IF

Linecard 1 Linecard 2Fabric

Not every router will be able to implement

100GE…

MM

M M

MM

M M

MM

M M


Some Challenges for a 100GE Linecard

Substantially larger and power hungry pluggable

Many current MAC Chip on FPGAs: more space, power

Forwarding Engine: 100GE != 2x50GE

Higher PPS, faster memory lookups, faster interfaces

Packet Buffering @ 100Gbps

Circuit Board challenges:

Real Estate – difficult to fit everything

Circuit routing – several layers

Xtalk between lanes @ high speed SerDes interfaces

Power dissipation


Pluggable Comparison: CFP vs XFP

XFP CFP

Length 78 cm 145 cm

Width 18 cm 77 cm

Height 8.5 cm 13.6 cm

Power

Consumption3.5W 32W


Key Takeaways

100Gb/s Ethernet is already a reality

Implementing 100Gb/s links poses many challenges to many currently deployed systems

Optical costs become much more relevant at 100Gb/s links

Careful planning is needed to take full advantage of the bandwidth gains


Documents

100 Gigabit Ethernet