100Gbit/s, 120km, PAM 4 Based Switch to Switch, Layer 2 Silicon Photonics based Optical Interconnects for Datacenters

Radhakrishnan Nagarajan, Sudeep Bhoja and Tom Issenhuth** Microsoft Corp, Redmond, WA

2Inphi Proprarch 25, 2014 2

Increased Complexity, Speed and Volumes Driving

Application Opportunities for Silicon Photonics

Data Center Connectivity Trends

Within the Rack (Server to TOR)

― 10GbE Deployed at Scale (DAC)

― 25GbE Transition happening now (DAC/AOC)

― 50GbE forecasted to start in 18/19 (SR/AOC)

Between Racks

― 40GbE Deployed at Scale (SR4/PSM)

― 100GbE Transition in 2016 (SR4/PSM/WDM4)

― Mix of 200GbE and 400GbE in 17/28 (PSM/WDM4)

Inter Data Center

― 10GbE/40GbE/100GbE Deployed at Scale (LR/ER/DWDM)

― 100GbE/OTN Deployed at Scale for 80km+ (Coherent)

― 200GbE Starting in 2017 (80km to 600km)

― 400GbE Standardized now (DR8/FR8)

3Inphi Proprietaarch 25, 2014 3

Virtual Datacenter Architecture: Latency Limited

Data Center 1 Data Center 2

InternetInternet

TOR Switch

Edge SwitchEdge Switch

TOR Switch

ServerServer

Core Switch Core Switch

Transport Transport

4Inphi Propriarch 25, 2014 4

Virtual Datacenter Architecture: Latency Elimination

Data Center 1 Data Center 2

InternetInternet

TOR Switch

Edge SwitchEdge Switch

TOR Switch

ServerServer

4Tb/s per

FiberCore Switch Core Switch

Transport Transport

5Inphi Proprietarch 25, 2014 5

0 2 10 100 600 10,000

Distance (km)

Metro Datacenter Interconnect (DCI) Gap

Point to Point Links

IEEE 802.3ba /

CR4,SR4, LR4

Sta

nd

ard

s

100G

10G

DWDM

SFP+

ROADM Networks

OIF / Coherent

QPSK/16-QAM

100G/200G

DWDM

OIF / Coherent QPSK/8QAM

100G/200G/400G

DWDM

No Cost Effective 100Gb/s

DWDM Solutions Today

100G Switch

100G Switch

4Tb/s (40 DWDM

channels)

80km

EDFA

EDFA + DCF

Inter-datacenter links are beingupgraded to 100G.

Intra-datacenter links which enablelarge distributed architectures arethe bottleneck.

100G DWDM QSFP-28: Costeffective approach to resolve theDCI bottlenecks

6Inphi ProprA March 25, 2014 6

100G DWDM QSFP-28 Module

4x2

5G

CA

UI

Ele

ctr

ica

l In

terf

ace

λ 1

λ2

λ1

λ2

M

Z

M

D

F

B

D

R

V

E

N

C

F

E

C

C

D

R

P

D

T

I

A

D

S

P

F

E

C

C

D

R

Power

Supplies

MCU

Other

ASIC’s

PAM ASIC Silicon Photonics

PAM 4 ASIC Si Photonics

Receive

Transmit

• Compatible with the QSFP28 MSA as described

in SFF-8665

• Standard CAUI4 electrical interface

• Typical power consumption < 4.5W

• Electrical Input: 4 x 25.78125Gbit/s NRZ

• Optical Output: 2 x 28.125 Gbaud PAM4

7Inphi Propriet5, 2014 7

Fat Pipes Between DC’s: DWDM QSFP 28

2λ_100Gbit/s, DWDM

D

C

M

D

C

M

OA

OA

OA

OA

100GHz Channel

Spacing Mux/DeMux

ITU 50GHz

spacing.

2λ’s in ITU 100GHz spacing,

With a 25GHz Laser offset.

Effectively a 50GHz system.

*Exact locations of OA’s and DCM’s in the link are subject to OSNR considerations.

OA: Optical Amplifier

DCM: Dispersion Compensation Module

EDFA: Erbium Doped Fiber Amplifier

8Inphi ProprietarMarch 25, 2014 8

PAM 4 ASIC Block Diagram

9Inphi Propriet March 25, 2014 9

PAM 4 ASIC: Architecture

CML Driver with CMOS backend

Enables wide swing range and low power

½ rate TX Clocking and ¼ Rate RX Sampling

7-bit ADC-DSP based receiver with SAR core

The clock path is CMOS based with regulators providing the

required power rejection

The data path are under independent regulator domains for

proper isolation

Multi-Tap FFE / DFE and Calibration in the DSP

Reference-less, clock recovered from CAUI RX.

10Inphi Proprieta March 25, 2014 10

Silicon Photonics: Mach Zehnder Modulator

Optical Eye

PAM 4

Optical Eye

NRZ

Direct Coupled Driver

• Traveling Wave Mach Zehnder Modulator

• Optical-Electrical S21 Data

11Inphi Proprietaarch 25, 2014 11

Silicon Photonics: Receiver Path

• High Speed Ge Photodetector

• Optical-Electrical S21 Data

• Polarization Beam Splitter

• Insertion Loss and Crosstalk

PD Dark Current

12Inphi Proprich 25, 2014 12

Commercial 100GHz Multiplexer: Flat Top Gaussian

CH 34 – 193.4THz

Filter Center Frequency on 100GHz Grid

13Inphi PropriMarch 25, 2014 13

Module Dual λ Optical Output: 28.125GBaud Data

25GHz

EDFA Noise Limit

Shot Noise

Limit

14Inphi ProprietaMarch 25, 2014 14

Two Way Traffic Using a Commercial Switch

• Line system Transmit, Receive EDFAs.

• Dispersion Compensation at Mid-Stage of EDFA’s.

Commercial Switch

36 100Gbit/s ports

~3.6Tbit/s Capacity

15Inphi Proprie March 25, 2014 15

120km BER vs. OSNR Performance

• 120km SMF Fiber 26.2dB Loss

• Baud Rate = 28.125GBaud

KR4 Correction Limit

IFEC Correction Limit

• Dispersion Tolerance ±100ps/nm

• ~1dB Penalty

16Inphi Proprietch 25, 2014 16

PAM 4 Link: FEC Performance

No Errors for 30mins

ONT Module

EDFA - Gain

EDFA - Noise Loading

4 x 25Gbit/s

interface

• BER vs. OSNR for 100GbE.

• Error free overnight at OSNR = 28dB.

• KP4 and KR4 are Reed-Solomon

FEC codes.

• IFEC is a Iterative Multi-Layer code.

17In

Thank You!