Electronics May03

8/8/2019 Electronics May03

http://slidepdf.com/reader/full/electronics-may03 1/49

High-speed backplane interconnect

Vladimir Stojanovic

(with slides from J. Zerbe, P. Desai, R. Kollipara)



Outline

Inside the router

Backplane channel problem

What can backplane designer do about it

What can IC designer do about it

Scaling the system to 10-100Tb/s



Inside the Router

MAC

MAC

TM/Fabric

IF

TM/

Fabric

IF

NPU

NPU

SerDes

SerDes

Optics

OpticsSerDes

SerDes

SerDes

SerDes

Crossbar

Crossbar

Line Cards:8 to 16 per System

Switch Cards:2 to 4 per System

Passive

Backplane

M E M

M E M

M E M

M E M

M E M

M E M

M E M

M E M

Past OC-12

622 MHz LVDSparallel

GigE

1.25 Gbps serial

Present OC48

2.5 Gbps serial

10GigE

XAUI (3.125 Gbps) serial



OpticsMAC/

Framer

NPU/

TM

Switch

Fabric IF

XAUI

4, 3.125 Gbps

Serial Links

SPI4.2 CSIX

ProprietaryBackplane

8 to 16

of 1-3.2Gbps

Serial Links

Line Card:

Switch Card:32 to 64

Backplane

Serial Links

(1-3.2 Gbps)

Switch

Crossbar

IC

Serial Links in Networking Systems



Backplane interconnect path

Backplane via

Backplane connector

Line card

trace

PackageChip

Line card

viaBackplane trace

Packageto board

transition

There are many components on the signal path,

potential source of problems



RaSer X Link Features

PLLPLL

SerializerSerializer Tx Link

20 bit

ParallelInterface

1-10 Gbps

RefClk

Tx

Eq

Deserializer andDeserializer and

CDRCDR

Rx Link

1-10 Gbps

Rx

Eq

20 bit

Parallel

Interface

TX

RX

PLL

Process 0.13µ CMOS

Power 40mW / Gb

Area 1mm2

2-PAM Range 2 – 6.4 Gb/s

4-PAM Range 5 – 10 Gb/s



Impedance-controlled (CML) I/Os

Integrated terminations

Adjustable output-voltage/common mode

I/O Driver Scheme (Example)

50 Ω 50 ΩVtt

Zo = 50 Ω

Zo = 50 Ω RxTx

Vtt



System Issues

Goal – Increase Router Throughput

Limitations

Backplane channel

Power

Mechanical/Physical density constraints

Backplane and linecard routing density

Connector pin density

Package I/O density



Outline

Inside the router







Backplane Component Effects

PCB only

PCB + Connectors

PCB, Connectors,

Via stubs & Devices



Deterministic Noise

0 2 4 6 8 10

-60

-50

-40

-30

-20

-10

0

frequency [GHz]

A t t e n u a t i o n [ d

B ]

FEXT

NEXT

THROUGH

0 1 2 3

0

0.2

0.4

0.6

0.8

1

ns

p u

s e r e s p o n s e

Tsymbol=160ps

Inter-symbol interference Dispersion (skin-effect, dielectric loss) - short latency

Reflections (impedance mismatches – connectors, via stubs,

device parasitics, package) – long latency



XTALK and reflections

Far-end XTALK (FEXT)

Desired signal

Near-end XTALK (NEXT)

Reflections

Primary reflection sources are at the connector/backplane

transition

Grouped in time – as a function of backplane length



Backplane channel variations

0 2 4 6 8 10

-60

-50

-40

-30

-20

-10

0

frequency [GHz]

A t t e n u a t i o n [

d B ]

9" FR4,via stub

26" FR4,via stub

26" FR4

9" FR4

Variability in trace length, routing layer and via stub

Significantly different transfer functions even within the same backplane



Test Backplane Example

Dielectric material FR-4 Nelco Roger

6000 4350

Dielectric constant 4.2 4 3.6

Loss tangent (1 MHz) 0.016 0.005 0.0035

Loss tangent (1 GHz) 0.017 0.007 0.0035

Thickness 0.295" 0.299" 0.297"

FR4 Cross Section

Trace lengths: 1.5”, 9”, 14”, 20” and 32”

Effective number of signal layers: 13

Effective number of total layers: 28



Backdrilling - A Solution to the Stub

Effect



Stub Effect Eye Pattern Analysis(2.5 Gbits/sec FR-4)

MAX STUB MIN STUB



MAX STUB MIN STUB




MAX STUB MIN STUB




Connector design

GBX

Teradyne



Connector Density

Teradyne’s GbX™ Connector Teradyne’s GbX™ Connector

DifferentialPairs/inch

Card Pitch Bandwidth/linear inch (at6.25 Gbps)

5 pair 69 1.25" min.

(30 mm)

431 Gb

4 pair 55 1.00" min.(24.7mm)

343 Gb

3 pair 41 .80" min. (20mm)

256 Gb

2 pair 27.5 .575" min.(14 mm)

171 Gb



Reducing Crosstalk within the

Connector

Cross talk is

reduced in the

mating interfaceby surrounding

each pair with a

ground shield

D/C Shield

B/P ShieldMated pair



Backplane Connector Considerations

Many connector types: Teradyne: VHDM, HSD, GbX, …

Tyco: HS3, HMZd, …

FCI: Metral 2000, 3000, 4000, …

3M/Harting: HSHM, … ERNI: ERmetZd, ErmetXT, …

Issues Loss, impedance profile, crosstalk, skew

Foot print: routability, pin density, via impedance

Single-ended and differential

Press-fit and SMT



10Gbps Test Package Design Example

Ceramic BGA

Wire-bonded

4-Layer

1 mm pitch

Source: Designed for Rambus by Kyocera



Example of a really good backplane

15” FR4

20” Roger

Works with simple

OC192 xcvr



Outline

Inside the router







Loss : Equalize to Flatten Response

Channel is band-limited

Equalization : boost high-frequencies relative to lower frequencies

+

=



Receiver Linear Equalizer

Amplifies high-frequencies

attenuated by the channel

Digital or Analog FIR filter

Issues Also amplifies noise!

Precision

Tuning delays (if analog)

Setting coefficients

Adaptive algorithms such as

LMS

…

WL-1

DDD

WLW

1

+

H(s)

freq



Transmitter Linear Equalizer

Attenuates low-frequencies Need to be careful about output

amplitude : limited output power

If you could make bigger swingsyou would

EQ really attenuates low-

frequencies to match highfrequencies

Also FIR filter : D/A converter

Can get better precision than Rx

Issues How to set EQ weights?

Doesn’t help loss at f

H(s)

freq



Transmit Linear Equalizer :

Single Bit Operation

0.0 0.3 0.6 0.9 1.2-0.3

-0.1

0.1

0.3

0.5

0.7

UnequalizedEqualization PulseEnd of Line

time (ns)

V o

l t a g e



Decision Feedback Equalization (DFE)

Don’t invert channel… just remove ISI

Know ISI because already

received symbols

Doesn’t amplify noise

Requires a feed-forward

equalizer for precursor

ISI

Reshapes pulse to

eliminate precursor

-

FIR filter

Decision (slicer)

FIR filter

Feed-forward EQ

Feed-back EQ



DFE Example



Transmit and Receive Equalization

Transmit and receive equalizers are

combined to make a range restricted DFE Tx equalizer functions as the feed-forward filter

Rx equalizer restricted in performance of loop

TAP SEL

LOGIC

TX

DATA

3

RX

DATA



Tx & Rx Equalization Ranges

TX Driver/Equalizer : 5 taps

1(pre)+1(main)+3(post)

RX Equalizer

5-17 taps after main

Pick any 5 taps



Pulse Amplitude Modulation

Binary (NRZ) is 2-PAM

2-PAM uses 2-levels to send

one bit per symbol

Signaling rate = 2 x Nyquist

4-PAM uses 4-levels to send2 bits per symbol

Each level has 2 bit value

Signaling rate = 4 x Nyquist

00

01

11

10

1

0

1

0



When Does 4-PAM Make Sense?

First order : slope of S21

3 eyes : 1 eye = 10db

loss > 10db/octave : 4-PAMshould be considered

0.0

-20db

-40db

-60db



Example : 5Gbps Over 26” FR4

With No Equalization




Correct Tx Equalization




Under Equalized




Over Equalized



26” FR4 Bot 3.125Gbps, 2P noEQ



26” FR4 Bot 3.125Gbps, 2P w/EQ



26” FR4 Bot 6.4Gbps, 2P w/3G EQ



26” FR4 Bot 6.4Gbps, 2P w/EQ



26” FR4 Top 6.4Gbps, 2P w/EQ





26” Nelco6k-cb Top 10Gbps, 4P



26” Nelco6k-cb Top 6.4Gbps, 2P



Scaling the router throughput System (Tot. throughput ~2.5Tb/s)

8-16 Line Cards

<40Gbs / LC 3-6Gbs links

40mW/Gbs Link power in 0.13um

Speedup ~2x

#links at switch card ~ 200

Limitations

#diff pairs at switch card 16*40Gbs/6Gbs*2*2 ~ 400 Switch card power from links ~200*6Gbs*40mW/Gbs ~ 50W

Connector density 50diff pairs/inch: (tot length=400/50= 8” )

BP/LC routing pitch 0.050”

Num. Layers (BP=13, LC=4) 100diff pairs/layer = 5” LC routing width

Package ball pitch (1mm/200um)

S



Scaling the router throughput System (Tot. throughput ~100Tb/s)

100 Line Cards

1Tbs / LC 10Gbs links 4mW/Gbs Link power in 0.065um

Speedup ~1x

#links at switch card ~ 10k

Limitations #diff pairs at switch card 20k

Switch card power from links in 0.13um~10k*10Gbs*4mW/Gbs ~ 400W

Connector density 50diff pairs/inch: (tot length=20k/50= 400” )

BP/LC routing pitch 0.050”

Num. Layers (BP=13, LC=4) 5k diff pairs/layer = 250” LC routing width

Package ball pitch (1mm/200um)

Documents

Electronics May03