Layout of the OSMOSIS Optical Switch Controller Board using Expedition

Layout of the OSMOSIS Optical Switch Controller Board using Expedition

or

IS hindsight nearly always 20/20 … ?

pdi, Layout of the OSMOSIS OSCB, May 2006

2

Outline

OSMOSIS Project Design Entry Board Structure, Materials Signals, Rules, Constraints 1st Approach 2nd Approach 3rd Approach “Final” Approach Conclusion


3

Design Entry

.

.

.

8 FPGA Designs

I/O Designer

DesignView

HyperLynx

PreLayoutSimulations


4

OSCB Chassis

Test chassis with apre-version of the OSCB Board

Pre-OSCB

OSCI Interface Cards


5

Size: 431.8 x 573mm (17" x 22.5")- fits into a 19" chassis

each slot connector has 125 pins single ended, 120 diff pin pairs; total of 285 signal pins

1 FPGA, 1020 pins

7 FPGAs, 1704 pins

40 Slot connectors 20 on front 20 on back

OSCB Board

3644 Components


6

OSCB

View from Top

Use Blind Vias

Connector Fanout Problem

1.8

mm

Blocked Routing Channels

Half Board Thickness2.0mm minimum (78mils) (16 Layers)

Free Routing Channels

OSCI Conn

OSCI Conn


7

12 Rows of signal pins

min 12 internal layers

(outer layers not used)

• ≥ 16 signal layers

1704 pin FPGA (Xilinx FF1704 Package)


8

High speed design requirements

12954 Nets— 4072 diff pairs (clocks + data)— 5000 single ended

Clock frequency: 125MHz, 156MHz I/O Technology used:

- LVPECL Master Clock- LVDS Data, Clk, Sync- LVCMOS Control, uncritical signals

Impedances:- LVPECL 100 Ω balanced- LVDS 100 Ω balanced- LVCMOS 60..70 Ω single ended

Long Wires: up to 750mm (30")


9

S1,HS2,V

S3,H

S4,V

S5,HS6,V

S7,H

S8,V

Board Structure 16s16p

B.C. Plane, (high . Plane, (high εεr)r)

B.C. Plane, (high . Plane, (high εεr)r)

"LoFlow" Prepreg

Hal

f B

oar

d T

hic

knes

s ~

2.3m

m (

90m

ils)

Blind Via 1-16

Thru Via, Drill: 0.45mm (18mil)Aspect Ratio 1:10

Blind Via 1-16, Drill: 0.2mm (8mil)Aspect Ratio 1:11Thru Via

+3.3V

GND

GND

GND

GND

GND

GND+2.5V

+1.5V

ExcessiveMaterial

Danger!


10

Isola IS620

Low Dielectric Loss: <0.01 @ 2..10GHz (FR4: 0.02)

Low Permitivity: εr = 3.5 @ 1GHz(FR4: 4.4)

Low vertical CTE: 40ppm/ºC(FR4: 175ppm/ºC)Lower risk of torn vias!

Cost: ~3x FR4 (“moderate”) FR4 compatible, but process parameter tuning

required

Board Material


11

Rules / Constraints

Done in CES:- 86 Signal Classes- 1 additional Scheme for BGA Areas- 7 Clearance Rules for Netclasses

General Rules Trackwidth (60..70 Ω):Track Width outer: 200 μm (8 mils),

inner: 150 μm (6 mils) Track-to-Track outer: 200 μm (8 mils),

inner: 100 μm (4 mils) Diff Pairs on inner Layers (100 Ω):

100-140-100, 100-115-100 (μm)


12

Wiring challenge

Each star consists of ~900 Signals

Exception:Local diff pair wiring


13

Challenges

Board Manufacturing:- Size alone not a problem, but...- 100μm Structures alone not a problem, but…- 32 Layers alone not a problem, but…- IS620 alone not a problem, but… All together, - can that be done at all?

Wiring:- 12900 Signals is much, but… 4000 Diff Pairs is incredibly much! but… most of these pairs wired in global stars

This is going to be tough!


14

1st Approach

PlacementTop: all major compsBottom: "Chickenfood"

Top Bottom

"Standard" routingmethod:- PWR/GND- Critical Signals-


15

1st Approach - Observations (1)

- 6 week effort Only 80% completion (still 1500 opens!)

- manual completion not feasible

need breakthrough!

- Autorouter takes looong (days!)

Almost impossible to do test runs


16


Autorouter cannot convert blind vias into thru vias: poor FPGA fanout

Manually add thru vias under the inner 4 rows of signal pins

Thru Via (1 btw)Blind Via (1-16)

(2 between)

4 inner rows

8 outer rows

Blind Via (1-16)(2 between)

12 rows

Power Supply

Thru Via


17

Autorouter cannot connect a diff pair to different vias

Manually convert blind vias to thru vias, where necessary

blind vias

thru vias

Diff Pairs


Autorouter does not know fences (hard or soft)

"Workaround":Use route obstructs to guide router


18


Question of an expert: Why are all FPGAs on the same side?

Discussion with manufacturer: FPGAs on both sides can be done

place 4 FPGAs on top side and

4 FPGAs on bottom side


19

2nd Approach

New FPGA placement – 4 on top, 4 on bottom

Rewire from scratch - except master CLK and supply

(0v2, 10sep05)


20

2nd Approach - Observations (1)

Much better results!(still ~1000 opens)

Did not solve the problem, Need breakthrough!

Asked Mike Bare from Mentor Graphics:Are we doing something wrong?

No principal mistakes, approach seems to be OK. settings seem to be OK.

Asked US top PCB „Guru“: Can this board be done?

Feasible; forget autorouter! Would do it manually, would need only 12 layers, would take 3 months – too late!

MG Switzerland runs an autorouter test without diff pair definition (all signals single ended)

99.8% Completion! 15 opens finished manually in under 1 hour


21


Congestion in the top connector area:Wide single ended bus causes partial blockage of diff pairs

Conclusion: Turn top daughtercard slots 180º


22


On the edge of despair…

Expedition very, very slow - e.g. „Save“ takes about 10 minutes - e.g. move and drop a simple component (e.g. Cap) can take 10-15 seconds - 2GB of memory not sufficient crashes - Routing passes can take several days - CES seems to extremely slow down Expedition


23

3rd Approach

Turn top daughtercard section 180º

Buy new PC- Athlon 64 X2 Dual Core 4800+- 4GB Memory- Latest MB technology

Rewire from scratch (except power supplies)


24

3rd Approach - Observations (1)

Further improvement???? opens

Still not the final solution, Need breakthrough!!!

Too many open diff pairs after autoroute:Router seems to have real difficulties with diff pair fanout out of the BGAs

No immediate solution, Manually place 2 diff vias individually!

Difficult / often impossible to place a diff pair of vias, even if there is enough space

Even with new PC:Slow performance still almost unbearable

Use ExtremePCB and ExtremeAR!

No immediate solution, Manual routing! Mostly easy!

Measure: Use 4 more layers


25

4th Approach – Add 4 more layers(possible without changing board thickness)

S1,HS2,V

S3,H

S4,V

S5,HS6,V

S7,H

S8,V

S1,HS2,V

S3,H

S4,V

S5,HS6,V

S7,HS8,V

S9,HS10,V

20s16p16s16p


26

4th Approach

Use 4 more layers

Install XtremePCB,setup server + 2 client sessions, and

Add one more person

Install XtremeAR setup server + 3 client processes

Remove CESCES slows down Expedition even more with the new pre-release required for running XtremeAR

Rewire from scratch


27

4th Approach – Use LDIR

Vertical layers are more utilized,especially 5 inner PFGAs need more vertical routing space


28

- All PWR / GND- Flow Control Bus (single ended)- Master Clock (DP) and Sync Signals (SE),

tuning, manual clean-up

- Diff Pairs:Partial route (per FPGA)

1) route opposite side - force thru vias

2) route FPGA side - force blind vias

3) route both sides

4) manual Clean-Up

4th Approach – Routing Method (1)


29

4th Approach – Routing Method (2)

- Repeat Partial Routing / Clean-up...

- Single Ended Signals- Tuning- DRC- Clean-Up- DRC- Generate Data


30

• Supply all done


31

• Flow Control Bus


32

• Flow Control Bus


33

• Flow Control Bus • Global Sync Signals


34

• Flow Control Bus • Global Sync Signals


35

• Flow Control Bus • Global Sync Signals • Master Clock (Diff Pairs)


36

• Flow Control Bus • Global Sync Signals • Master Clock (Diff Pairs)


37

• Flow Control Bus • Global Sync Signals • Master Clock (Diff Pairs) • Local Diff Pairs (1)


38

• Flow Control Bus • Global Sync Signals • Master Clock (Diff Pairs) • Local Diff Pairs (1)


39

• Flow Control Bus • Global Sync Signals • Master Clock (Diff Pairs) • Local Diff Pairs (1) • Local Diff Pairs (2)


40

• Flow Control Bus • Global Sync Signals • Master Clock (Diff Pairs) • Local Diff Pairs (1) • Local Diff Pairs (2)


41

• Flow Control Bus • Global Sync Signals • Master Clock (Diff Pairs) • Local Diff Pairs (1) • Local Diff Pairs (2) • Global Diff Pairs (1)


42

• Flow Control Bus • Global Sync Signals • Master Clock (Diff Pairs) • Local Diff Pairs (1) • Local Diff Pairs (2) • Global Diff Pairs (1)


43

• Flow Control Bus • Global Sync Signals • Master Clock (Diff Pairs) • Local Diff Pairs (1) • Local Diff Pairs (2) • Global Diff Pairs (1) • Global Diff Pairs (2)


44

• Flow Control Bus • Global Sync Signals • Master Clock (Diff Pairs) • Local Diff Pairs (1) • Local Diff Pairs (2) • Global Diff Pairs (1) • Global Diff Pairs (2)


45

• Flow Control Bus • Global Sync Signals • Master Clock (Diff Pairs) • Local Diff Pairs (1) • Local Diff Pairs (2) • Global Diff Pairs (1) • Global Diff Pairs (2) • Global Diff Pairs (3)


46

• Flow Control Bus • Global Sync Signals • Master Clock (Diff Pairs) • Local Diff Pairs (1) • Local Diff Pairs (2) • Global Diff Pairs (1) • Global Diff Pairs (2) • Global Diff Pairs (3)


47

• Flow Control Bus • Global Sync Signals • Master Clock (Diff Pairs) • Local Diff Pairs (1) • Local Diff Pairs (2) • Global Diff Pairs (1) • Global Diff Pairs (2) • Global Diff Pairs (3) • Global Diff Pairs (4)


48

• Flow Control Bus • Global Sync Signals • Master Clock (Diff Pairs) • Local Diff Pairs (1) • Local Diff Pairs (2) • Global Diff Pairs (1) • Global Diff Pairs (2) • Global Diff Pairs (3) • Global Diff Pairs (4)


49

• Flow Control Bus • Global Sync Signals • Master Clock (Diff Pairs) • Local Diff Pairs (1) • Local Diff Pairs (2) • Global Diff Pairs (1) • Global Diff Pairs (2) • Global Diff Pairs (3) • Global Diff Pairs (4) • Global Diff Pairs (5)


50

• Flow Control Bus • Global Sync Signals • Master Clock (Diff Pairs) • Local Diff Pairs (1) • Local Diff Pairs (2) • Global Diff Pairs (1) • Global Diff Pairs (2) • Global Diff Pairs (3) • Global Diff Pairs (4) • Global Diff Pairs (5)


51

• Flow Control Bus • Global Sync Signals • Master Clock (Diff Pairs) • Local Diff Pairs (1) • Local Diff Pairs (2) • Global Diff Pairs (1) • Global Diff Pairs (2) • Global Diff Pairs (3) • Global Diff Pairs (4) • Global Diff Pairs (5) • Single Ended (1)


52

• Flow Control Bus • Global Sync Signals • Master Clock (Diff Pairs) • Local Diff Pairs (1) • Local Diff Pairs (2) • Global Diff Pairs (1) • Global Diff Pairs (2) • Global Diff Pairs (3) • Global Diff Pairs (4) • Global Diff Pairs (5) • Single Ended (1)


53

• Flow Control Bus • Global Sync Signals • Master Clock (Diff Pairs) • Local Diff Pairs (1) • Local Diff Pairs (2) • Global Diff Pairs (1) • Global Diff Pairs (2) • Global Diff Pairs (3) • Global Diff Pairs (4) • Global Diff Pairs (5) • Single Ended (1) • Single Ended (2)


54

• Flow Control Bus • Global Sync Signals • Master Clock (Diff Pairs) • Local Diff Pairs (1) • Local Diff Pairs (2) • Global Diff Pairs (1) • Global Diff Pairs (2) • Global Diff Pairs (3) • Global Diff Pairs (4) • Global Diff Pairs (5) • Single Ended (1) • Single Ended (2)


55

• Flow Control Bus • Global Sync Signals • Master Clock (Diff Pairs) • Local Diff Pairs (1) • Local Diff Pairs (2) • Global Diff Pairs (1) • Global Diff Pairs (2) • Global Diff Pairs (3) • Global Diff Pairs (4) • Global Diff Pairs (5) • Single Ended (1) • Single Ended (2) • Tuning, Clean-Up


56

4th Approach – Observations (1)

Marginal improvement of routing results

- Further improvement of placement to free routing channels

Now we have quick feedback, Can do router test runs!

- Routing passes take now hours (instead of days)

Can be easily completed manually in most cases!

- Beyond 90% completion autorouter leaves more and more nets (DPs!) open


57


Tedious manual repair- Router introduces many diff pair separations


58


Tedious manual repair- Odd FPGA routing

Partial blockage


59


Still not the final solution- Further improvement

Fallback Solution: Partial wiring

97.7% completion (672 opens!) – after manual effort

- Time has now become the determining factor! manufacturing deadline

Design does not fully meet requirements reduced performance


60

Board Statistics

Parts Placed: 3643 Pins: 42142 Nets: 12954 Differential Pairs: 4072 Connections: 29246

Routed: 97.7% (672 Opens) Total Trace Length: 2.59 km (1.6 miles) Parts Placed: 3643 Total plated holes: 50987


61

Tools Used

I/O Designer

Hyperlynx (pre Layout Simulation)

DesignView

CES

Expedition

Hyperlynx(Post Layout Simulation)

Xtreme

1 2 3 4


62

Overall Results (1)

Our approach seems to be OK, no obvious mistakes

Extra four layers (due to poor router performance): Symptom treatment, not fix! caused additional manufacturing problems

CES performance inadequate!Especially for large designs, where CES is really needed, it becomes almost unusable

Xtreme AR brought breakthrough!

Outstanding support from MG Switzerland!

Bare in Mind: According to MG this is one of themost complex and demanding designs at the time


63

Overall Results (2)

Router has difficulties with diff pair fanout of large BGAs, many opens can easily be routed manually- Our view: Diff pair fanout of large BGAs not

solved

Router introduces separated diff pairs

Lack of router control, e.g.- cost factors, - x/y vs. orthogonal routing,- control script,- fences (hard and soft)


64

Overall Results (3)

No "Advanced Fanout":- Fanout traces and vias should be flagged accordingly and considered part of device, even after device is placed

much less hassle when large device with fanout needs to be moved/pushed

Spread doesn't seem to work properly with diff pairs

Too much hidden automatism,should be left to the user when to use auto functions


65

Oh, by the Way - Some (Personal) Insights

The answer is: Yes, almost always!

When changing over to a new PCB tool – better don't start with a design such as this one!

For designs like this: There is no quick solution!

OSCB design brought tools to the limits - BUT – to make things clear – OSCB is an exception

- Tool supports newest technology - Mentor Graphics is committed to do their part

Documents

Layout of the OSMOSIS Optical Switch Controller Board using Expedition