DSP beam control Overview

A. Blas 09 Jan 2009 DSP BC Daughter cards 1/21

DSP beam control Overview

Freq. program Phase Cavity servoing& radial loop & synchro loop

PowerPC

DSP A DSP B

VME 64x

Soft timings monitor 40 MHz clock

Timing event

DSP C

VME64x J 2/P2 Tagged clock

Tagged clock fanout

Bup/Bdown

J 2/P2

C T R V

DSP FPGAs

Rear transition module

Rad PU

fREVPhase RF

Cavities 1 & 2 1 & 2

Harmonics(monitoring) PU ref

J 2/P2

DDC

DSP FPGAs


C T R V

C T R V

J 2/P2


DSP FPGAs

MDDS

CavityControl

I nterface

Cavities gap relay, beam status

DDCDDCSDDS SDDS

LEIR BCFrom M.E. Angoletta

DSP BC Daughter cardsA. Blas 09 Jan 2009 2/21

DSP Board6 unit VME board with:

1 DSP - ADSP-21160M from Analog Devices – 32 bit floating point – Super Harvard Architecture (SHARC), Clocked at 80 MHz (40 MHz external), 600 MFLOPS - with separate data and instruction busses, dedicated I/O processor with separate bus - 64 bit external data bus clocked at 40 MHz (320 MB/s), 6 byte link Ports at 80 MHz (80 MB/s), 2 serial ports at 40 Mbits/s.

1 MB Flash memory, to restore the DSP software.4 MB DSP RAM, to store external or calculated functions (like voltage and rf program, steering …)

4 daughter board sites, with two 69 pin connectors for each site

1 Timing interface, based on an AlteraTM FPGA1 VME interface, based on an AlteraTM FPGA1 Data interface, based on an AlteraTM FPGA that controls the data flux on Board.4 MB RAM, for acquisition of internal signals (diagnostics)16 Trigger inputs (via the RTM)1 Event link, basically a serial link connected to the rear transition module that allows taking into account 256 events with a hundred ns resolution (not used at CERN).6 link ports, each port allows up to 8 bit data transfer per DSP internal clock cycle (80 MHz)


MDDS Master DDS daughter card

AD9858 1GHz clock DDSADF4106 PLL + CLV1000A VCO (10MHz => 1 GHz)No SRAMAltera Stratix EP1S10F484C5 2 Tagged clock outputs with IEEE1394 mechanical standard Tagging inside the FPGA with external ELMEC 1.2ns delay (needs control of logical cells placement)

Designed in

November 2004


IEEE 1394b clock distribution

1394b bilingual connector 8 x 5 mm

1.5 ns tag 333 Ms/s equivalent throughputStandard used up to 3.2 Gb/s over 5m cables9 pin connectorLVDS electrical standard


Clock Fan-out VME card

1 input with IEEE1394 mechanical standard 1 Optical input / 1 optical output9 outputs IEEE1394Delay error between channels below 390 ps

Designed in

June 2005


DDC Digital Down Converter daughter card

4 channels 14 bit ADC AD 9245BCP-80 (80 MHz max, 366 mW, pipeline = 7 cycles)

2 MB/10 ns SRAM (1 M x 16 bits)

1 Altera Stratix EP1S20F484C5

1 Tagged clock input with IEEE1394b mechanical standard and de-tagging and divide-by-2 circuit (50% duty cycle and minimum pulse width required by the ADC).


DDC Digital Down Converter daughter card – FPGA content

Modulatedrf ADC

I

Q

Anti-aliasfilter

Sin (hLO*wRF)

Cos (hLO*wRF)

hCLK*fRF

14

15

15

15

15

CIC

CIC

FIR

FIR

1. 4 down mixers with 4 different LO frequencies

2. Programmable CIC, 0 -> 3 stages, 1 ->15 decimation, 1 -> 255 comb delay, normalization gain

3. CIC, FIR and LO parameters loaded synchronously with a specific double-tag. Allows on-flight MDDS harmonic change

4. I/Q frequency discriminator

5. LO frequency measurement (counter)

6. 4 different acquisition lines memorizing 4 of 28 different inner signals. Sampling at a chosen sub-harmonic of the main clock. Up to 1 M points per signal at a maximum sampling of 80 MHz.

7. Channel, CIC, FIR ON/OFF

8. Reset phase accumulator at each tag

1 channel / 4


SDDS Slave DDS daughter card

4 channel 14 bit DAC AD 9754BCP-80 (125 MHz max, 185 mW, no pipeline delay)2 MB/10 ns SRAM (1 M x 16 bits)Altera Stratix EP1S20F484C51 Tagged clock input with IEEE1394 mechanical standard and de-tagging and divide-by-2 circuit (50% duty cycle and minimum pulse width required by the ADC).High/Low DAC gain (0 – 18 dB) 17 bit DAC for lowest amplitude signals


SDDS Slave DDS daughter card

DAC

I

Q

Sin (hLO*wRF)

Cos (hLO*wRF)

Modulatedrf

1. 4 up-converters with 4 different LO frequencies

2. LO parameters loaded synchronously with a specific double-tag. Allows on-flight MDDS harmonic change

3. LO frequency measurement (counter)

4. 4 different acquisition lines memorizing 4 of 28 different inner signals. Sampling at a chosen sub-harmonic of the main clock. Up to 1 M points per signal at a maximum sampling of 80 MHz.

5. Channel ON/OFF

6. Reset phase accumulator at each tag

7. Automatic DAC gain setting with respect to required output amplitude

8. Phase modulation from SRAM

Stratix 1 FPGA filling-up

!

DDC with only 2/4 LO’s + only 2nd order FIRAnd no monitoring of inner signals

Compiled with an old version of Quartus; now it is much faster

A. Blas 09 Jan 2009DSP BC Daughter cards

10/21


Daughter card – FPGA interconnections(Here the DDC)

FPGADSPaddr [14..0]

DSPdata [63..32]

BCLK (CHAN n CLK)

FLAG0-FLAG1 (System Reset)

DSPIRQ0

DSPWRH-RDH

DSP ACK

SELn0-SELn1

SRAM1M x

16 bits

RAM Address [17..0]

RAM Data [15..0]

RAM BLE

RAM BHE

RAM WE

RAM OE

RAM CE

Site n Connect 1

69 pins

Site n Connect 2

69 pins

Site n+1 Connect 1

69 pins

Site n+1 Connect 2

69 pins LINK PORT n+2

33 10X

JTAG spare

Spare (6)

4XADC

Ch1-4 ADC out [13-0]

Ch1-4 OTR (out of range)

PDWN

JTAG

LINK PORT n+1

33 10X

Reset

Spare (6) Clockinterface

MDDS/2 (50% DC)or 40 MHzor FPGA clkor Wide-tag

TAG + Wide Tag

Clock Selection

Testconnector

20 pins


FPGA market(to be completed, very fast survey!)

XilinxVirtex 2XC2V2000-6FG676C(Tuner loop)

Altera StratixEP1S20F484C5(Leir daughter cards)

XilinxVirtex 5XC5VLX110 1FF676I

AlteraStratix IIIEP3SL150F780C4N

Altera CycloneEP3C80F780C6

Logic cells 24,192 18,460 110,592 142,000 81,284

Clock frequency

< 269 MHz < 450 MHz (-1)< 550 MHz (-3 speed grade

< 450 MHz< 600 MHz (C2 version)

<340 MHz

RAM [kb] 1,000 1,600 4.718 5,500 2,745

18 x 18 multipliers

56 80 384 244

I/O 456 361 440 480 429

Price [USD]

278 350 2053 (Avnet) 2184 (Altera) 368

Serial links

177 LVDS channels (840 Mbps each)


FPGA market(to be completed, very fast survey!)


DC to Mother Board data exchange

The DSP on the motherboard sends/receives values to/from registers on the Daughter cards.

Some of these register values are within a servo loop and the acquisition time is important for the stability of this loop.

In the present Leir BC, the data flow within a loop is as follow:

1. The DDC measures I/Q of a variable

2. I/Q is read by the DSP which processes the value taking into account values from other channels, other daughter cards and other DSP boards if required. The DSP process is triggered by a so-called fast clock.

3. A correction value is sent to the MMDS to change the rf clock frequency and possibly parameters of the SDDS


DC to Mother Board data exchange


Loop delay

Rule of thumb:

The loop will be sufficiently stable if its delay leads to a phase lag < p/4 at the unity loop gain frequency

Loop computation time + Hdw delay < [1/(8.Fmod)]< 21 us for PSB & LEIR (3.Fs=6kHz)< 26 us for PS (3.Fs=4.8 kHz)

In LEIR, the in-loop DSP is sampling the data every TS-DSP = 12.5 us (80 kHz).The loop delay within the DSP = import data from DDC (<50ns) + compute error (<7us) + send error to MDDS or SDDS (via another DSP or not < 150 ns) + equivalent 1st order S/H delay (6.25 us) => ≈14us delay within the DSPWith a 80 kHz DSP sampling clock, an averaging (CIC) of 1000 80-MHz-samples in the DDC would be adequate. We actually use 256, which means <6.4 us extra delay. This means that we are approaching the reasonable limits required for the LEIR and PSB phase loop.

The DSP process time in LEIR is the most time consuming and multiplying by a factor 2 this process speed would almost double the possible bandwidth.


Phase jitter within the Loop

The error signal within a loop as described in the previous slide is transferred from the rf clock domain (DDC) to another clock domain (DSP) and again into another clock domain (MDDS or SDDS).

From DDC to DSP the uncertainty is 1 rf clock period (<25 ns)+ 1 wait state of the DSP (25 ns) The induced phase jitter is (+/- 360o x FMOD) x Δti = +/- 18 mo / kHz of modulation

From DDC to MDDS the uncertainty is just 1 MDDS clock period (8 ns)(the 125 MHz on the frequency word acquisition side)

The total phase jitter within the loop is thus around +/- 21 mo / kHz of modulation

To this value should be added-up the effect of any change in the DSP computation time from one DSP interrupt clock tic to the following and also all the erratic wait states of the DSP acquisitions that occur in between this DDC acquisition to the MDDS frequency change.


Communication links

With the new motherboard, the Daughter cards will communicate with the DSP via a FPGA. This architecture allows the use of many kinds of communication links, serial or parallel.

VMEBUS

FPGA

RAM

SpecialFunction

Board

DSP

SpecialFunction

BoardSpecial

FunctionBoard

OtherDSP

BoardOtherDSP

BoardOtherDSP

BoardOtherDSP

Board

Do we want the “Daughter card” concept, or do we use detached “special function” boards?

Do we use one dedicated link for each daughter card or do we share a single link (all cards receiving the interrupt clock)?

SpecialFunction

BoardSpecial

FunctionBoard

SpecialFunction

Board

OtherDSP

BoardOtherDSP

BoardOtherDSP

BoardOtherDSP

Board


DC to Mother Board data flowSamtec QSE, 80 (4 small ribons of 20 coax cables on a single connector

160 Gb/s (at 2 GHz)

3.2 Gb/s (at 40 MHz)

Used in LHC

Gigabit serial link with 8b/10b coding16bit// 50 MHz -> 1Gb/s serial

800Mb/sFull duplex at the same rate

Used in LHCVery common standard

DSP bus + mother board to daughter card bus at 40 MHz and 32 bit word: 1.28 Gb/s (to be divided by the number of wait states +2 = 4)

320 Mb/s Used in LeirDSP <-> DC

Link port clocked at 40 MHz (can be used at 80 MHz)

320 Mb/s Used in LeirDSP <-> DSP

Single 32b transfer: < 25ns+25ns = 50 ns with // DSP bus< 32/800MHz + 20 ns + 8b @ header ? = 60 ns + 10ns? with Gigabit 8b/10b link


Conclusion

The FPGA on the daughter cards needs to be upgraded (except MDDS) . It could be the same as the one used on the DSP board (market survey !!)

Finding ADCs and DACs not-requiring 50% duty cycle would allow an increase of the rf sampling clock

Anti-alias filter off-Board or with an easy plug-in / plug-out connector

JTAG connector from VME and Front-Panel

The clock distribution connectors should be more robust, same as for the data serial link (market survey !!)

The data link from DC to FPGA could be serial

Having an ADC (resp. DAC) on a DAC (resp. ADC) board would be convenient and it would allow to use the board for a 1TFB and a TFB (there is a developer available for this job!)


Conclusion

Possible architecture

Documents

DSP beam control Overview