91
A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Embed Size (px)

Citation preview

Page 1: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

A Unified Approach toFast Digital Processing for

Beam Dampers, Instrumentation, & Controls

Bill Foster

Beam Instrumentation Workshop

May 6, 2004

Page 2: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

A Digital Manifesto

Or,

Example Application:

3-coordinate Bunch-by-Bunch Beam Damper

for Fermilab Main Injector

Implemented on a Single Altera Stratix FPGA

Five other applications using this same hardware

Page 3: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Once Upon a Time, there was a job called:“Audio Frequency Analog Engineer”

Their products:

• Mixers

• Equalizers

• Crossover networks

• Reverbs

• Fuzz Boxes….

Bob Widlar, “inventor of the IC Op-Amp” and other analog gemshttp://www.elecdesign.com/Globals/PlanetEE/Content/3080.html

Page 4: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Nowadays, an “Audio Frequency Analog Engineer” is any high-school kid with a PC and SoundBlaster CardTheir products:• Mixers• Equalizers• Crossover

networks• Reverbs• Fuzz BoxesEmulated on a PC

PLUS:

• Synthesizers that can fool my ears

• Time compressors that squeeze 20% off of a song’s play time without altering the pitch.

• Real-time tone substitution makes even Leonard Cohen sing on key

…try doing that with an op-amp!

Page 5: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

What Unemployed the Audio Analog Engineers?

• ADC Sampling Rates and Accuracies exceeded requirements – Audio requirements set by human ear

• Digital Processing capability exceeded requirements at reasonable cost– 2 GHz CPU executes 50,000 instructions per

audio waveform sample

Page 6: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Once Upon a Time, there was a job called: “Low-Level Radio Frequency Analog Engineer”

Their products:

• Mixers

• Equalizers

• Phase Shifters

• Down converters

• Phase-locked Loops

Fermilab’s Booster Low-Level RF systemas it exists today!

Page 7: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Nowadays, a “LLRF Analog Engineer”

is (or should be) any old programmerwith a fast digitizer and an FPGATheir products:• Mixers• Equalizers• Phase

Shifters• Down

converters• PLLs

Implemented in FPGA’s

PLUS:

• Direct Digital Synthesis of complex RF waveforms

• Built-in system diagnostics

• Digital Reproducibility (&spares!)

• High Speed Serial Links

• Multi-user support

Page 8: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

What Unemploys the Analog RF Engineers?

• ADC Sampling Rates and Accuracies exceed requirements ~ 4 samples per RF clock gives bunch-by-bunch phase

and amplitude

• Digital Processing capability exceeded requirements at reasonable cost– FPGAs and DSP’s

For ~100 MHz or less, it is: GAME OVER

Page 9: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Generic Hardware Concept for Accelerator Instrumentation & Control

Monster FPGA

Clock, control, ...

MinimalAnalogFilter

FASTADC

Cables from Tunnel

MinimalAnalogFilter

FASTADC

.........

FASTDAC

CPU BusVME/VXI/PCI/PMCetc.

OR SERIAL LINK

INPUTS:BPMStripline PickupResistive WallFlying Wire PMTRF FanbackKicker Monitor…etc.

OUTPUTS:Stripline KickerRF FanoutAnalog Monitor…etc.

Page 10: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

53 MHz, TCLK, MDAT,...

All-Coordinate Digital Damper

Monster FPGA

MinimalAnalogFilter

FASTADC

Stripline Pickup

MinimalAnalogFilter

FASTADC

12

212 MHz

Stripline Kicker

PowerAmp

MinimalAnalogFilter

FASTADC

Resistive Wall Monitor

Broadband Cavity

FASTDACs

> 27 MHz

FASTDACs

PowerAmp

TransverseDampers

IdenticalX & Y

Longi-tudinal

(Z)Damper

12

12

424 MHz

CPU:VME/VXI/PCI/PMCetc.

OR SerialLINK

Page 11: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

The Board

Alexi Seminov, Sten Hansen, Bill Ashmanskas, Dennis Nicklaus, Hyejoo Kang…

Page 12: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Some Example Applications using this same basic hardware:

1) Universal Beam Position Monitors (BPMs)

– Handles full variety of FNAL beam RF structure

2) Generic instrumentation readout “Scope”

– ex: Flying Wire readout for arbitrary bunches

3) Beam Loading Compensation

4) Universal Beam Dampers / Beamline Tuner

5) Entire Low-Level RF system

Page 13: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Fast, High Precision Pipelined ADC’s

• AD6645: 14 Bits, 105 MHz

• AD9430: 12 Bits, 210 MHz

• AD12500: 12 Bits, 500 MHz (hybrid)

• Several : 8 bits, ~1-2 GHz (‘scopes)

Private opinion: it appears that ADCs are about to fall off

of Moore’s Law curve the same way that CPU’s have…

Page 14: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004
Page 15: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

AD6645 Functional Block Diagram

• Two-Stage Pipelined ADC• Internal Track & Hold• Differential Analog Inputs

Page 16: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

This ADC can sample 53 MHz signals at 4 samples per cycleto measure both In-Phase and Quadrature on each cycle

Page 17: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004
Page 18: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Board Layout for High-Speed ADCsis a Lot Easier Than it Used to Be

• LVDS signals eliminate digital noise

– 0.25V differential swing far quieter than TTL

– Direct “glueless” interface to FPGAs

• Fast input op-amps and surface mount

components with small parasitics

– Front-end layout is not critical since it is

physically small

Page 19: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Clock Distribution for ADCsis a Lot Easier Than it Used to Be

• Clock and Signal timing can be fixed ex post facto via FPGA firmware timing adjustments

• Some A/Ds & D/As have internal PLLs to reduce or eliminate effects of clock jitter

• FPGAs have high-quality clock distribution which can be used to drive external A/D & D/As

• FPGA clock distribution can challenge Dedicated Clock Distribution Chips (on not…L. Dolittle)

Page 20: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Q: What ADC Clock Speed is needed?

A: 4x RF Bunch Frequency

• Minimum needed for bunch-by-bunch Phase and Amplitude measurement

• In frequency domain, 4x RF sampling measures both in-phase and quadrature components.

• For Fermilab’s 53 MHz RF 212 MHz ADC’s

Page 21: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

212 MHz Sampling of RWM Pulse

Low-pass Filter

Spreads signal +/-5ns in time so it will not be missed by

ADCFilterReduces ADC

Dynamic Range

requirement, since spike does not

have to be digitized

FilteredOutput

Input Pulse from RWM

DB

E

C

A

-8

-6

-4

-2

0

2

4

6

8

10

-25 ns -15 ns -5 ns 5 ns 15 ns 25 ns

single-bunch "phase" signal = samples (B - D)

single-bunch "intensity" signal = C - average(A,E)

Page 22: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

212 MHz Sampling of Stripline Signal

Roles of “Phase” and “Amplitude” signals are reversed

from unipolar case.

FilteredOutput

Input Doublet

D

B

E

C

A

-10

-8

-6

-4

-2

0

2

4

6

8

10

-25 ns -15 ns -5 ns 5 ns 15 ns 25 ns

Single-Bunch Amplitude signal is(B - D) samples of filtered output

Phase Signal isC - Average(A,E)

Page 23: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Repetitive Waveform looks like simple sine wave, but contains bunch-by-bunch phase and amplitude

“A - B” gives bunch-by-bunch “in-

phase” signal

Vector Sum sqrt(I**2 +Q**2) is

insensitive to clock jitter

FilteredOutput

Input Doublet

B

A

EDC

-10

-8

-6

-4

-2

0

2

4

6

8

10

-25 ns -15 ns -5 ns 5 ns 15 ns 25 ns

“D - (C+E)/2” gives

bunch-by-bunch “out-of-

phase” or “quadrature”

signal

Page 24: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Bunch-By-Bunch Phase vs. Turn NumberMeasured with MI Digital Damper

-2000

0

2000

4000

6000

8000

10000

1 501 1001 1501 2001 2501 3001 3501

• Damper Output comes from derivative of individual bunch phase errors

Page 25: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Bunch-By-Bunch Intensity

Page 26: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Synchronous vs. Asynchronous ADC Sampling

• The choice is between – N*53 MHz beam phase locked sampling, or– Asynchronous sampling at a (possibly) lower rate

• Asynchronous sampling of a waveform will allow you to recover all the information, IF:– you know that the input is a pure sine wave, or– you know the input is repetitive (stored beam), or

– the sampling rate is much higher than fMAX

My belief is, undersampling is just a bad idea…

Page 27: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

The Perils of Undersamplinga Single-Pass Beam

FilteredOutput

Input Doublet

Asynchronous ADC SAMPLES

-10

-8

-6

-4

-2

0

2

4

6

8

10

-25 ns -15 ns -5 ns 5 ns 15 ns 25 ns

If a single-pass beam does not have uniform

bunch populations,

the ADC input is NOT a good sine wave and

an undersampled waveform can

give an erroneous

picture of the beam signal.The signal CAN

be reconstructed

with many passes of

stored beam.

Page 28: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Beam Bunch Structures in Fermilab Main Injector

53 MHzBATCHES

(84 Bunches)

SHORTBATCHES

5-15 Bunches

2.5 MHzBUNCHES

COALESCED2.5 MHz

BUNCHES

2.5 MHzAntiprotons

7.5 MHzCOALESCED

7.5 MHzAntiprotons

… dealing with this variety of beams would be painful in Analog…

Page 29: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Recycler BPM Preamp Signal with 4 bunches at 2.5 MHz Prieto's Low-Q Preamp with 10 nsec (100 MHz) Digitization

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

0 500 1000 1500 2000 2500 3000

Time (ns)

BP

M S

ign

al (

V)

Digital Filter looking at many samples can still extract individual bunch transverse positions

Page 30: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Advantages of Digital Processing

• Digital filters more reproducible (=>spares!)

• Inputs and Outputs clearly defined (& stored!)

– filters can be developed & debugged offline

• Digital filter can also operate at multiple lower

frequencies ...simultaneously if desired.

• Re-use Standard hardware with new FPGA code

– or same code with different filter coefficients

Page 31: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Conclusions on ADC Clock Rate

• A Bunch-by-Bunch processing system must sample the raw waveform at a minimum of 4x the Bunch frequency

• You can never be:–too rich–too thin–or have too many ADC samples

Page 32: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

What is an FPGA?• Reconfigurable Logic Array ~106 logic gates• Pre-built logic subassemblies: “Megafunctions”

– Multipliers/Accumulators

– Multi-port RAMs

– Gigabit serial links

– Entire CPUs

– Phase Locked Loops

– Complex I/O pads

• More transistors than a Pentium• Impressive Support software

Page 33: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

XYLINX And

ALTERAAre theIndustryLeaders

Page 34: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004
Page 35: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

What is an FPGA Good At?• Big Synchronous Arithmetic Pipelines

– 400 MHz multiply/accumulators, filters..

• High-Speed Interface with Modern Parts– ADCs, DACs, Serial Links

• Built-in system diagnostics– “Digital Scope on every signal”

• Flexibility and Multiple Applications– Use one board design for many applications– Add features without hardware changes

Page 36: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

SYNCHRONOUS PIPELINES

• When people say “Analog is simple”, they are often referring to the deterministic execution time (“propagation delay”).

• Analog circuits never fail to respond in time because they are off servicing an interrupt.

• FPGA synchronous pipelines provide dedicated logic which responds at a deterministic time.

• This captures a big advantage of Analog.

Page 37: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004
Page 38: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

FPGA Programming Languages:Graphical vs. Text-mode

• Graphical Schematic Entry is useful for:– diagramming data flow– giving talks

• Text Mode features – Text is faster to enter and more concise– Can “diff” two files to see what’s changed– Code management systems can handle text well

I PERSONALLY RECOMMEND TEXT MODE

Page 39: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

FPGA Programming Languages:Proprietary vs. Industry Standard

• Proprietary languages often lock you into a single vendor.– I use one (Altera AHDL) anyway.

• The “industry standard” VHDL language– It is extremely verbose & repetitive– Translating AHDL into VHDL increases the text length

by a factor of ~2.

LIKE TRANSLATING A DOCUMENT TO FRENCH

Page 40: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Some Development Models

How do you Download & Talk to this Board that you’ve just built?

1. FPGA Programming cable

2. Firmware Serial Port Model

3. Crate Backplane Bus Access

4. On-Board CPU w/Ethernet

5. Compiled-in On-chip CPU w/ Ethernet

Page 41: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

CPU Access to FPGA Registers

• Usually want ADDRESS/DATA R/W model for CPU access to Control Registers

• These “address and data busses” are synthesized in firmware

Example (AHDL) for 32-bit read/write register:

(Bus[],Outputs[]) = BUS_REG( ) WITH(

ADDRESS = H“08402020”,

WIDTH = 32 );

Page 42: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Development Model #1: FPGA Programming Cable

• The programming cable needed to program the FPGA (usually through the PC printer port) can also be used for limited communication

• Not clear how useful this is for real-time response since it works through serial port driver

• Altera provides “compiled-in logic analyzer” which provides output that can be compared with simulation.

Page 43: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Development Model #2: Crate Backplane Bus Access

• Crate Backplane Bus connections to FPGA can provide CPU access to registers in internal address space of FPGA

• Internal Address Space is defined in firmware

• Requires many bits of bus buffers, etc. • Be Careful of Backplane Noise (TTL)

Page 44: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Development Model #3: Serial Port to PC

• A firmware-defined Serial Port can be used for 2-wire communication with the .COM port of a PC

• Terminal emulator can provide simple read/write access the internal address space of the FPGA

• Can also connect to spread sheets, etc via Visual Basic access to .COM port

Page 45: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Development Model #4: On-Board CPU w/Ethernet

• “Postage Stamp” Ethernet CPU or homebuilt DSP can provide Ethernet and Web access to FPGA registers

• Firewire is an alternative

• Remote update of firmware possible

• NIM-like modules without need of crate backplane

Page 46: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Development Model #5: Compiled-in On-chip CPU with Firmware-defined Ethernet

1. High-end end FPGAs have built-in or firmware-defined CPUs fast enough to support IP stack, Web Servers, etc.

2. These are available on Demo Boards3. C-language programming of these is

integrated into FPGA development environment (no new software!).

Page 47: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Adding a new ACNET Device

Takes about 10 minutes from concept to Fast-Time Plot

1) Add register(s) to FPGA Firmware

2) Start Recompile (takes ~6 minutes)3) Meanwhile, use DABBEL/D80 to define

properties of new ACNET device4) Download Firmware & Reboot Crate (~2 min.)

Page 48: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Application #1: Universal BPM(Beam Position Monitor)

• Measures position of each bunch on each pass around the ring with full-bandwidth FIR filter

• (R-L)/(R+L) for each bunch measurement.

• Multi-bunch averages available for lower noise

– per batch, per turn, many turns, different bandwidths

• Multiple users can share hardware w/o conflicts

– ADC is always active, FPGA stores data many ways

Same Hardware OK for Booster, MI, RR, TeV, & beamlines.

Page 49: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

FPGA Based “Universal BPM”

Monster FPGA(s)

53 MHz, TCLK, MDAT,...

MinimalAnalogFilter

ADC

Split Plate Pickup #1

MinimalAnalogFilter

FASTDAC

14 CPUVME/VXI/PCI/PMCEthernetetc.

MinimalAnalogFilter

MinimalAnalogFilter

212 MHz

Pickup #4

Analog Position MonitorTest Point (Optional)

ORModulation Output for Synchronous Lock-in Detection Technique

ADC

ADC

ADC

......

R

L

T

BSerial

Link toReal

TimeOrbit

ControlSystem

Page 50: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

“Universal BPM” Application: Signal Processing Steps

1) Bandwidth-Limit input signal to ~53 MHz2) 12 Bit Digitization at 212 MHz3) FIR filter(s) to get single-bunch signal(s)4) Sum & Difference of plate signals5) (Difference / Sum) gives position6) Linearization lookup table or polynomials7) Bunch, Batch, Multiturn Averaging8) “Scope Trace Buffers” on every signal

Multiple users can be acquiring and filtering data multiple ways without conflicts

InsideFPGA

FrontEnd

Page 51: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Main Injector BPM Response Map

Linearization can be done in FPGA or readout software

J. Crisp

Page 52: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

“Universal BPM” Signal Processing Step #7: Averaging and Filtering

Many Types of averaging possible:Position Averaging over Bunches in a BatchMulti-Turn Averaging of Positions

Multi-turn averaging of Raw SignalsFitting to betatron frequency (injection errors)

- this gives info for -function measurementEmulation of DDC chip functionsSpectrum analysis of position & phase

Different filters can be simultaneously

active

Page 53: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

MAIN INJECTOR VERTICAL BPM (8 Bits)

DIGITAL DAMPER POSITIONSIGNAL (Batch Average)

1mm

Page 54: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Single-Bunch BPM Measurement was tested by blowing out nearby bunches during Stacking Cycle

Page 55: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

MAIN INJECTOR VERTICAL BPM (8 Bits)

DIGITAL DAMPER POSITIONFOR SINGLE 53 MHz BUNCH

SINGLE-TURN (non-averaged)

1mm

BPM Resolution for 212 MHz Digitization of Single 53 MHz Bunch

Page 56: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Multi-User Support• FPGA CODE SUPPORTS:

– 31 different users on different machine cycles

– Different averaging algorithms, simultaneously active

– Each user can sit and observe a different single bunch

– Different bunch frequencies on each cycle

No User Interferences since Separate Dedicated Logic is used for each purpose

Page 57: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Application #2: Generic Instrumentation Readout Scope

What we want in a “Generic ‘Scope”:

1) Ability to trigger on TCLK events, Beam Synch Events, analog threshold crossings of different channels, etc.

2) Multiple Users Sharing without conflicts

- separate copies of trigger logic

- separate buffers to store captured signals

- separate filter algorithms run simultaneously

3) Common hardware & software among systems

Page 58: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Example Application of Generic ‘Scope: Flying Wire PMT Readout

Monster FPGA

53 MHz, TCLK, MDAT,...

MinimalAnalogFilter

FASTADC

PMT(s) in Tunnel

MinimalAnalogFilter

FASTADC

DAC

14

CPU BusVME/VXI/PCI/PMCetc.

MinimalAnalogFilter

FASTADC

MinimalAnalogFilter

FASTADC

106 MHz

Encoder Signals

MotorMotor Drive

Page 59: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Example Application of Generic ‘Scope: Flying Wire PMT Readout

• Photomultiplier Tube (PMT) pulses presented to Analog filter to limit BW

• Summing circuits in FPGA give total PMT pulse height in narrow and wide gates

• Individual gates report signals for 36x36 or more bunches, average over many turns, etc.

• FPGA can be used to control & trigger the fly

• Raw PMT pulses can be simultaneously looked at via “multi-user” hooks

Page 60: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Application #3: God’s Own Beam Loading Compensation

1) Digital Pipeline to reproduce I&Q signals from RW bunch monitor with N-turn delay. (N=1...1/S)

2) Digital filters for transients and synchrotron osc.

• Inputs: Resistive-wall monitor & RF fanback.

• Digitization: bunch-by-bunch I & Q signals

• Outputs: I&Q to damper cavity, or LLRF– frequency swing issues for LLRF drive

– Antiproton vs. Proton timing

Page 61: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Application #4: “Universal Damper”

• A single FPGA has enough capability to do damping calculations for X,Y, & longitudinal.

• Digital Filter which operates on I & Q signals from individual 53 MHz bunches can also be reprogrammed to operate at lower frequencies.

• Frequency swing during acceleration introduces some timing complications, which can be fixed by components (FIFOs, Dual-Port RAMS) inside of FPGA.

Page 62: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

“Universal-Damper” Application: Signal Processing Steps

1) Bandwidth-Limit input signal to ~53 MHz2) 12 Bit Digitization at 212 MHz3) FIR filter to get single-bunch signal4) Sum & Difference of plate signals5) Multi turn difference filter (FIR) w/delay6) Pickup Mixing for correct Betatron Phase7) Bunch-by-bunch gain, dead band etc.8) Timing Corrections for Frequency Sweep9) Pre-Emphasis for Kicker Power Amp10) Power Amp for Kicker

InsideFPGA

Front End

Buy

Page 63: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Longitudinal Beam Instability in FMI

• Driven by cavity wake fields within bunch train

First Bunch ~ OK 7th Bunch Trashed

• Occurs with 7 bunches filled (out of 588)

• Prevents low emittance bunch coalescing

Page 64: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Longitudinal Damper FPGA Logic

ADC

ResistiveWall Pickup

14

8-Turn FIRcalculates

derivative of bunch phase

Bunch-by-Bunch Digital Phase Detector

+THRESH

-THRESH

Multi-TurnMemory

Bunch IntensityFIR Filter

+THRESH

+/- KICKto DAMPER

Individual Bunches are kicked + or – depending on whether they are moving right or left in phase

option (currently unused)

Page 65: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

FPGA Code for Universal Damper (8-turn Filter)

Page 66: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Transverse Damper 3 - Turn Filter

• Damper kick is calculated from single BPM position reading on 3 successive turns.

BPM

KICKER

Arbitrary Betatron Phase of Kicker can be accommodated

Page 67: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

HERA-P Damper uses a 3-turn Digital FIR Filter

• Digital Bunch by Bunch @ 96ns Spacing

• Immediate digitization following peak detection

Klute,Kohaupt et. al. EPAC ‘96

Page 68: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

3 Turn Filter Coefficients

• Damper kick is weighted sum of beam positions on the 3 previous turns.

• 3 Filter Coefficients Uniquely Determined by:– System Gain– Betatron Phase Desired at Kicker– Constraint that sum of filter coefficients = 0

(so that filter does not respond to DC offsets.)

Page 69: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Frequency Sweep Issues• Machines with frequency sweep (Booster!)

must adjust ADC input clock and DAC output clock phases as frequency sweeps.

• This can be generated with Phase-locked loops and delay-locked loops present inside FPGAs.

• This requires access to both the RF clock, and a cable-delayed version of the RF clock, as timing references.

• One-turn digital delay using FIFOs in FPGAs.

Same hardware can be used for Booster thru Tevatron.

Page 70: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

RF Clockingwith acceleration

Equal Length Cable Fanoutso Beam Sees Same RF Phaseat all Cavities as RF Frequency Sweeps During Acceleration # of Clock Cycles

per Turn is harmonic number h

N/2 L

BEAM

RFClock

Equal LengthFanout Cables

RF Cavity

RF

Page 71: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

ADC Clockingduring frequency sweep

Round-Trip Cable Delay on ADC Clock ensuresADC Clock & Beam InputStay in Phase asBeam Accelerates

May need additional phase adjustment to track phase jumps at transition, etc.

N/2 L

RFClock

BeamPickup

ADC ClockDelay Cable

Clock

INADCSignal Cable

RF

Page 72: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

DAC Clockingduring frequency sweep

Propagation Delay: CK DAC Cable Kickershould match RF Fanout Delayso Kick Stays in Phase as Beam Accelerates

N/2 L

RFClock

RFClock

DAC OUT

BEAM

KICKER

Cable

Page 73: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Generic Dampertolerating frequency sweep

FIFO needed due to phase shifts between DAC and ADC clocks as beam accelerates

RFClock

BeamPickup

ADC ClockDelay Cable

Clock

INADCSignal Cable

Clock

DAC OUT

BEAM

KICKER

Cable

PipelinedDigital Calc.

FIFO

CK

All LogicInsideFPGA

Page 74: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Damper Output Precompensation

• 53 MHz Bunch-by-bunch kicker wants a 19ns square pulse with a good flat top to minimize timing sensitivity

• Power Amp and Cable have non-ideal response for square wave (ringing and tail)

• DAC operating at 424 MHz is used to produce specially sculpted pulse necessary to convince Amp & Cable to make a nice flat pulse at kicker.

Page 75: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004
Page 76: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Echotek Card Used for Initial Dampers

105 MSPS AD6645

•Echotek Board Originally Built to SLAC Design Specification•65MHz DDC version to be used for RR BPM upgrade•105 MHz version (with DAC “daughter card”) used for Dampers

212 MHz DAC Daughter Card

(S. Hansen/ PPD)

Page 77: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Butchering the Echotek Board• Scorched-Earth FPGA rewrite (GWF)

– ~65 pages of firmware since Jan ‘03

• 212 MHz DAC “Daughtercard”– Sten Hansen & T. Wesson (PPD)– 3 channels for X,Y,Z

• 212 MHz Output FIR (W. Schappert, RFI)– Pre-emphasis compensation for analog outputs– Prototype for 424 MHz output on final board

• Input Buffer Amp/Splitter Box (Brian Fellenz,RFI)

Page 78: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004
Page 79: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Multi-batch w/o and with transverse dampers

with dampersw/o dampers

1 to 11 Booster turns2.31013

@ 8.9 GeV/c: X=26.43, Y=25.42, X=-20, Y=-16@ 11.7 GeV/c: X=26.39, Y=25.425

@ 8.9 GeV/c: X=26.44, Y=25.47, X=-5, Y=-5@ 11.7 GeV/c: X=26.36, Y=25.46

9 Booster turns

Page 80: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Pushing up the intensity…

13-14 Booster turns

3.31013

Beam extracted at 0.65 s, just before transition

@ 8.9 GeV/c: X=26.43, Y=25.47 X=Y=-5@ 10.3 GeV/c:X=Y=-5@ 11.7 GeV/c:X=26.36, Y=25.46

Transverse Dampers ON

Page 81: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

What if… I turn dampers off ?

turn dampers off

intensity

horizontal vertical

Page 82: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

DON’T TELL BOB MAU

(or the rest of the green-yellow

color blind operators)THAT WE DID THIS

Page 83: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Filter for Undamped, Damped, and Anti-Damped Bunches

Page 84: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Blowing Selected Bunches out of the Machine (in X,Y, or both)

Neutrino Communications! …1110111001110001111…

Page 85: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Application #5: “LLRF System on a Chip”

• The single FPGA on the damper board has 12 Phase-locked loops and enough capability synthesize all the signals needed for a complete LLRF system.

• The Damper Board has already been used (in a simple way) to drive the FNAL Debuncher Ring LLRF system.

• A more ambitious (but achievable) goal is to replace the entire Booster LLRF with a Damper.

Page 86: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

MOTIVATION

Booster Low-Level RF.

The Final Frontier.

Page 87: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Booster Low-Level RF

Page 88: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

2. RPOS

3. TCLK

1. WCM

5. MI RF4. BDOT

A&B DRIVE OUT

Notching and Cogging

Page 89: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

Booster LLRF External Connections• ~5 Inputs:

1. Wall-Current Monitor (Phase)2. Transverse Pickup (RPOS) (BNL Uses two…)3. Start Pulse (TCLK)4. BDOT (Low bandwidth… replace w/lookup?)5. MI AA Marker (Phase lock & notch cogging)

• Two Outputs: Cavity A&B Drives• (Optional?) Beam Clock Output

Page 90: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

TCLK, 53 MHz, MI AA, MDAT,...

Digital Booster LLRF Concept

Monster FPGA

DDS Beam Synched Clock 160-212 MHz

(4x Booster RF)

MinimalAnalogFilter

FASTADC

WallCurrentMonitor(PHASE)

FASTDAC 12

12

ETHERNET

MinimalAnalogFilter

FASTADC

BPM

MinimalAnalogFilter

FASTADC

12RPOS

[crystal] 400 MHz

FASTDAC

“A” Drive

FASTDAC

“B” Drive

12

Page 91: A Unified Approach to Fast Digital Processing for Beam Dampers, Instrumentation, & Controls Bill Foster Beam Instrumentation Workshop May 6, 2004

CONCLUSIONS• Fast ADCs and Huge FPGAs are

revolutionizing Accelerator Instrumentation

• The same basic hardware can perform a large

number of Instrumentation & Control functions

• A good first application of this technology is

the 3-coordinate bunch-by-bunch beam damper