The Use of LabVIEW FPGA in Accelerator Instrumentation.download.ni.com/pub/devzone/tut/lv_fpga_at_sns_ornl.pdf · The Use of LabVIEW FPGA in Accelerator Instrumentation. ... We have

The Use of LabVIEW

FPGA in Accelerator

Instrumentation.

Willem Blokland

Research Accelerator Division

Spallation Neutron Source

OAK RIDGE NATIONAL LABORATORY

U. S. DEPARTMENT OF ENERGY

NI Week 2010, Aug 2nd 2010, Austin, TX,

Introduction

Spallation Neutron Source at Oak Ridge National Laboratory:

$1.4B facility using a 1 GeV proton beam to generate pulsed neutrons

Built by a collaboration of five national laboratories

Neutron scattering to study materials

2




Introduction

The Beam Instrumentation Group at the Spallation Neutron Source is responsible for measuring various attributes of the particle beam, such as losses, electrical charge, and position.

Many instruments can and are implemented with a general OS (XPe). However, some measurements must be done consistenly and quickly (16ms) to protect the machine or may require very fast reaction times (10 µs)

We began using FPGA on NI Hardware as well as LabVIEW FPGA for various instruments:

Beam Accounting systems that use FPGA and LabVIEW but was directly programmed with the Xilinx tools (fast charge calculator and differential monitor)

A Beam Loss Monitor that was programmed in LabVIEW FPGA on cRIO

A new version of the Beam Accounting system, using FlexRIO and LabVIEW FPGA

3




Spallation Neutron Source Beam Pulse

At SNS:

The beam pulse repeats at 60 Hz

Beam is chopped at ~1 MHz

RF is 402.5 Mhz and 805 MHz

4

mini-pulse

Chopper system makes gaps

16.6 ms

Cu

rren

t1ms

Accumulated in

the Ring

945 ns

Cu

rre

nt

1 ms macro-pulse

2.5 ns

Cu

rre

nt

Cu

rre

nt




Accelerator Beam Instrumentation Features

5

400+ PCs

16+ VME systems




Accelerator Beam Instrumentation Features

A beam instrument’s function can be one of the following:

Measurements to a studier: Is my accelerator setup better or worse?

• Only needed during studies

• Interactive (1s)

Measurements to an operator: Is the accelerator operating within specifications?

• 24/7, Needed during production

• Interactive (1s) and/or automatically monitored

Measurements for beam accounting: What are the statistics?


• Automatic (<16.6ms)

Measurements to the abort system: Is something going wrong?


• Automatic either with alarm or direct hardware connection to abort system (e.g. abort beam in less than 20µs)

6




Example of Beam Instruments

SNS Wire Scanner (PC, LabVIEW)

Steps a wire through the beam

Tied to a max beam rep rate of 1Hz to protect wire

Only used when performing a study

Single failures allowed

Beam Position (PC, LabVIEW)

Measures the position of the beam pulse

Only needed at 1Hz (beam rep rate is 60Hz)

Digitizes at 40Mhz

Single failures allowed

Beam Loss Monitors (VME, VxWorks)

Measures beam losses to protect machine

Software must run at 60Hz

Analog hardware to abort beam within 20 µs

No failures allowed (exceptions can be made)

7

Fork

BPM Plates

Neutron detectors




Accelerator Beam Instrumentation Types

8

Machine

Downtime

µs s

Now and

then

Always

Response time

Mostly

Consequences

Damage

Time lost

ms

Win, Mac, Linux,…

VxWorks,

LVRT, PLC, …

Circuits, FPGA,

DSP, …..

Uptime

Hardware

Real-Time

Non RT

standalone

Non RT

interactive




How Do We Implement An Instrument?

We are interested in the fastest and cheapest way to implement a reliable instrument

Small group can’t afford to know everything

• Budget can be used to buy time from contractor with needed expertise

• Work with other groups within division

Have to work with different parts from different vendors

• Real-time OS, Compilers, FPGA Board, Digitizer, Slot-zero controller, Backplane, Amplifiers

Maintenance

Solutions

If deployment is large enough and time is available, then learning curve is not as significant and in-house hardware can be developed

Buy turn-key system

Buy off-the-shelf parts and integrate to complete

9

Considerations

Cost

Available personnel

Development time

Reliability

Performance

Obsolescence

Maintenance




Use of FPGA in Accelerator Instrumentation

These are unique instruments, one or two copies of each at most. We have used LabVIEW FPGA for various small projects, such as:

Laser position feedback loop

Pulse delay generator

Accelerator Instruments:

Pulse-by-Pulse Beam Charge Monitor (PBCM)Calculate the charge of a pulse and transmit digital value over optical fiber within 2-3 ms to Target Hall Instruments

Differential Beam Current Monitor (DBCM)Subtract the signals of two current transformers and determine if the difference is too large (5, 50, 500 µs sliding windows). Abort beam within 5 µs.

For these instruments LabVIEW FPGA was not available with a fast enough digitizer. We choose to use COTS parts (NI PCI-5122EX), but involve a contractor (Maverick Systems) to write the Xilinx FPGA code.

10

Beam pulse




PBCM Setup

11

PXI crate with 5122EX

PC with EPICS and PCI Timing card

To target

instruments

Toroid in tunnel




PBCM Singal Processing

12

The FPGA on PXI version ofdigitizer calculates independentlyfrom LabVIEW and OS the chargeand beam pulse edge.Processing of each channel includes:• Baseline correction,• FIR/IIR filter or no filter or test pattern,• Charge integration, and• Detection of beam pulse edge.

Fiber-optic Transmission includes:• Transmission of the charge and begin

of beam pulse over two digital outputs,and

• Electrical to optical conversion (Opticalconverter box by R. Riedel) to transmitvalue to the Target Group forcorrelation with neutronmeasurements.




PBCM Use For Commissioning

13

To target

instruments




DBCM Setup

14

PXI with EPICS and 5122EX card

Sliding window traces

Beam with 5% beam loss (zoomed)

Chopped Beam

Red trace is

differenceDifferential Beam Current Monitor




Flow Chart Of DBCM Signal Processing

15

The FPGA on PXI version of

digitizer calculates

independently from LabVIEW

and OS the difference between

two current monitorsProcessing of each channel

includes:

• Baseline correction,

• FIR/IIR filter or no filter or test

pattern,

• Charge integration (5, 50, 50 µs)

Abort includes:

• Digital Out to Machine Protection

System




Experiences with 5122EX (PBCM CBCM each)

Contractor

9-12 months (not full time)

Labor cost ~$85k

Learning Curve

Taken care of by contractor

Hardware

$17k

Specification

2 weeks (within group: no cost)

Integration to Control System

2-3 weeks (within group: no cost)

Upgrades

Any change would require outside help as expertise is not within group -> high cost and time overhead

16




Beam Loss Monitor

One unique instrument for use in a test facility. Similar to a standard Beam Loss Monitor, but resources to modify existing systems were scarce. cRIO was identified as viable alternative.

RF Test Facility Beam Loss Monitor Specifications

Sample 12 channels at 100kHz

• Baseline correction, calibration, and integration

Alarm in 20usec when integrated beam loss is above threshold

Control and read back HV power supplies voltages

Send test pulse to test Photo Multiplier Tubes

17

Feature VME cRIO

Sample rate 100kS/s 100kS/s

Resolution 24 16

Alarm Analog FPGA

Analog Channels 16 12-16

Hardware Cost* $29k $12k-13k* CPU, Crate, Power supply,

ADC, DAC, DIO




Beam Loss Monitor cRIO

18




Beam Loss Monitor Diagram

19

Impressive:

Adding test feature required minimal code and development time




Beam Loss Monitor cRIO

20

BLM IOC CPU with EPICS

BLM Amplifiers

BLM cRIO

HV Power supply

RF Cavity Video System PXI crate




Beam Loss Monitor VME Rack

21

Analog front-end

VME Controller,

Digitizer, Digital IO,

Custom Timing Card

Custom Analog Card

Abort System Modules

Power Supply

HV modules

Analog Front-end




Experiences with cRIO-based BLM

Contractor

3 months part-time (to get started)

Labor cost ~$7.5k, also covers LabVIEW RT setup

Learning Curve

Sample-by-sample processing versus standard

Setting up LabVIEW FPGA

Hardware

$12k.

Specification

1 week (within group: no cost)

Integration

1 week (within group: no cost)

Upgrades

Easily done for small changes ~1 day

22




Comparison

BLM VME programming versus LabVIEW FPGA & RT

Sample-by-sample processing fitted requirements very well and got rid of issues such as

• Sequential processing bottlenecks: Waveform acquisition takes 2 ms, then the transfer to slot zero, then processing. This must fit within 16.6msec. Balancing act between digitizing speed, backplane data transfers and CPU processing. FPGA can process data sample-by-sample without data transfers.

• Separate hardware board to perform Abort functions. An analog board with integrator and analog threshold is used to implement an abort within 20 µs. FPGA can set alarm in each 10 µs cycle.

cRIO Hardware integration takes care of low-level drivers

• Our project requirements matched very well with cRIO capabilities and no FPGA hardware knowledge was required.

VME approach is standard in accelerators

• Custom timing decoding card and software is available

• Many groups (Controls) have experienced people

• Floating point math

• Multi-year project (multiple labs)

23




FlexRIO

FlexRIO is now available and the Differential Current Monitor can now be implemented with LabVIEW FPGA. The goal is to upgrade the DCM (change trigger behavior and simplify interface) and improve maintenance but stay with a 100Mhz single cycle loop (aka 1000x faster).

The implementer is a student in the Physics Bachelors Program with some LabVIEW background.

NI provided with a proof of principle example program bringing us 90% to completion (Of course, the last 10% is at least 90% of the work)

NI Support during development

24

Courtesy of Mariano Padilla

Integration Windows




Experiences Using FlexRIO at 100MHz

Student

2.5 Man-Months in 4 months, about 0.5 MM to complete

Labor cost <$10k

Learning Curve (~60%)

Point-by-point processing versus standard

FlexRIO details

Hardware

$14k.

Specification

1 week (within group: no cost but time)

Integration

Estimated at 3 weeks (within group: no cost but time)

Upgrades

Easily done for small changes (~1 day)

25




FlexRIO Experience

26

Experiences with 100MHz single cycle loop:

Detailed knowledge of LabVIEW FPGA implementation is required:Number of pipeline delays

Fixed point math and exact conversion details

Detailed knowledge of FlexRIO hardware is required:Being able to estimate most likely utilization of FPGA resources

Knowing how DDR memory is interfaced: how many channels/timing

No Xilinx expert required

Iterate many times to complete project




FlexRIO Experience

27

When performance requirements fit well below the maximum performance of what LabVIEW FPGA and hardware supports, you can expect fast development with a good hiding of the underlying technology.

When challenging the performance, development becomes slower and more complex.

BUT

It is an integrated development environment by one vendor

Can cover at least up to 100Mhz




Comparison

28

Xilinx cRIO FlexRIO

Processing Speed 100 MHz 100 kHz 100 MHz

Project Length 12 Months 3 Months 6 Months

(Learning)

Expertise VHDL

FPGA DSP

Hardware

FPGA DSP FPGA DSP

Hardware

Labor/Hardware

Cost ratio

5:1 1:1 1:1 (Student)



NI Week 2010, Aug 2nd 2010, Austin, TX, 29

Future LabVIEW FPGA Based Diagnostics

Instrumentation

Systems with 100kS/s – 1MS/s, reaction within 100 µs, rep rate of 60 Hz:

BLM

Charge/Power Measurements

Systems with > 1MS/s, reaction within 10 µs, rep rate of 60 Hz:

Pulse-by-Pulse BCM

Long Term:

Wire Scanners

Harp

Image Analysis

Damper (1GHz)




Conclusion

30

LabVIEW FPGA has enabled us to cover a wider range of instrumentation

developed and supported within our group

LabVIEW FPGA FlexRIO

LabVIEW FPGA cRIO

Machine

Downtime

µs s

Now and

then

Always

Response time

Mostly

Consequences

Damage

Time lost

ms

Win, Mac, Linux,…

VxWorks,

LVRT, PLC, …

Circuits, FPGA,

DSP, …..

Uptime

Hardware

Real-Time

Non RT

standalone

Non RT

interactive

Documents

The Use of LabVIEW FPGA in Accelerator Instrumentation.download.ni.com/pub/devzone/tut/lv_fpga_at_sns_ornl.pdf · The Use of LabVIEW FPGA in Accelerator Instrumentation. ... We have