areaDetector workshop

Summary and notes…

Overview

• iocBuilder• File Writer Changes• areaDetector driver pattern• shared vector• Asyn client without an IOC• Parallel detectors• V4 NTNDArray and areaDetector• AreaDetector at NSLS-II• ADC support• Spectroscopic detector support• Python• gstreamer

iocBuilder

• Demo of iocbuilder and guibuilder• Discussion of sharing these tools:– Guibuilder is portable– iocBuilder is not very portable – but could possibly be

made so with some effort• There is interest in making this public• ACTION: Tom to put DLS tags in AD templates and

issue pull request• Decision: move to putting autosave tags in the info

fields

File Writer Changes• Proposal to simplify the file writer base class NDPluginFile from having 3

modes to just one mode (i.e. “stream” with some additional functionality to support the other modes use-cases)

• MR propose to remove the AutoSave mode as it can cause unintentional saving of large amount of data

• These proposed changes require a client interface change and should be deferred to next major release version

• Discussion about how DLS Pilatus driver can delete temporary Pilatus data files. – DLS to investigate if we have forked code on Pilatus or areaDetector

• Discussion on the location of parameter definitions in the class hierachy– The file-releated parameters appear to be defined too deep in the base classes– Tidying this up is basically a major re-ordering of the class hierachy– Not finding a parameter in a list should not return 0 as this is also a valid parameter ID

• File reader discussion: a reader should really not be implemented in a writer class– Proposal to remove the readfile() virtual functions

• Buffering plugin to be added to ADCore– ACTION: DLS to add the “scope” plugin

AreaDetector driver pattern (Jon)• Jon showed a number of patterns that he re-use in areaDetector drivers:• Lock taker object which automatically release the lock when going out of

scope– MD suggest: base has a MutexGuard which does the same– Scope can be defined by simply adding a couple of { scope here… }

• NDArrayRef class– NTNDArray from V4 uses MD’s shared_ptr to basically achieve the same– Discussion (again) about NTNDArrays: areaDetector need these to be

provided/implemented by V4…• Statemachine pattern demonstrated• Another Boost discussion – it looks like this is the way the world is moving

– V4 uses boost– Issue with boost support on VxWorks

• Parameter callbacks demonstrated – similar to MR proposal to not use a single callback to writeInt32() with a large switch/case statement

• Driver classes grow to large (to fit in a single class)– Demonstrating the PCO example where he split into a frame-grabber and control

class

shared vectors (Michael)

• MDs shared_ptr class presented• This is used heavily by the V4 implementation

libraries – lives in pvData• Could be used to replace the internal void* pData in

areaDetectors NDArray• V4 NTNDArray implementation uses this internally• Large and opinionated discussion regarding

development in:– C++ (asynPortdriver) vs. database vs. client side scripting

Asyn Client without IOC (Mark)

• Overnight, MR hacked up a demonstration of an asyn client which can be instantiated in a simple C/C++ program without the need for an IOC

• Demonstrated the simple instantiation of a simDetector and a statsPlugin– Monitoring and manipulating a couple of parameters of each

object• Parameters are created as stand-alone “asynClient” objects

– UP and JT suggest that these should be available from the driver/plugin object – but this requires some introspection of the asynPortDriver

– JT to look into creating parameter introspection in the asynPortDriver

• Can this be wrapped in python?

Parallel detectors (Ulrik)

• Presentation of the parallel Excalibur and Percival detector use-cases

• Presenting the new features of the HDF5 library and file format – and how it helps:– Single Writer Multiple Reader (SWMR)– Virtual Dataset (VDS)

• Discussion about how to implement the parallel live processing

V4 NTNDArray and integration with areaDetector (Dave)

• The existing plugin (producer of data on a V4 channel) and driver (consumer/monitor of a V4 channel) is being put together on github

• Question: how to replace NDArray with NTNDArray?– Does V4 libraries implement a suitable full NTNDArray class?

• Options for replacing the existing transfer mechanism between driver-plugin with V4 service and monitor– 3 options presented (see slides)

• Discussion: dropped arrays– In V4 there is just a single “dropped array” bit set– areaDetector requires counting the dropped frames – could be

implemented on the areaDetector side• Discussion: how to deal with multiple plugin subscribers to a

single ‘driver’ (or producer) without transmitting multiple copies?

areaDetector at NSLS-II (Michael)

• First light on beamlines to use areaDetector “out of the box”

• Future plans include more advanced service architecture

• Discussion about when to do processing live or after acquisition (post)– Should areaDetector provide general plugins which can

be re-used in post-processing?– If plugins can be instantiated in a stand-alone

application then it can be re-used in other applications

ADCs (Tom)

• Tom presented his work with ADCs (BPM) and live processing (FFT and other) in plugins

• Proposal (TC): create a new ADC base class to cater for this particular type of detector– ADC detector support does not share much with MCA

type detectors due to it’s streaming nature• Discussion about how best to chunk streaming ADC

data for the various types of processing required• ACTION?

Spectroscopic Detector support

• Device that produces 2D data– one dimension for detector element, one dimension for

channel– MCA record is 1D (single channel) but xmap/xpress inherently

2D• How much does one do in a record, or in a plugin?

– Mca record has 32 ROIs (similar functionality as AD ROI plugin)

– Xpress3 also has ROI code, could be used in MCA• Some indication that it may be possible to retire the mca

record in favour of similar functionality in AD plugins

Python and areaDetector (all)• UKP presented adPython (by TC)

– Allows users to implement processing plugins in pure python (python interpreter embedded in IOC)

– adPython uses no 3rd party wrapper technology: simply uses the python and numpy C interfaces

– adPython to be put out on github– Potential issue: single interpreter with a global lock – don’t instantiate

lots of these plugins in a single IOC!• UP would like to see python wrappers for plugins and drivers: to

allow running areaDetector elements in a python script– Lima (Tango) has this functionality– MK is interested in this and may be able to help eventually

• Asyn would need to support proper destruction (asynPortDriver?)

Gstreamer (Tom)

• Tom presents and demonstrate the use of gstreamer (out of the gnome project)

• Demonstration uses simDetector to produce some data – and V4 transfer to publish this data

• A V4 client monitor application passes the NDArrays into “gobjects”– allow using the vast range of gstreamer processing elements

• TC compared the features of gstreamer and areaDetector• Gstreamer plugin for areaDetector could be used to

replace ffmpeg