areaDetector and GStreamer

Similarities and Differences

Tom Cobb

What is GStreamer

• Streaming media framework – audio and video• Many plugins available for data formats / data

manipulation• Higher level than just input / filters / output– Networking, audio/video mixed streams, auto data handling– Various options utilizing hardware accelerators

• Has been extended for use in some scientific communities

• Written in C with Python (and other language) bindings

What is areaDetector

• I hope I don’t need to say anything here...

What does GStreamer do that areaDetector does?

• Chains plugins for data processing• Options to run any plugin in its own thread• Image transformations and conversion (rotate, scale

and offset, ROI, debayer, overlay, color convert)• File and network I/O in various datatypes• Some stats functions• Rewirable chain on the fly• GigE cameras• Windows, Linux, MAC OS X

What does GStreamer do that areaDetector doesn’t?

• Some data formats (YUV is about the only one that is immediately useful)

• Some file formats (MJPEG, H264 multicast streaming)• Autoplugging chain based on data formats• A lot of plugins that we could make use of (opencv

functions, • Synchronization of different clock sources• V4L cameras• Could make use of audio processing for ADC data

What does areaDetector do that GStreamer doesn’t?

• Some data formats (Int32, Float32, Float64)• Some file formats (HDF5, netCDF)• Some stats functions• Most of our detectors• EPICS integration• vxWorks

Who else is using GStreamer?

• LIGO: gravitational-wave data analysis tools for 1D data (see tutorial)

• Military: various 2D sources, mainly Camera Link and CoaXPress framegrabbers from Active Silicaon, NI, DALSA, Matrox, EDT (see presentation)

• Software Radio: FFTs and other 1D data functions• Lots of A/V people...

What neat ideas does it have?

• GObject based, so written in C but get Python wrapper for free

• Message passing on bus• Video format (Caps) negotiation• Bins for groups of plugins with a bit of logic• Multiple clock sources (e.g. for stream

synchronisation)• Connections can be run in push or pull mode (e.g.

For limited rate outputs, or attribute readers)

Demo

• EPICS V4 monitor on NTNDArray producing data to a pipeline of GStreamer plugins