Practical Uses of Python and Epics for Mortals

Beamline Controls with Epics and Python

Why Python is worth learning and using for both experts and occasional programmers and scientists:

Clean Syntax easy to learn and remember.

Matt Newville, CARS, Univ of Chicago

High Level Language no pointers!! automatic typing of variables.

Free (both senses) good documentation and books (at bookstores!).

Interpreted Language no compile step: code runs directly from source.

Cross Platform works on Unix, Windows, Mac: code is portable.

Many libraries included or easily downloaded and installed:

Extensible with C, C++, Fortran, Java, COM for example, Epics Channel Access

Object Oriented or not – your choice. Also: good ‘Name Spaces’

Numeric (array math), Image Processing, Email, Databases (MySQL), Web, CGI.GUI toolkits: Tk, WxWindows, Qt, Gnome,….

Epics Channel Access: Why Python?? IDL and Ezca give a very easy-to-use interface to Epics Channel Access:

What Can Python do that IDL cannot?Send Email CGI Programs Connect to SQL databases(??)Create programs you can give to someone without an IDL license.

~>idlIDL Version 5.5 (linux x86). (c) 2001, Research Systems, Inc.

IDL> .run ezcaIDLIDL> caInitIDL> status = caget('13IDC:m15.VAL',x)IDL> print, x 76.349005

~>python Python 2.2.1 (#1, Aug 30 2002, 12:15:30)

>>> from Ezca import Ezca>>> ezca = Ezca()>>> print ezca.caget('13IDC:m15.VAL')76.349005127

Python Example 1: send email for alarms

## send email when we run out of Helium

import Ezca, smtplib

ezca = Ezca.Ezca()pv = "13IDA:eps_mbbi68"mail_to = "newville@cars.uchicago.edu"mail_from = mail_tomail_msg ="""From: newville@cars.uchicago.eduSubject: Out of Helium

Time to change the helium bottle..."""

value = ezca.caget(pv)

if (value == 0): s =smtplib.SMTP('millenia.cars.aps.anl.gov') s.sendmail(mail_from,mail_to, mail_msg) s.quit()

smtplib module: create and send email messages from your program.

Format the email text

Run this as a cron job, but may want to add a check that mail hasn’t already been sent…. Can also add checking status of many other PVs.

Check the PVs value

Send mail If the valuemeans the gauge is low.

Python Example 2: web form for beamline status

This form gives us access to beamline status information without needing to run MEDM (at home, offsite,….).

It uses the Python Ezca module to fetch the PV Values and the CGI module to generate and display the HTML when you access on the form.

It also ties into an mySQL Database of archived PV data – similar to the Epics Channel Archiver

Clicking on a link gives…

Python Example 2: web form for archived data

… a plot of this PV over the past 24 hours.

~1000 PVs are logged into an SQL database (mysql) when the value changes, with a polling rate of 1 minute (was 10min). Data goes back to 2001. Plots made by the Python Gnuplot module.

All of this (archiving, database interaction, web form generation) is in python, and with a regular apache web server. No C code. No undocumented data formats.

Epics, Python, Apache, and mySQL are each used for their appropriate functions.

Beamline Controls: XYZ microscope with video cameraWhat Can Python do better than IDL?

Both have GUIs for motor controls that are better than MEDM forms. Python can also fetch and display images from a Web Camera. I think the Python/Tkinter toolkit is better than IDL’s GUIs too.

For the GSECARS microprobe, we mount samples on a motorized XYZ stage, and view the same with a video camera fed to a video web server. This is a key user control.

When the user enters motor positions in this form (Tkinter, Pmw):

- don’t need to keep the cursor in the box.

- non-numeric values are silently ignored.

- values outside the motor soft limits are shown highlighted (right), and will not be entered.

Positions can be stored by name, and returned to by pressing one button.

Entering a position also automatically saves the video image, and writes the X,Y,Z positions to an HTML log file.

Beamline Controls: XYZ microscope with video camera

Clicking on the camera icon will ‘snap’ the current sample image, and prompt the user for a filename to save the JPEG image.

Motors can be moved with button controls, with step sizes from an editable drop-down list.

Configuration is kept in a window-style INI file, read with python’s ConfigParser module.

The HTML log file:

XYZ microscope, web video-camera capture

The x-ray beam on the YAG phosphor

Typical Sample Image

We’re still collecting much of the x-ray microprobe data with IDL (python version in slow progress), but the users take home a Python program to view their data.

Data Viewer for Data from Scan Record

Doesn’t use Epics, but is important beamline program.

Uses BltPlot for 2D plots, Image module to show map images, and Mark’s MED module to read XRF spectra. (May add spec data file support soon).

Uses py2exe to make stand-alone executable from python source, so user does not even need python. Distributed as a setup.exe installer.

Data Viewer: more screenshotsCursor options in BltPlot: one fixed crosshair, The other follows the mouse and keeps a live update of x,y position.

For 2D map data (X-Y scans), map images can be shown (below), and two elementCorrelations can be plotted (lower left).

Ezca Python Implementation: HistoryComing from ezcaIDL, I first made an Ezca extension for Perl, and then Python using the wonderful SWIG program.

I never cared much about the difference between Ezca and “Real” Channel Access. An easy-to-use “caget” and “caput” was (and is!!) most important.

But: I recently got convinced to want faster CA calls, and to rely on Monitors. So the EpicsCA PV class wraps CA with SWIG 1.3, similar to what Geoff Savage had done, but way easier to implement (at least for me!) and not as much like CA in C as like Ezca:

~>pythonPython 2.2.1 (#1, Aug 30 2002, 12:15:30)

>>> from EpicsCA import PV>>> p = PV('13IDC:m15.VAL')>>> print p.value76.449

Both p.value and p.get() will get the latest value.

Channel Access Monitors are used so that repeated get()s will only really fetch a new value when it has changed.

EpicsCA PV ClassFeatures of EpicsCA.PV class: -Can get or assign using pv.value as well as pv.get(), pv.put()

>>> p = PV('13IDC:m15.VAL')>>> print p.value76.449>>> p.value = 76.3

-Can get information about connection, PV type:>>> p1 = PV('13IDC:m15.VAL') # double >>> p2 = PV('13IDC:m15.DIR') # enum>>> print p1<PV=13IDC:m15.VAL, status=connected, count=1, type=double, read/write>>>> print p2<PV=13IDC:m15.DIR, status=connected, count=1, type=enum, read/write>>>> print p1.host_name‘ioc13idc.cars.aps.anl.gov:5064’

-Can get string representation of variables:>>> print p1.value, p1.char_value76.449 76.4490 # char_value reflects precision>>> print p2.value, p2.char_value0 Pos # char_value is Enum String

EpicsCA PV Class (continued)

>>> print p2.get_enum_strings()[‘Pos’,’Neg’]

-Will use Numeric arrays for array data, if Numeric is installed.>>> p3 = PV(’13IDC:scan1.P1PA’)>>> print p3<PV=13IDC:scan1.P1PA, status=connected, count=1000, type=double,read/write>>>> print p3.char_value, p3.use_numeric, type(p3.value)<array(1000)> 1 <type ‘array’>

-Can get a list of all Enum Strings for enum data types:

- put() can wait for record to finish (might take a while!!):

>>> p.put(72.0, wait=1)

-Can handle string waveform data (strings longer than 40 characters!).

>>> p4 = EpicsCA.PV('13IDC:EDB:dir')>>> p4.value '/cars5/Data/gpd_user/Mar03/templeton/basalts'

EpicsCA PV Class (continued)-Supports Monitor Callbacks: user-defined python function to run when a monitor occurs. This is most useful when updating values in GUIs.

Example: the callback function is defined with all keyword arguments. The default values may be reset with set_monitor(), or even check_monitor().

import EpicsCA, time, sys

def my_callback(pv=None, a=2, **kw): if (pv==None): return print “my_callback:”, pv.pvname, “ = “,\ pv.char_value, 'user var a = ', a pv = EpicsCA.PV('13IDC:m15.VAL')pv.set_monitor(callback=my_callback,kw={'pv':pv,'a':4})

# simple event timer loopwhile 1: try: if (pv.check_monitor() == 1): print time.ctime(), pv.pvname, pv.char_value time.sleep(0.2) except KeyboardInterrupt: sys.exit(1)

check_monitor()=1 onlywhen the value changes.

check_monitor() will execute the callback.

Set callback function and optional kw arguments with set_monitor()

Note: an instance of the pv is passed to the callback function

Define callback function

Ezca and EpicsCAEzca.py is now implemented simply as holding a list of EpicsCA.PVs.

That means the PV values are really cached, and only retrieved from the IOC when needed. But the same Ezca interface and all the code based on it still works too.

The whole set of Epics/Python codes (EpicsCA, Ezca, and CaChannel) installs with the ‘normal’ python distribution utilities:

~> python setup.py install

Only need to have an environmental variable EPICS_BASE pointing to the epics source base.

Binary installer available for Windows.

http://cars.uchicago.edu/~newville/Epics/Python

For source, binaries and documentation.

Work in Progress / Future Work

- Scanning Program in Python: using Epics for most Record interface, but not using the Epics Scan Record, as it is too restrictive and does not support enough detector types. Will scan from a workstation, not IOC, but use Epics Database as central location for scan commands (requires long strings).

- Improved XYZ stage control to use and manipulate captured image for defining maps.

- Viewing spec files from DataViewer.