OBSERVATIONAL EVALUATION OF EVENT CAMERAS PERFORMANCE

IN OPTICAL SPACE SURVEILLANCE

Michał Żołnowski(1)

, Rafal Reszelewski(2)

, Diederik Paul Moeys(3)

, Tobi Delbruck(3)

, Krzysztof Kamiński(4)

(1) 6 Remote Observatories for Asteroids and Debris Searching (6ROADS),Godebskiego 55a, 31-999 Kraków, Poland,

michal.zolnowski@6roads.com.pl (2)

Koszalin University of Technology, rafal.reszelewski@6roads.com.pl (3)

Sensors Group, Institute of Neuroinformatics, UZH-ETH Zurich, Winterthurerstr. 190, CH-8057 Zurich,

tobi@ini.uzh.ch (4)

Astronomical Observatory Institute, Faculty of Physics, A.Mickiewicz

University, Słoneczna 36, 60-286 Poznań, Poland, chrisk@amu.edu.pl

ABSTRACT

Dynamic vision sensor (DVS) event cameras possess a

unique feature of outputting only sparse and

asynchronous brightness changes rather than the

conventional image sensor measurement of average

intensity level during a fixed exposure time. This new

technology opens a window of opportunity for SST,

especially for survey observations, where users are

mostly interested in detecting objects moving within the

telescope's field of view. In this work we present a

comparison between a regular global-shutter CMOS

camera (QHY174-GPS) and several DVS-based DAVIS

cameras, which can concurrently output standard frames

and DVS events. The measurements include new

sensors, so far uncharacterized for space surveillance,

specifically the first back illuminated DAVIS

(BSIDAVIS) and a DAVIS with more sensitive

temporal contrast threshold (SDAVIS). The sensors

were observationally tested during stellar observing runs

with varying telescope tracking speed to simulate SST

targets on different orbits, using identical optics and

under the same weather conditions. Observations

included daytime sky targets with high sky brightness.

The minimum detectable object magnitudes and

maximum object speeds were quantitatively assessed.

The potential of existing event-based sensors is

evaluated and future upgrades to DVS designs to fully

utilize this technology in SST are discussed.

1. INTRODUCTION

Space surveillance and tracking (SST) is of increasing

importance because of the growing number of active

satellites and space debris which increases the risks of

collision, and the threat of the Kessler phenomenon,

which could make low earth orbits (LEO) inaccessible.

SST optical methods are traditionally considered

primary source of tracking and surveillance data for

higher orbits due to satellites low proper motions and

relaxed image timing requirements which are possible to

fulfill using regular astronomical CCD and CMOS

image sensor (CIS) cameras [1]. Optical surveillance in

the LEO region has been developing rapidly in recent

years because of the emergence of low cost and high

spatial resolution cameras delivering high lateral

position precision [2]. It is an interesting supplement, or

even alternative, to expensive radar installations that

provide precise orbit altitude, but poor lateral precision.

By contrast, optical SST can use mount encoder

readings or fixed stars as known reference points to

compute precise astrometric positions.

1.1 Standard camera optical SST

Optical SST observations are divided into two main

categories: survey and tracking. The principle of optical

survey is to watch the sky to detect and identify any

moving object in the sensor field of view, and to add

new objects identified as Earth satellites to a catalog. In

tracking observations, the aim is to follow up already

known objects to improve their orbital parameters,

characterise maneuvers, and to update close pass

predictions for collision avoidance. In both cases

camera image timing is of critical importance. For

higher orbits it is sufficient to time images with

accuracy at the level of tens of milliseconds, but for

LEO sub-millisecond accuracy is required to obtain 1

arcsec astrometric accuracy. It is obviously very hard to

obtain such an accuracy with mechanical shutter and

large CCD array. Moreover short exposure times

translates to read noise dominated images. Therefore

low-noise CMOS image sensor (CIS) with global

electronic shutter is recently becoming a standard

detector for low orbit tracking and survey.

LEO objects move rapidly across the image; for

example a satellite in 500km orbit moves at an apparent

speed of 0.9 deg/s at zenith. It crosses the field of view

(FOV) of the CIS camera we used for this study in only

1.4s in zenith. Assuming diffraction size of 2arcsec, it

spends only 0.7ms over each pixel. If the object moves

too quickly, the time spent by it over each pixel

Proc. 1st NEO and Debris Detection Conference, Darmstadt, Germany, 22-24 January 2019, published by the ESA Space Safety Programme Office

Ed. T. Flohrer, R. Jehn, F. Schmitz (http://neo-sst-conference.sdo.esoc.esa.int, January 2019)

becomes too small and the signal is lost in the noise.

SST observations could benefit from recently developed

dynamic vision sensors (DVS) event cameras in the

following areas. Day-time observations thanks to high

dynamic range, data flow reduction due to the nature of

these cameras and possible astrometric accuracy

improvement due to separate time records from each

pixel.

1.2 Dynamic vision sensor (DVS) event cameras

Fig. 1 shows the recorded output of a “dynamic and

active pixel vision sensor” (DAVIS) event camera [3],

[4]. A DAVIS concurrently outputs conventional global

shutter frames and DVS brightness change events. The

brightness change events are in the form of timestamped

address-events (x,y,p,t), where x and y are the pixel

coordinates, p is the sign (ON or OFF) of the brightness

change, and t is the time of the event in microseconds.

The DVS output is attractive for various applications

because of its low latency, sparsity, and high dynamic

range. It allows applications to only process the

changing pixels and to do so with sub millisecond

latency.

Event camera prototypes are now commercially

available, although the most advanced types are still

laboratory objects. A main aim of this paper was to

evaluate new event camera prototypes for SST

applications.

Figure 1. In response to the spinning dot, a DAVIS

outputs frames (on demand) and a concurrent stream

of brightness change events (adapted from Delbruck,

2018, unpublished).

1.3 Optical SST with DVS

Using DVS for SST, was first proposed in [1]. The

results reported in that paper stimulated the current

study, which aimed for a quantitative comparison of star

magnitude and speed detection limits from CIS and

DAVIS.

Fig. 2 shows an example of BSIDAVIS (see Sec. 2) data

collected for this study from a GlobalStar M008 satellite

flyby event at altitude 1530km. The satellite transited

about 350 pixels in 3s (~100 pixels/s). For clarity, the

raw data was filtered using jAER1 software noise

1 http://jaerproject.net

reduction filters using parameters listed in the figure

caption, which removed about 96% of the events.

A

B

Figure 2. Spacetime data from flyby of GlobalStar

25309. Approximately 96% of the raw data was

filtered out using SpatioTemporalCorrelationFilter

and HotPixelFilter with the criterion of at least

3 active neighbours of the closest 36 pixels during the

past 10ms. A: spacetime plot. B: accumulated events

over 3.4s. Speed: 101 pix/s. Green: ON events.

Red: OFF events.

2, EQUIPMENT and METHODOLOGY

Cameras: Tab. 1 compares the cameras used in this

study. The global-shutter CIS camera (QHY174-GPS2)

used in this paper is the first commercially available

astronomical camera which is integrated by its

manufacturer with a GPS receiver. The camera is

specifically designed for time resolved astronomy and

provides exposure time with a precision claimed to be at

the level of 10-6

s. It also includes electronic cooling

which was not used for this study due to short exposure

times.

2 http://www.qhyccd.com

The two DAVIS sensors used in this study are the first

back illuminated DAVIS (BSIDAVIS) [5] and the first

sensitive-DVS DAVIS camera (SDAVIS) [6]. These are

experimental cameras developed by the Sensors Group

at the Institute of Neuroinformatics. The BSIDAVIS has

higher quantum efficiency from its 100% effective fill

factor, and the SDAVIS is sensitive to smaller

brightness changes from the increased gain in its

preamplifier.

Observatory: We compared the cameras using an

experimental observatory located in Krakow during the

months of September and December 2018. The

observatory is equipped with a Takahashi Sky90

(90mm aperture, f/5.5) and a star+satellite tracking

motorized mount Bisque Paramount MX.

Table 1. Camera specifications.

QHY174-

GPS

BSIDAVIS

[5] SDAVIS [6]

Type CIS, global

shutter DAVIS DAVIS

Pixel array 1920x1200 346x260 188x192

Pixel size (um) 5.86 18.5 18.5

/Active area(mm) 11.3x7.0 6.4x4.8 3.5x3.6

Peak QE (%) 78 92 ~20

Fill factor (%) NA ~100 21.2

APS Dark current

@25°C (e/s) ~37 ~16k 16k

CIS DR (dB) 43-75* 53 53

CIS read noise (e) 1.6-5.3* 60 60

CIS conv gain

(uV/e) NA 22 22

DVS min

threshold (%) NA 15* 3.5*

DVS DR (dB) NA >120 >100

Pixel image scale

[arcsec]♱ 2.39 7.70 7.70

FoV [arcmin]♱ 76x49 44x33 25x25

* - depending on gain settings;

♱ - using 90mm f/5.5 telescope.

In this paper we simplified the problem of detection to

visually observing image streaks in the CIS and

accumulated-event DVS images, as explained below.

2.1 Nighttime observations

The goal was to assess the minimum magnitude object

that each sensor could detect. The BSIDAVIS has about

4X higher QE than the equivalent front-side illuminated

DAVIS [5] and about 12X higher QE than the

DAVIS240C used in [7]. Therefore we were interested

in whether this higher QE allows detecting fainter and

faster moving objects compared with the CIS.

We used the Pleiades constellation as a collection of

stars with known magnitude that we could easily

identify in a star catalog. The goal here was to traverse

the stars with speeds ranging from 7 arcsec/s to 3.5

deg/s (DAVIS 1.8 pix/s to 1800 pix/s, CIS 5.7 pix/s to

5700 pix/s). We use sidereal (fixed mount) for the

slowest speed and the telescope drive motor for faster

speeds. We captured data with 1X, 10X, 100X, and

300X sidereal speeds.

2.2 Daytime observations

The BSIDAVIS minimum temporal contrast threshold is

about 15% change in intensity, so it is not suitable for

detecting low contrast objects such as those expected in

daytime observation against a bright sky. The SDAVIS

QE is only about 20%, and the circuit design does not

function well at very low photocurrents. However, it’s

photoreceptor preamplifier provides it a higher temporal

contrast sensitivity (down to 3.5%). Therefore we were

particularly interested in exploring whether it allowed

observation of daytime or evening/dawn satellite

tracking.

To assess this possibility, using the SDAVIS, we

imaged several brighter stars during daytime (Vega and

Deneb) using the same apparent motion methods as for

nighttime observations. Cloudy winter weather and

difficulty adjusting the prototype’s sensor parameters

severely limited observational time so these results are

very preliminary.

2.3 Pixel transit time considerations

CIS: During any finite CIS exposure, the satellite may

pass over many pixels, resulting in a streak in the image.

The detection limits for CIS SST are mainly from the

limited time that a satellite spends over each pixel. The

increased light exposure resulting from the satellite is

inversely proportional to the speed of the satellite. The

faster the satellite moves, the less time it spends over the

pixel, and the dimmer is the resulting streak. The entire

image is also exposed to dark current, which increases

the value of each pixel according to the product of dark

current and exposure time.

DVS: Since pixels continuously monitor the

photocurrent, a satellite is only detected if the

momentary increase in photocurrent caused by the

transit is detected as an event, i.e. if it causes a relative

change of the filtered photocurrent exceeding the

temporal contrast threshold. [8] analyzed this case in the

context of particle tracking velocimetry. It found that

the required pixel bandwidth is proportional to the

inverse of the pixel transit time, i.e. if the object

produces a Gaussian deflection of photocurrent and the

full width at half maximum of this deflection is 1ms,

then the required bandwidth to produce a full-contrast

response is a few times . The

DVS pixel bandwidth is a linearly increasing function of

the photocurrent [3]. It means that the brighter the star,

the faster the bandwidth. However, the bandwidth of the

pixel is also controlled by the analog bias currents for

the photoreceptor and source follower buffers. By

settings these bias currents small, the front end circuit

can limit its bandwidth and hence increase its

integration time, as well as reducing the shot noise.

However, the quantitative value of the bandwidth

produced by a particular magnitude is difficult to

estimate, and has only been measured under particular

bias conditions for some DVS [3], [6]. As for CIS, the

situation is complicated by the dark current, which in

the case of DVS decreases the photocurrent contrast. In

the case of a dim star, the contrast of the star is reduced

by the background dark current.

CIS: For the measurements reported here that start with

the slowest transit speed of 1X sidereal, we determined

an optimal time of exposure of 500ms, as follows. We

estimate star point spread function (PSF) FWHM is 1.9

pixels, or 4.5 arcsec, and the CIS pixel resolution is 2.39

arcsec.

With the speed 1X sidereal, star travels 15 arcsec/s, or

6.28 pixels/s. In 200ms, a star travels only 1.26 pixels,

which is less than its own PSF. But in 500ms, it travels

3.14 pixels, which is about 150% of its diameter.

Therefore with a 500ms exposure, we are assured that

any star will cross at least several pixels, causing a full

exposure of the pixel to the star.

DVS: Since the DVS has no exposure time, we simply

recorded continuous DVS output during the

constellation transit. When possible by sufficient

observation time, we varied the photoreceptor and

source follower bias currents to study the effect of front

end bandwidth on maximum transit speed and noise and

used the subjectively optimal parameters for the results

reported here.

Detection: We determined the limiting magnitude star

by visual inspection of the DVS accumulated event

image and the single CIS image. We considered the

dimmest track with clearly detectable start and end

points as the limiting magnitude.

3. RESULTS

3.1 Limiting magnitude (night)

telescope speed: 1x sidereal

telescope speed: 10x sidereal

telescope speed: 100x sidereal

telescope speed: 300x sidereal

Figure 3. BSIDAVIS events accumulated during 320ms

for Pleiades transits at different speeds. V magnitudes

are overplotted for selected stars.

Fig. 3 shows raw BSIDAVIS DVS event data for the

tracking experiment. Each panel is a 2D histogram of

accumulated DVS events during part of the scan. Stars

are visible as streaks. Identified stars are labelled with

their magnitudes for each experiment. At 1X sidereal,

many more stars are visible than at 300X. For each

speed, the faintest visible streak was identified by eye.

telescope speed: 1x sidereal

telescope speed: 10x sidereal

telescope speed: 100x sidereal

telescope speed: 300x sidereal

Figure 4. 500ms CIS frames for Pleiades transits at

different speeds. V magnitudes are overplotted for

selected stars.

Fig. 4 shows the same type of data but for the CIS.

Tab. 2 and Fig. 5 show the lower limiting magnitude for

DVS and CIS versus speed. For each sensor, the faster

the movement, the brighter must be the star to observe a

track. The CIS can detect stars that are dimmer on

average by about 1.6 magnitude than the DVS (a factor

of about 4.3).

Table 2. Limiting V magnitude

Speed

(x sidereal)

DAVIS

(mag.) CIS (mag.) diff.

1x 10,38 11,9 1,52

10x 9,22 10,52 1,30

100x 7,19 8,97 1,78

300x 5,45 7,19 1,74

Figure 5. Estimated nighttime limiting magnitude

versus relative sidereal speed.

3.2 Limiting magnitude (daytime)

We repeated the experiment during daytime using the

SDAVIS, although we were not able to compare with

the QHY CIS. As mentioned, observation time was

limited to a single session with clear weather and

daytime conditions. At about 11:00 on a slightly

overcast November day in Krakow with high cirrus

clouds, we located and recorded observations of Vega

(mag 0). Vega could be observed by eye in the

telescope. Using the APS mode of SDAVIS, we

observed that the sky brightness was 2.2e4 DN/s3. Vega

produced 2.75e4 DN/s and thus had a contrast of about

1.25X against the sky.

A caveat on the data presented here is that the SDAVIS

3 DN is digital number; 1 DN=1.23mV=56e

-.

was not optimally configured owing to a lack of

understanding of the rather complex controls. It means

that the background activity rate (of noise OFF events)

was very high and the sensitivity was not optimized.

There was also a periodic oscillation of the activity

probably caused by power supply load positive

feedback. This periodic pulsation at a frequency of

about 1.7Hz somewhat obscured the star tracks.

We observed Vega using the telescope drive to create

transits. Fig. 6 shows traces from two of the recordings.

We could easily track Vega for transits up to 0.6 deg/s

(270 pix/s).

We did not succeed to catch any daytime satellites in

this session. Subsequently we were able to adjust the

SDAVIS biases for better performance but cloudy skies

prevented any further observations.

Figure 6. Daytime tracks of DVS events from SDAVIS.

showing only ON events. Tracks of Vega at two

speeds: 318ms at 0.04deg/s and 80ms at 0.6deg/s.

No noise filtering was applied.

4. DISCUSSION

The results of this comparative study show that for

nighttime observation, the current state of the art DVS

sensor (BSIDAVIS) offers about 1.6 mag worse limiting

magnitude compared to the QHY CIS camera. Current

prototype DVS sensors do not offer superior absolute

detection capability, despite its larger pixel size and

higher QE peak. It is however worth noting that the

FWHM of CIS images was significantly lower

(~5 arcsec compared to ~15 arcsec), which was caused

partially due to DVS charge leakage between pixels and

partially due to focusing errors.

From the DVS event tracks, although we have not

demonstrated it in this paper, we believe it would be

possible in real time to precisely estimate the satellite

location, with precision down to sub-millisecond and

system latency of a few milliseconds.

From the CIS streaks, it should also be possible to

accurately determine object speed. From the CIS start

and end frame times it should also be possible to

determine absolute sky position with the timing

precision of the known shutter times.

Therefore, both cameras appear to offer roughly

comparable capability of survey and tracking, although

the CIS sensor clearly offers higher spatial resolution.

The data volume from DVS especially after noise

filtering is much lower than from CIS, even at low CIS

frame rate. At 2Hz (500ms exposure time) CIS frame

rate, a QHY sensor output data rate is 4.6M pixel/s. A

CIS sensor of same spatial resolution to the BSIDAVIS

would output a data rate of about 346x260x2=180k

pixels/s. This data rate is still much higher than the raw

BSIDAVIS data rate of about 30k events/s. After

modest correlation and hot pixel noise filtering of

BSIDAVIS, the data rate drops to less than 5k events/s,

which is a factor of 36 smaller than from the

downscaled CIS sensor. This low rate puts real time

analysis of the DVS data in reach of small embedded

platforms like Raspberry Pi.

Therefore, the main results of this study are the

following:

1. For nighttime observation, the prototype

BSIDAVIS offers 4X lower sensitivity and

timing resolution than the QHY CIS sensor,

but at factors of 10s or 100s less data rate.

2. For daytime observation, the prototype

SDAVIS can detect Vega in noonday northern

latitude conditions at speeds up at least

0.6 deg/s (450 pix/s).

A sensor with higher pixel count that combines the

higher event sensitivity of the SDAVIS pixel circuit

with the high quantum efficiency of the BSIDAVIS

back illumination process technology would appear to

offer useful capabilities for survey and tracking

applications, for both nighttime and perhaps daytime

observation of LEO objects.

A DVS camera outputs pixel position and time for every

event on each pixel while CIS outputs only two times

(beginning and end of exposure) for all pixels. A single

image from CIS is usually transformed using tools such

Vega at 0.6 deg/s

Vega at 0.04 deg/s

as sextractor4 to a target's single pixel position at a

single time of mid-exposure. DVS data creates an

opportunity to develop a dedicated astrometric

procedure that would utilize additional timing

information and analyze satellite positions at much

higher rate than possible from CIS images. It is possible

that for bright targets, such a procedure would produce a

superior astrometric accuracy compared to regular CIS

measurements. The shorter latency of these

measurements could also enable real time telescope

drive control to track new objects. These capabilities of

course require verification with future observational

tests of DVS cameras.

Acknowledgments

We thank Greg Cohen for discussion leading to this

study. The EU projects CAVIAR, SEEBETTER, and

VISUALISE and Samsung Inst. of Technology helped

fund the DAVIS camera developments. inilabs.com and

inivation.com provided technical support with camera

hardware.

REFERENCES

[1] M. Shoemaker and L. Shroyer, Historical Trends in

Ground-Based Optical Space Surveillance System

Design. adsabs.harvard.edu, 2007 [Online]. Available:

https://ui.adsabs.harvard.edu/\#abs/2007amos.confE...1S

[2] K. Kamiński, E. Wnuk, J. Gołębiewska, M. Krużyński,

and M. K. Kamińska, “LEO Cubesats Tracking with a

Network of Polish Optical SST Sensors,” in Proceedings

of the AMOS conference, Hawaii, USA, 2018.

[3] P. Lichtsteiner, C. Posch, and T. Delbruck, “A 128x128

120dB 15us Latency Asynchronous Temporal Contrast

Vision Sensor,” IEEE J. Solid-State Circuits, vol. 43, no.

2, pp. 566–576, Feb. 2008 [Online]. Available:

http://dx.doi.org/10.1109/JSSC.2007.914337

[4] C. Brandli, R. Berner, M. Yang, S.-C. Liu, and T.

Delbruck, “A 240x180 130 dB 3 us Latency Global

Shutter Spatiotemporal Vision Sensor,” IEEE J. Solid-

State Circuits, vol. 49, no. 10, pp. 2333–2341, Oct. 2014

[Online]. Available:

http://dx.doi.org/10.1109/JSSC.2014.2342715

[5] G. Taverni et al., “Front and Back Illuminated Dynamic

and Active Pixel Vision Sensors Comparison,” IEEE

Trans. Circuits Syst. Express Briefs, vol. 65, no. 5, pp.

677–681, May 2018 [Online]. Available:

https://ieeexplore.ieee.org/document/8334288/?arnumber

=8334288&source=authoralert

[6] D. P. Moeys et al., “A Sensitive Dynamic and Active

Pixel Vision Sensor for Color or Neural Imaging

Applications,” IEEE Trans. Biomed. Circuits Syst., vol.

PP, no. 99, pp. 1–14, 2017 [Online]. Available:

http://dx.doi.org/10.1109/TBCAS.2017.2759783

[7] G. Cohen et al., “Event-based Sensing for Space

Situational Awareness,” in Proc. the Advanced Maui

Optical and Space Surveillance Technologies

Conf.,(AMOS, 2017, p. 25 [Online]. Available:

https://amostech.com/TechnicalPapers/2017/Optical-

4 www.astromatic.net/software/sextractor

Systems/Cohen.pdf. [Accessed: 03-Feb-2018]

[8] D. J. Borer, “4D Flow Visualization with Dynamic

Vision Sensors,” ETH Zurich, Zurich, Switzerland, 2014

[Online]. Available: https://e-

collection.library.ethz.ch/view/eth:47156