-
ORIGINAL PAPER
Damian O. Elias Æ Bruce R. Land Æ Andrew C. MasonRonald R.
Hoy
Measuring and quantifying dynamic visual signals in jumping
spiders
Received: 8 October 2005 / Revised: 7 February 2006 / Accepted:
11 February 2006 / Published online: 17 March 2006� Springer-Verlag
2006
Abstract Animals emit visual signals that involvesimultaneous,
sequential movements of appendages thatunfold with varying dynamics
in time and space. Algo-rithms have been recently reported (e.g.
Peters et al. inAnim Behav 64:131–146, 2002) that enable
quantitativecharacterization of movements as optical flow
patterns.For decades, acoustical signals have been rendered
bytechniques that decompose sound into amplitude, time,and spectral
components. Using an optic-flow algorithmwe examined visual
courtship behaviours of jumpingspiders and depict their complex
visual signals as ‘‘speedwaveform’’, ‘‘speed surface’’, and ‘‘speed
waterfall’’plots analogous to acoustic waveforms, spectrograms,and
waterfall plots, respectively. In addition, these‘‘speed profiles’’
are compatible with analytical tech-niques developed for auditory
analysis. Using examplesfrom the jumping spider Habronattus
pugillis we showthat we can statistically differentiate displays of
different‘‘sky island’’ populations supporting previous work
ondiversification. We also examined visual displays fromthe jumping
spider Habronattus dossenus and show thatdistinct seismic
components of vibratory displays areproduced concurrently with
statistically distinct motionsignals. Given that dynamic visual
signals are common,
from insects to birds to mammals, we propose thatoptical-flow
algorithms and the analyses described herewill be useful for many
researchers.
Keywords Motion displays Æ Multimodalcommunication Æ Courtship
behaviour ÆMotion analysis
Introduction
Studying animal behaviour often means the analysis ofmovements
in time and space. While techniques arereadily available for static
visual patterns and ornaments(Endler 1990), this is less so with
dynamic sequences ofvisual signals (motion displays) (but see
Zanker and Zeil2001).
An extensive literature exists on the study of motionas it
pertains to neural processing, navigation and theextraction of
motion information from visual scenes(reviewed in Barron et al.
1994; Zanker and Zeil 2001).In neurobiology in particular,
techniques have beenmotivated by the need to accurately describe
biologi-cally relevant features of motion as an animal movesthrough
its environment to identify coding strategies inthe processing of
visual information (Zanker 1996; Zeiland Zanker 1997; Zanker and
Zeil 2001; Eckert andZeil 2001; Tammero and Dickinson 2002). While
thesestudies have been integral to an examination of
visualprocessing, such techniques have limited application
instudies of behavioural ecology and communication asthey do not
provide simple, intuitive depictions ofmotion for quantification
and comparison. Anotherextensive body of literature on the analysis
of motionexists in the study of biomechanics, particularly in
thekinematics of limb motion (Tammero and Dickinson2002; Jindrich
and Full 2002; Nauen and Lauder 2002;Vogel 2003; Fry et al. 2003;
Alexander 2003; Hedricket al. 2004). Such techniques could, in
principle,provide extensive information on motion signals butthese
computationally intensive approaches may not
Damian O. Elias and Bruce R. Land contributed equally
D. O. Elias Æ B. R. Land Æ R. R. HoyDepartment of Neurobiology
and Behavior,Cornell University, Seeley G. Mudd Hall,Ithaca, NY
14853, USAE-mail: [email protected]
A. C. Mason Æ Present address: D. O. Elias (&)Department of
Life Sciences,Integrative Behaviour and Neuroscience,University of
Toronto at Scarborough,1265 Military Trail, Scarborough,ON, M1C 1A4
CanadaE-mail: [email protected].: +1-416-2877465Fax:
+1-416-2877642
J Comp Physiol A (2006) 192: 785–797DOI
10.1007/s00359-006-0116-7
-
efficiently capture aspects of visual motion signals thatare
most relevant in the context of communicationsignals. In addition,
both techniques present theexperimenter with large data sets and it
is oftendesirable to reduce the data in order to glean
relevantinformation.
In particular, Peters et al.(2002) (Peters and Evans2003a, b)
have provided a significant advance in theanalysis of motion
signals in communication. Peterset al. (2002) described powerful
techniques for theanalysis of visual motion as optical flow
patterns in anattempt to demonstrate that signals are
conspicuousagainst background motion noise (Peters et al.
2002;Peters and Evans 2003a, b). Here we build upon thesepioneering
techniques by making use of these previousalgorithms to develop
another depiction of visual signalsand use these to analyse
courtship displays of jumpingspiders from the genus Habronattus. In
addition, weshow that optical flow approaches are suitable
forquantification and classification by methods equivalentto audio
analysis (Cortopassi and Bradbury 2000).
Male Habronattus court females by performing anelaborate
sequence of temporally complex motions ofmultiple colourful body
parts and appendages (Peck-ham and Peckham 1889, 1890; Crane 1949;
Forster1982b; Jackson 1982; Maddison and McMahon 2000;Elias et al.
2003). Habronattus has recently been usedas a model to study
species diversification (Masta2000; Maddison and McMahon 2000;
Masta andMaddison 2002; Maddison and Hedin 2003; Hebetsand Maddison
2005) and multicomponent signalling(Maddison and Stratton 1988;
Elias et al. 2003, 2004,2005). In these studies there has been an
implicitassumption that qualitative differences in dynamic
visualcourtship displays can reliably distinguish amongspecies
(Richman 1982), populations (Maddison 1996;Maddison and McMahon
2000; Masta and Maddison2002; Maddison and Hedin 2003; Hebets and
Maddi-son 2005), and seismic signalling components (Eliaset al.
2003, 2004, 2005). It has yet to be determined,however, whether
such qualitative differences can standup to rigorous statistical
comparisons (Walker 1974;Higgins and Waugaman 2004). To test
hypotheses onsignal evolution and function it is crucial to
under-stand the signals in question. Thus it is necessary totest
whether qualitative signal categories are consis-tently
different.
Our method reduces the dimensionality of visualmotion signals by
integrating over spatial dimensions toderive patterns of motion
speed as a function of time.This method may not be adequate for
some classes ofsignal (e.g. which differ solely in position or
direction ofmotion components). Our results demonstrate,
however,that for many signals this technique allows
objectivequantitative comparisons of complex visual motionsignals.
This will potentially provide a wide range ofuseful behavioural
measures to a variety of disciplinesfrom systematics and
behavioural ecology to neurobi-ology and psychology.
Methods
Spiders
Male and female H. pugillis and H. dossenus were fieldcollected
from different mountain ranges in Arizona(Atascosa—H. dossenus and
H. pugillis; Santa Cata-lina—H. pugillis; Santa Rita—H. pugillis;
Galiuro—H.pugillis). Animals were housed individually and kept
inthe lab on a 12:12 light:dark cycle. Once a week, spiderswere fed
fruit flies (Drosophila melanogaster) and juve-nile crickets
(Acheta domesticus).
Recording procedures
Recording procedures were similar to a previous study(Elias et
al. 2003). We anaesthetized female jumpingspiders with CO2 and
tethered them to a wire from thedorsum of the cephalothorax with
low melting pointwax. We held females in place with a
micromanipulatoron a substrate of stretched nylon fabric (25·30
cm). Thisallowed us to videotape male courtship from a predict-able
position, as males approach and court females intheir line of
sight. Males were dropped 15 cm from thefemale and allowed to court
freely. Females were awakeduring courtship recordings. Recordings
commencedwhen males approached females. ForH. pugillis, we
usedstandard video taping of courtship behaviour (30 fps,Navitar
Zoom 7000 lens, Panasonic GP-KR222, SonyDVCAM DSR-20 digital VCR)
and then digitized thefootage to computer using Adobe Premiere (San
Jose,CA, USA) with a Cinepak codec. Video files were storedas *.avi
files. For H. dossenus, we used digital high-speedvideo (500 fps,
RedLake Motionscope PCI 1000, SanDiego, CA, USA) acquired using
Midas software (Xci-tex, Cambridge, MA, USA). We selected suitable
videoclips of courtship behaviour based on camera steadinessand
length of behavioural displays (
-
programs for each analysis step are available
athttp://www.nbb.cornell.edu/neurobio/land/PROJECTS/MotionDamian/
Cropping/intensity normalization
Video sequences were shot at either 30 (H. pugillis) or500 fps
(H. dossenus). High-speed sequences (500 fps)were reduced to 250
fps for analysis and the intensity ofeach frame normalized because
the high-speed cameraautomatic gain control tended to oscillate
slightly.Normalization (PN) was achieved by the
followingequation:
PN = POPAvgPFAvg
� �0.75;
where PN is the normalized pixel intensity, PO is theoriginal
individual pixel intensity, PAvg is the mean pixelvalue for the
whole video sequence, and PFAvg is themean pixel intensity value in
the individual frame.Frames were cropped so that the animal was
completelywithin and spanned nearly the entirety (>75%) of
theframe.
Optical flow calculation
The details of this algorithm are published elsewhere(Barron et
al. 1994; Zeil and Zanker 1997; Peters et al.2002). Briefly, we
used a simple gradient optical flowscheme to estimate motion. If a
2-dimensional (2D)video scene includes edges, intensity gradients,
ortextures, motion in the video scene (as an object sweepspast a
given pixel location) can be represented aschanging intensity at
that pixel. Intensity changes canthus be used to summarize motion
from video segments.Such motion calculations are widely used in
robotics andmachine vision to analyse video sequences (e.g.
http://www.ifi.unizh.ch/groups/ailab/projects/sahabot/).
Our video data were converted into an N·M·Tmatrix where N is the
number of pixels in the horizontaldirection, M is the number of
pixels in the verticaldirection, and T is the number of video
frames. The 3Dmatrix was smoothed with a 5·5·5 Gaussian
convolu-tion kernel with a standard deviation of one pixel(Barron
et al. 1994). Derivatives in all three directionswere computed
using a second-order (centred, 3-point)algorithm. This motion
estimate is based on theassumption that pixel intensities only
change fromframe-to-frame because of motion of objects passing
bythe pixels. The local speed estimate (vg) was calculated
asfollows:
vg ¼ �ðrIÞ ðdI=dtÞjjðrIÞjj2
;
where vg is the local object velocity estimate in thedirection
of the spatial intensity gradient, I is an array of
intensities of pixels, t is the frame number, and || is
themagnitude operator (Barron et al. 1994). The local speedestimate
is defined as the magnitude of vg.
Speed profile plots
The speed waveform is a simple average of the localspeed
estimates (vg) for objects over all pixels in theframe (Peters and
Evans 2003a). We also defined a speedsurface (analogous to a
spectrogram). The speed surfaceis a 2D plot with frame number on
the x-axis, pixelspeed bins on the y-axis, and the colour in each
binrelated to the log of the number of pixels moving at thatspeed.
In other words, at each frame, we plotted a his-togram of the
number of pixels showing movement at aparticular speed range. Both
of these plots representedthe complete speed profiles of each video
clip. We alsoconstructed a speed ‘‘waterfall’’ plot which
representsthe speed surface as a 3-dimensional (3D) plot, with
thez-axis showing the log of the number of pixels associatedwith a
speed bin in any given frame.
Maximum cross-correlation of 1D and 2D signals
Similarity between speed profiles was computed bynormalized
cross-correlation of pairs of sample plots(with periodic wrapping
of samples). Waveforms beingcompared were padded with the mean of
the sequence,so that the shorter one became the same length as
thelonger one. Both speed waveforms and speed surfaceswere
analysed, using a 1-dimensional (1D) correlationfor the speed
waveforms and a 2D correlation (withshifts only along time) for the
speed surfaces. For thenext stage of the analysis, we used a
measure ofdissimilarity (1.0 minus the maximum correlation) as
adistance measure to construct a matrix of distancesbetween all
pairs of signals.
Multidimensional scaling (MDS)
The distance (dissimilarity) matrix was used as input fora MDS
analysis (Cox and Cox 2001). MDS provides anunbiased, low
dimensional representation of the struc-ture within a distance
matrix. A good fit will preserve therank order of distances between
data points and give alow value of stress, a measure of the
distance distortion.MDS analysis normally starts with a 1D fit and
increasesthe dimensionality until the stress levels plateau at a
lowvalue. A Matlab subroutine was used (Steyvers,
M.,http://www.psiexp.ss.uci. edu/research/ software.htm) toperform
the MDS. We used an information theoreticanalysis on the entropy of
clustering (Victor and Pur-pura 1997) on both the 1D and 2D
correlations anddetermined that more information was contained in
the1D correlation (data not shown); hence, all furtheranalyses were
performed on the 1D correlation matrices.
787
-
We fitted our data from one to five dimensions. Most ofthe
stress reduction (S1) occurred at either two or threedimensions (H.
pugillis: 1st dimension, S1=0.38,R=0.68; 2nd dimension, S1=0.23,
R=0.78; 3rddimension, S1=0.15, R=0.84; 4th dimension,
S1=0.12,R=0.88; 5th dimension, S1=0.09, R=0.89; H. dossenus:1st
dimension, S1=0.32, R=0.69; 2nd dimension,S1=0.19, R=0.81; 3rd
dimension, S1=0.12, R=0.88;4th dimension, S1=0.09, R=0.92; 5th
dimension,S1=0.07, R=0.93), hence all further analysis was
per-formed on 3D fits. Plots of the various signals in MDS-space
showed strong clustering. The axes on the MDSanalysis reflect the
structure of the data; we thereforeperformed a one-way ANOVA along
different dimen-sions to calculate statistical significance of the
clustering.A Tukey post hoc test was then applied to
comparedifferent populations (H. pugillis) and signal compo-nents
(H. dossenus). All statistical analyses were per-formed using
Matlab.
Results
Motion algorithm calibration
In order to test the performance of the motion algorithmagainst
a predictable and controllable set of motionsignals, we simulated
rotation of a rectangular baragainst a uniform background using
Matlab. Texturewas added to the bar in the form of four
nonparallelstripes (Fig. 1a). We programmed the bar to pivotaround
one end using sinusoidal motion. We then sys-tematically varied the
width (w) and length (l) of the bar,as well as the frequency (F)
and amplitude (A) of themotion (Fig. 1a).
The three examples in Fig. 1 show the effect of a step-change in
frequency (F), amplitude (A) and bar length(l), respectively (Fig.
1b–d). In each case, the analysisdepicts the temporal structure of
the simulated move-ment very well (Fig. 1). As frequency,
amplitude, or barlength increase, the computed average speed
increasespredictably (see below). The speed waveform and sur-face
plots show that more pixels ‘‘move’’ at higherspeeds after the step
increase. This detailed shape isdepicted particularly well in the
surface plot.
The amplitude of the average motion of the simulatedbar is
related to the amplitude of the input wave and itsfrequency by a
square law. [Fig. 1b(iv), c(iv)]. Detailedexamination of the image
sequence suggests that athigher speeds, the motion in a video clip
‘‘skips pixels’’between frames, hence this square law is a result
of theproduct of the speed measured at each pixel (which islinear)
multiplied by the greater number of pixels aver-aged into the
motion at higher speeds. The amplitude ofthe average motion of the
simulated bar is related to barlength by a cube law. [Fig. 1d(iv)].
This results from theaforementioned square law increased by another
linearfactor, i.e. the number of pixels covered by the edge ofthe
bar. Both bar width and texture change average
motion amplitude only weakly (data not shown). Thissmall change
in average motion can be attributed to theincrease in the total
length of edge contours.
We also modelled amplitude (AM) and frequency(FM) modulated
movement by animating a rotating barat a fixed carrier frequency
and either AM or FMmodulating the carrier. The carrier and
modulatingfrequencies are clearly discernable for both AM(Fig. 2a)
and FM (Fig. 2b) movements in all the speedprofiles. AM and FM
movements are also easily dis-tinguishable from one another. Both
simple (sinusoidal)and complex (modulated) motions are thus
faithfullyand accurately reflected in the analysis.
Habronattus pugillis populations
H. pugillis is found in the Sonoran desert and localpopulations
on different mountain ranges (‘‘skyislands’’) have different
ornaments, morphologies,and courtship displays (Masta 2000;
Maddison andMcMahon 2000; Masta and Maddison 2002).
Courtshipdisplays from four different populations of H. pugillisare
plotted in Fig. 3. Several repeating patterns areapparent,
especially in Galiuro (Fig. 3a), Atascosa(Fig. 3c), and Santa
Catalina (Fig. 3d) populations. Bycomparing videos of courtship
behaviour with theircorresponding speed profiles, we verified that
features inthe speed profiles corresponded to qualitatively
identi-fiable components of the motion display. For example,in the
Atascosa speed profiles (Fig. 3c), high amplitude‘‘pulses’’ (e.g.
frames 130–150) correspond to singleleg flicks and lower amplitude
‘‘pulses’’ (e.g. frames150–200) correspond to pedipalp and
abdominal move-ments. Speed profiles also reveal more subtle
features ofmotion displays. For example, animals from the SantaRita
mountains make circular movements with theirpedipalps during
courtship (Maddison and McMahon2000). It is evident from the speed
surface (Fig. 3b,frames 0–200) that this behaviour does not occur
in asmooth motion, but rather as a sequence of briefpunctuated,
jerky movements (Fig. 3b).
Different populations of H. pugillis vary in behavio-ural and
morphological characters (Maddison andMcMahon 2000). We evaluated
two populations thatinclude unique movement display characters
[Galiuro—First leg wavy circle, Santa Rita—Palp motion
(circling)]and two that have similar courtship display
characters[Atascosa and Santa Catalina—Late-display leg
flick(single)] (Maddison and McMahon 2000). Santa Catalinaspiders
include the rare Body shake motion character, butthis was not
analysed (Maddison and McMahon 2000).Using MDS, all four groups can
be discriminated fromeach other by the speed profiles (Fig. 4).
Clustering wasstrong for all population classes. In order to
evaluate thesignificance of each cluster we performed a
one-wayANOVA on dimension 1 (F3,19=25.61, P=6.9·10�7) andsaw that
the Santa Catalina population was significantlydifferent from the
Galiuro (P
-
l
w
A
t = 1/FA)
0 20 40 60 80 1000
0.1
0.2
0.3
0.4
0.5
Frame Number20 40 60 80
10
20
30
40
50
01000
0
Speed Bin
1.5
log(
# pi
xels
) -
log
( m
ean
# pi
xels
)
0.5
-0.5
-1.5
20
40
0
100
20 4060
Frame Number80
A=40
A=80
Frame Number
0 10 20 30
0.0
0.04
0.12
0.08
0.1
Bar Length0 40 60 120
0
0.02
0.04
0.06
0.08
0.1
Input Amplitude
0 20 40 60 80 1000
0.1
0.2
0.3
0.4
0.5
Ave
rage
Spe
ed
t = 1/3t = 1/5
20 40 60 80 100
10
20
30
40
50
00
0
Speed Bin
1.5
log(
# pi
xels
) -
log
( m
ean
# pi
xels
)
0.5
-0.5
-1.5
20
40
0
100
20 4060
Frame Number80
Spee
d B
in
Frame Number
B) C)
l =15
l =20
D)
ii)
iii)
iv)
i
v)
i)
0 1 2 3 4 5 60
0.1
0.2
0.3
0.4
Peak
Am
plitu
de
Frequency
0 20 40 60 80 1000
0.1
0.2
0.3
0.4
0.5
10
20
30
40
50
020 40 60 80 1000
0
0
Speed Bin
1.5
log(
# pi
xels
) -
log
( m
ean
# pi
xels
)
0.5
-0.5
-1.5
20
40
0
100
20 4060
Frame Number80
-1-0.6-0.20.20.61
-1-0.6-0.20.20.61
Fig. 1 Simulated movements. a A bar of fixed length (l), width
(w)and starting angle (h) was simulated and rotated sinusoidally at
afixed peak-to-peak amplitude of A and frequency (F). Thefrequency
(b), amplitude (c), and bar length (d) were thensystematically
changed. The time-course of the corresponding
stimulus parameters are shown in panel i. Panels ii–iv show
theresulting analysis of the simulated movement and step changes:
2D‘‘speed waveform’’ plots (ii), 3D ‘‘speed surface’’ plots (iii)
and 3D‘‘speed waterfall plots’’ (iv); and summary of simulated
motionamplitude as different parameters are changed (v)
789
-
(P0.05) [Fig. 4a(ii)]. Also, the Galiuro populationwas
significantly different from the Santa Rita (P
-
0
log(# pixels) - log ( mean # pixels)
20
40
010
020
030
0
Fram
e N
umbe
r
Speed
Bin
1.5
-1.50
400
010
020
030
040
00
0.04
0.08
0.12
A H
. pug
illi
s (G
aliu
ros)
B
H. p
ugil
lis
(San
ta
Rita
s)
010
020
030
040
00
0.1
0.2
0.3
Average Speed
010
020
030
040
0
1020304050
Fram
e N
umbe
rFr
ame
Num
ber
C
H. p
ugil
lis
(Ata
scos
as)
010
020
030
040
0Fr
ame
Num
ber
DH
. pug
illi
s (S
anta
C
atal
inas
)
Speed Bin
Fram
e N
umbe
r
i. ii. iii.
0 1.0
-1.0
00
20
40
010
020
030
040
0
Fram
e N
umbe
r
Speed
Bin
0
20
40
010
020
030
040
0
Fram
e N
umbe
r
Speed
Bin0
1.0
-1.0
010
020
030
040
00
0.1
0.2
0.3
010
020
030
000.04
0.08
0.12
100
200
300
1020304050
0
20
40
010
020
030
0
Fram
e N
umbe
r
Speed
Bin
1.0
-1.00
00
100
200
300
400
103050 0
0
2040
iv.
-1-0.6
-0.20.2
0.6
1 -1-0.6
-0.20.2
0.6
1
1020304050 0
Fig. 3 Different populations of Habronattus pugillis.
Representa-tive from the Galiuro (a), Santa Rita (b), Atascosa (c),
and SantaCatalina (d) mountain ranges are shown. Top panel (i)
shows anexample of a single video frame at the resolution used in
the
analysis. Second panel (ii) shows the 2D ‘‘speed waveform’’
plots.Third panels (iii) show the 3D ‘‘speed surface’’ plots
(second row).Fourth panel (iv) shows the 3D ‘‘speed waterfall
plots’’ (third row).Frame rate is 30 fps
791
-
analysis on dimension 3 (F3,19=3.68, P=0.0303), theSanta
Catalina and Atascosa populations are significantlydifferent
(P0.05, Fig. 4b). Hence,all population classes are statistically
distinguishable in atleast one of the dimensions used in the
analysis.
Habronattus dossenus signals
Courtship displays from five different individuals wereselected
and the visual component of different seismicsignals recorded
(scrape, N=5; thump, N=10; buzz,N=5) for each individual spider
(Elias et al. 2003). Twoclasses of thumps, distinguishable by their
seismiccomponent, were selected for each individual but werenot
distinguishable based on their speed profiles, hencethey were
combined into one class (Elias et al. 2003).First we plotted the
speed profiles for each of the signalclasses (Fig. 5). Speed
surfaces capture relatively subtledetails of movements. For
example, during individualscrape signals the forelegs first come
down followed byabdominal movement upward (Elias et al. 2003)(Fig.
5a). Individual scrapes produce a rocking motionthat can be
observed clearly in the speed surface asa characteristic double
peak (e.g. 1–3 in Fig. 5a).Furthermore, individual abdominal
oscillations areresolved in the buzz speed surface (Fig. 5c).
To test whether this technique could distinguishamong the three
qualitative signal classes, we applied thesame analysis described
above. Clustering was strong forall signal classes. A one-way ANOVA
on dimension 1(F2,18=20.66, P=2.2·10�5) showed that scrapes
aresignificantly different from thumps (P
-
that (Fig. 7 right panels) show location of motion withinthe
video frame with time on the z-axis. These plotsparallel the speed
surface plots used in the analysesabove, but whereas speed surface
plots depict the dis-tribution of motion speed (amplitude) as a
function oftime, speed isoform plots depict the location of
imagemotion as a function of time. These could, in principle,be
analysed similarly to the speed profiles above.
Discussion
Jumping spiders communicate using a complex reper-toire of
visual ornaments and dynamic visual (motion)signals (Jackson 1982;
Forster 1982b). Here we useoptical flow techniques for the
depiction and quantifi-cation of motion signals and use the
technique as the
40 80 120 160
10
20
30
40
50
0040 80 120 160
10
20
30
40
50
00
Scrape
12
23
1
23 4
Thump
0 100 200 300 400Frame Number
10
20
30
40
50
Ave
rage
Spe
ed
123
1 23
2
3
1
4
2
3
1
4
21
21
i.
ii.
iii.
iv.
0 100 200 300 4000
0.05
0.1
0.15
0.2
Frame Number
0 40 80 120 1600
0.5
1
1.5
0 40 80 120 1600
0.1
0.2
0.3
Spee
d B
in
Frame Number
v.
-1-0.6
-0.20.20.61
0
20
40
0 4080 120
160
log(
# pi
xels
) -
log
( m
ean
# pi
xels
)
Speed Bin
1.0
0
-1.0
Frame Number
0
20
40
0 100 200
300 400
Frame Number
log(
# pi
xels
) -
log
( m
ean
# pi
xels
)
Speed Bin
1.0
0
-1.0
-1-0.6
-0.20.20.61
Frame Number
log(
# pi
xels
) -
log
( m
ean
# pi
xels
)
0
20
40
0 4080 12
0 160
Speed Bin
1.0
0
-1.0
12 1
2
0
A B C Buzz
Fig. 5 Different signals of Habronattus dossenus.
Representativeexamples of scrape (a), thump (b), and buzz (c)
signals are shown.Top panel (i) shows an example of a single video
frame at theresolution used in the analysis. Second panel (ii)
illustrates bodypositions with numbers (1–4) illustrating movements
of the forelegsand abdomen. Third panel (iii) shows the 2D ‘‘speed
waveform’’
plots. Fourth panel (iv) shows the 3D ‘‘speed surface’’ plots.
Fifthpanel (v) shows the 3D ‘‘speed waterfall plots’’. Panels iii–v
areshown in the same time scale, with numbers (1–4) corresponding
tothe body movements illustrated in panel ii. Frame rate is 250
fps(reduced from 500 fps)
793
-
basis of a statistical analysis to assess motion signals
injumping spiders.
Quantitative analysis of courtship signals
‘‘Sky island’’ populations
We examined variation in the courtship displays ofdifferent
‘‘sky island’’ populations of H. pugillis(Maddison and McMahon
2000; Masta 2000; Mastaand Maddison 2002). By calculating the
differencesbetween the speed profiles of displays from
differentpopulations, we were able to show that courtshipdisplays
were different between all of the populationsstudied. We could
easily distinguish between popula-tions with unique display
elements [Galiuro—First legwavy circle, Santa Rita—Palp motion
(circling)](Maddison and McMahon 2000). Importantly, we couldalso
discriminate the Santa Catalina and Atascosapopulations that had
qualitatively similar late stagevisual displays (Late-display leg
flick) (Fig. 4b). Maddi-son and McMahon (2000) in their initial
descriptionsand analysis of courtship coded this display as being
thesame between spiders from the Santa Catalina andAtascosa
Mountains. Masta and Maddison (2002)demonstrated that fixation
rates between neutral (mito-chondrial genes) and male phenotypic
traits (morpho-logical and behavioural characters) were different
andused this as evidence to suggest that sexual selection
wasdriving diversification in H. pugillis. This study not
onlysupports those previous studies, but also suggests thatmale
courtship phenotypes are fixed to an even greaterextent than
previously demonstrated.
Multicomponent signals
Elias et al. (2003) showed that males in the jumpingspider H.
dossenus produced at least three different
seismic signals all coordinated with motion signals.Given the
strict coordination of seismic and motionsignals, the authors
suggested that the signal componentsin different modalities are
functionally linked (Elias et al.2003). If emergent properties of
the multimodal signal(seismic and visual) are important, one would
predictthat unique seismic components would have uniquemotion
components. Similar predictions can be made ifunique motion
displays serve to focus attention oncorresponding seismic
components (Hasson 1991). Dis-tinct motion signals would function
to prevent habitu-ation and ensure attention to seismic components
(Halland Channel 1985; Dill and Heisenberg 1995; Post andvon der
Emde 1999; Busse et al. 2005). We measureddifferent motion signals
and found that distinct seismicsignals occurred with specific
motion signals suggestinginter-signal interactions either to focus
attention or toconstruct integrative signals (Partan and Marler
1999,2005; Hebets and Papaj 2005). While this is not a con-clusive
test on whether there exist inter-signal interac-tions, it is
suggestive that selection has worked on theintegrated
multicomponent, multimodal signal.
Overall implications and limitations
In general, there are many potential applications of
thistechnique for measuring motion signals. Any aspect ofthe
repeated motion patterns can be measured (i.e.intervals between
patterns, duration of patterns, maxi-mum and minimum motion of
patterns, etc.) for use insubsequent analysis, and multiple aspects
of the speedprofiles can be treated simultaneously in
multivariateanalyses.
Rigorous classification techniques are desirable inmany
disciplines particularly in studies of animal com-munication. For
example, at the level of entire courtshipdisplays, this could be
used to identify motion parame-ters as characters for phylogenetic
analyses. At the level
2.5 1.5 0.5 -0.5 -1.5
-1.5
-0.1
0.1
1.5
1
2
3
4
5
6
7
89
10
11
12 1314
15
16
1718
19
20
21
Dimension 1
Dim
ensi
on 2
-1.5
-0.5
0.5
1.5
p
-
of individual signals, this is potentially useful in evalu-ating
natural variation in signals (Ryan and Rand 2003)and as a way to
measure signal complexity (e.g. howmany categories of visual
signals can be objectivelydiscriminated). These techniques could
also be valuablein comparative studies. For example, closely
relatedspecies that signal in different visual environments couldbe
compared to investigate the effect of the visual envi-ronment on
the design of motion displays (Endler 1991,1992; Peters et al.
2002; Peters and Evans 2003a, b).
This method of constructing speed profiles severelyreduces the
information present in the original video-tapes. Optic flow
analyses reduce video data to essen-tially five dimensions (speed,
speed spatial distribution,orientation, orientation spatial
distribution, and time)(Zeil and Zanker 1997; Zanker and Zeil 2001;
Peterset al. 2002). Some of these other motion parameters havebeen
used in other systems (Peters et al. 2002; Peters andEvans 2003a,
b). We chose to concentrate on speed andtime since jumping spider
displays can be very complexand often include the movement of
various body parts(forelegs, third leg patella, pedipalps)
superimposedupon the movement of the entire spider (Fig. 7). Any
ofthese dimensions however can be plotted for jumpingspider
displays (Fig. 7). Speed and time parameters maybe especially
important in jumping spiders due to thestructure of their visual
system. Jumping spiders havetwo categories of eyes: primary eyes
which have a
small field of view and are specialized for fine
spatialresolution, and secondary eyes which have a large fieldof
view are specialized for motion detection (Land 1969,1985; Land and
Nilsson 2002). Motion signals wouldmost likely stimulate secondary
eyes and therefore tim-ing and not spatial location is likely to be
importantsince secondary eyes integrate over a wide field of
view(Forster 1982a, b; Land 1985). This hypothesis remainsto be
tested and it is possible that jumping spiders arereducing the
visual field into timing information (inputfrom the secondary eyes)
and spatial information (inputfrom the primary eyes) independently.
In this scenario,our use of speed and timing parameters would
match‘‘data reductions’’ performed by secondary eyes.
Thisunderlines the importance of picking the correct datareduction
strategy based on insights from sensoryphysiology and behaviour.
Combining both spatial andtemporal analyses with an analysis on the
primary andsecondary eye fields of view could give insights into
howinformation is channelled into the nervous system(Strausfeld et
al. 1993). Complex motion signals in dif-ferent communication
systems may be specialized fordifferent dimensions. By combining
our technique withalternative analyses that focus on spatial rather
thantemporal motion patterns, it may be possible to developa
battery of analytical approaches to identify andanalyse the salient
parameters of a wide range ofcomplex visual motion displays.
20
40
60
80
100
0
40
60
80
100
120
0
0 20 40 60 80 100 120
20
40
60
80
100
Speed integrated over timey
pixe
l nu
mbe
r0
40
60
80
100
120
y pi
xel
num
ber
0
45 90 135 180x pixel number
Orientation integrated over time
1
0
1
2
x pixel number
A)
B)
1
0
1
2
Speed isoform surfaces
x pixel number
y pixel number
Fram
e nu
mbe
r
400
0
100
200
300
0
120 80 0
x pixel number
y pixel number
Fram
e nu
mbe
r
400
0
100
200
300
0100 100 75 25 0
0 45 90 135 1800 160 40
0
Fig. 7 Further optic flow parameters for courtship displays
ofdifferent populations of Habronattus pugillis.
Representativeexamples (same examples as in Fig. 3) from the
Galiuro (a) andSanta Catalina (b) mountain ranges are shown. First
column showsspeed spatial distributions throughout the video
integrated throughtime. Second column shows speed orientation
spatial distribution
throughout the video integrated through time. Third column
showspeed distribution isoform surfaces throughout the video with
timeon the z-axis and speed distribution on the x- and y-axis.
Timingpatterns are difficult to observe in speed and orientation
distribu-tion space. Frame rate is 30 fps
795
-
Regardless of this data reduction, distinct signalcategories can
still be discriminated using average speedand time parameters. The
sheer complexity of motiondisplays makes data reduction attractive
and we feel thatthis method reduces the data while constructing
accurate,intuitive depictions of the structure and timing of
motiondisplays in a way that may be biologically meaningful tothe
organism in question. While other parameters are nodoubt important,
we feel that using speed parameters andtime allows one to easily
observe repeating patterns in away that is difficult in other
parameter spaces (Fig. 7).
One potential limitation in this technique is the con-founding
effect of the number of edges on total motion.For example, if a
moving appendage differed betweentwo species solely in the number
of stripe patterns, thenfor equivalent leg movements, our analysis
would recorda higher motion signal for the animal with more
stripes.Such a difference in recorded motion signals due
toornaments could, however, reflect true differences in
theperceived signal at the receiver. Neural processing ofmotion in
animal brains is based on the movement ofedges defined by luminance
contrast and not edges de-fined by chromatic contrast. Edges
defined by chromaticcontrast are usually not perceived by animals;
thus, totaledge motion is total motion (Borst and Egelhaaf
1993).Therefore our algorithm analyses motion in a
biologicalway.
By expanding the technique developed by Peters et al.(2002), we
have developed a novel way to visualizemotion data analogous to
spectrogram representationsof auditory data as well as
demonstrating statisticaltechniques for analysing motion data. This
study dem-onstrates the utility of using optic flow techniques
toreduce and analyse motion in a variety of contexts (Zeiland
Zanker 1997; Zanker and Zeil 2001; Peters et al.2002; Peters and
Evans 2003a,b).
Acknowledgements We would like to thank M.C.B. Andrade,
C.Botero, C. Gilbert, J. Bradbury, B. Brennen, M.E. Arnegard,
E.A.Hebets, W.P. Maddison, M. Lowder, Cornell’s
NeuroethologyJournal Club, an anonymous reviewer, and members of
the Hoylab for helpful comments, suggestions, and assistance.
Spiderillustrations were generously provided by Margy Nelson.
Fundingwas provided by NIH and HHMI to RRH (N1DCR01 DC00103),NSERC
to ACM (238882 241419), NIH to BRL, and a HHMI Pre-Doctoral
Fellowship to DOE. These experiments complied with‘‘Principles of
animal care’’, publication no. 86–23, revised 1985 ofthe National
Institute of Health, and also with the current laws ofthe country
(USA and Canada) in which the experiments wereperformed.
References
Alexander RM (2003) Principles of animal locomotion.
PrincetonUniversity Press, Princeton
Barron JL, Fleet DJ, Beauchemin SS (1994) Performance of
opticflow techniques. IJCV 12:43–77
Borst A, Egelhaaf M (1993) Detecting visual motion: theory
andmodels. Rev Oculomot Res 5:3–27
Busse L, Roberts KC, Crist RE, Weissman DH, Woldorff MG(2005)
The spread of attention across modalities and space in
amultisensory object. PNAS 102:18751–18756
Cortopassi KA, Bradbury JW (2000) The comparison of
har-monically rich sounds using spectrographic cross-correlationand
principal coordinates analysis. Bioacoustics 11:89–127
Cox TF, Cox MAA (2001) Multidimensional scaling. Chapmanand
Hall, Boca Raton
Crane J (1949) Comparative biology of salticid spiders at
RanchoGrande, Venezuela. Part IV an analysis of display.
Zoologica34:159–214
Dill M, Heisenberg M (1995) Visual pattern memory withoutshape
recognition. Philos Trans R Soc Lond Ser B Biol Sci349:143–152
Eckert MP, Zeil J (2001) Towards an ecology of motion vision.
In:Zanker JM, Zeil J (eds) Motion vision: computational, neural,and
ecological constraints. Springer, Berlin Heidelberg NewYork, pp
333–369
Elias DO, Mason AC, Maddison WP, Hoy RR (2003) Seismicsignals in
a courting male jumping spider (Araneae: Salticidae).J Exp Biol
206:4029–4039
Elias DO, Mason AC, Hoy RR (2004) The effect of substrate onthe
efficacy of seismic courtship signal transmission in thejumping
spider Habronattus dossenus (Araneae: Salticidae).J Exp Biol
207:4105–4110
Elias DO, Hebets EA, Hoy RR, Mason AC (2005) Seismic signalsare
crucial for male mating success in a visual specialist
jumpingspider (Araneae:Salticidae). Anim Behav 69:931–938
Endler JA (1990) On the measurement and classification of color
instudies of animal color patterns. Biol J Linnean Soc
41:315–352
Endler JA (1991) Variation in the appearance of guppy
colorpatterns to guppies and their predators under different
visualconditions. Vision Res 31:587–608
Endler JA (1992) Signals, signal conditions, and the direction
ofevolution. Am Nat 139:S125–S153
Forster L (1982a) Vision and prey-catching strategies in
jumpingspiders. Am Sci 70:165–175
Forster L (1982b) Visual communication in jumping
spiders(Salticidae). In: Witt PN Rovner JS (eds) Spider
communica-tion: mechanisms and ecological significance.
PrincetonUniversity Press, Princeton, pp 161–212
Fry SN, Sayaman R, Dickinson MH (2003) The aerodynamics
offree-flight maneuvers in Drosophila. Science 300:495–498
Hall G, Channel S (1985) Differential effects of contextual
changeon latent inhibition and on the habituation of an
orientatingresponse. J Exp Psychol Anim Behav Process
11:470–481
Hasson O (1991) Sexual displays as amplifiers: practical
exampleswith an emphasis on feather decorations. Behav Ecol
2:189–197
Hebets EA, Maddison WP (2005) Xenophilic mating preferencesamong
populations of the jumping spider Habronattus pugillisGriswold.
Behav Ecol 16:981–988
Hebets EA, Papaj DR (2005) Complex signal function: developinga
framework of testable hypotheses. Behav Ecol
Sociobiol57:197–214
Hedrick TL, Usherwood JR, Biewener AA (2004) Wing inertia
andwhole-body acceleration: an analysis of instantaneous
aerody-namic force production in cockatiels (Nymphicus
hollandicus)flying across a range of speeds. J Exp Biol
207:1689–1702
Higgins LA, Waugaman RD (2004) Sexual selection and variation:a
multivariate approach to species-specific calls and
preferences.Anim Behav 68:1139–1153
Jackson RR (1982) The behavior of communicating in
jumpingspiders (Salticidae). In: Witt PN, Rovner JS (eds) Spider
com-munication: mechanisms and ecological significance.
PrincetonUniversity Press, Princeton, pp 213–247
Jindrich DL, Full RJ (2002) Dynamic stabilization of rapid
hexa-pedal locomotion. J Exp Biol 205:2803–2823
Land MF (1969) Structure of retinae of principal eyes of
jumpingspiders (Salticidae : Dendryphantinae) in relation to
visualoptics. J Exp Biol 51:443–470
Land MF (1985) The morphology and optics of spider eyes.
In:Barth FG (ed) Neurobiology of arachnids. Springer,
BerlinHeidelberg New York, pp 53–78
Land MF, Nilsson DE (2002) Animal eyes. Oxford UniversityPress,
Oxford
796
-
Maddison WP (1996) Pelegrina franganillo and other
jumpingspiders formerly placed in the genus Metaphidippus
(Araneae:Salticidae). Bull Mus Comp Zool Harvard Univ
154:215–368
Maddison W, Hedin M (2003) Phylogeny of Habronattus
jumpingspiders (Araneae : Salticidae), with consideration of
genital andcourtship evolution. Syst Entomol 28:1–21
Maddison W, McMahon M (2000) Divergence and reticulationamong
montane populations of a jumping spider (Habronattuspugillis
Griswold). Syst Biol 49:400–421
Maddison WP, Stratton GE (1988) Sound production and associ-ated
morphology in male jumping spiders of the Habronattusagilis species
group (Araneae, Salticidae). J Arachnol 16:199–211
Masta SE (2000) Phylogeography of the jumping spider
Habro-nattus pugillis (Araneae: Salticidae): recent variance of sky
is-land populations? Evolution 54:1699–1711
Masta SE, Maddison WP (2002) Sexual selection driving
diversi-fication in jumping spiders. PNAS 99:4442–4447
Nauen JC, Lauder GV (2002) Quantification of the wake of
rain-bow trout (Oncorhynchus mykiss) using three-dimensional
ste-reoscopic digital particle image velocimetry. J Exp
Biol205:3271–3279
Partan SR, Marler P (1999) Communication goes multimodal.Science
283:1272–1273
Partan SR, Marler P (2005) Issues in the classification of
multi-modal communication signals. Am Nat 166:231–245
Peckham GW, Peckham EG (1889) Observations on sexual selec-tion
in spiders of the family Attidae. Occas Pap Wisconsin NatHist Soc
1:3–60
Peckham GW, Peckham EG (1890) Additional observations onsexual
selection in spiders of the family Attidae, with some re-marks on
Mr. Wallace’s theory of sexual ornamentation. OccasPap Wisconsin
Nat Hist Soc 1:117–151
Peters RA, Evans CS (2003a) Design of the Jacky dragon
visualdisplay: signal and noise characteristics in a complex
movingenvironment. J Comp Physiol A 189:447–459
Peters RA, Evans CS (2003b) Introductory tail-flick of the
Jackydragon visual display: signal efficacy depends upon duration.J
Exp Biol 206:4293–4307
Peters RA, Clifford CWG, Evans CS (2002) Measuring the
struc-ture of dynamic visual signals. Anim Behav 64:131–146
Post N, von der Emde G (1999) The ‘‘novelty response’’ in
anelectric fish: response properties and habituation. Physiol
Behav68:115–128
Richman DB (1982) Epigamic display in jumping spiders
(Araneae,Salticidae) and its use in systematics. J Arachnol
10:47–67
Ryan MJ, Rand AS (2003) Sexual selection in female
perceptualspace: how female tungara frogs perceive and respond
tocomplex population variation in acoustic mating signals.
Evo-lution 57:2608–2618
Strausfeld NJ, Weltzien P, Barth FG (1993) Two visual systems
inone brain: neuropils serving the principle eyes of the
spiderCupiennius salei. J Comp Neurol 328:63–72
Tammero LF, Dickinson MH (2002) The influence of visuallandscape
on the free flight behavior of the fruit fly
Drosophilamelanogaster. J Exp Biol 205:327–343
Victor JD, Purpura KP (1997) Metric-space analysis of
spiketrains: theory, algorithms and application. Netw Comp
Neural8:127–164
Vogel S (2003) Comparative biomechanics: life’s physical
world.Princeton University Press, Princeton
Walker TJ (1974) Character displacement and acoustic insects.
AmZool 14:1137–1150
Zanker JM (1996) Looking at the output of two-dimensional
mo-tion detector arrays. IOVS 37:743
Zanker JM, Zeil J (2001) Motion vision: computational,
neural,and ecological constraints. Springer, Berlin, Heidelberg,
NewYork
Zeil J, Zanker JM (1997) A glimpse into crabworld. Vision
Res37:3417–3426
797
Measuring and quantifying dynamic visual signals in jumping
spidersAbstractIntroductionMethodsSpidersRecording proceduresMotion
analysisCropping/intensity normalizationOptical flow
calculationSpeed profile plotsMaximum cross-correlation of 1D and
2D signalsMultidimensional scaling \(MDS\)ResultsMotion algorithm
calibrationHabronattus pugillis populationsFig1Fig2Fig3Habronattus
dossenus signalsSpatial informationFig4DiscussionFig5Quantitative
analysis of courtship signals ldquo Sky island rdquo
populationsMulticomponent signalsOverall implications and
limitationsFig6Fig7AcknowledgementsReferencesCR1CR2CR3CR4CR5CR6CR7CR8CR9CR10CR11CR12CR13CR14CR15CR16CR17CR18CR19CR20CR21CR22CR23CR24CR25CR26CR27CR28CR29CR30CR31CR32CR33CR34CR35CR36CR37CR38CR39CR40CR41CR42CR43CR44CR45CR46CR47CR48CR49CR50CR51CR52CR53CR54
