Tashtego software: Real-time display and event detection for MMO assistance

Distance calculation Examples of tracked whales

Detection and Tracking of Whales Using a Ship-Borne, 360° Thermal Imaging SystemL. Kindermann1, D.P. Zitterbart1,2, E. Burkhardt1 and O. Boebel1

1AWI, Bremerhaven, Germany; 2Institute for Condensed Matter Physics, Department of Physics, Erlangen University, GermanyContact Information: lars.kindermann@awi.de

Marine mammal observations Shut-down of under-water noise producing activities in thepresence of whales is meanwhile a standard mitigationprocedure required by many regulating agencies Determination of whale presence is thereby mostly based on visual sightings of the whale’s blow by dedicated observers Visual sightings are restricted to daylight hours, require utmost concentration by observers and large teams when conducted during month-long cruises An automatic detection system with night-vision could support MMO’s detection efforts

Mot

ivat

ion

Thermal images provide clear signals of blows under varying (polar/subpolar) environmental conditions

water temperature = -0.23°C water temperature = 6.15°Cwater temperature = 3.46°C

This project was funded by the German Federal Ministry of Education and Research (grant Nr. BMBF 03F0479I) and the Federal Ministry for the Environment, Nature Conservation and Nuclear Safety (grant Nr. BMU 370891101-01). The responsibility for the contents of this publication rests solely with the authors.

Det

ectio

n

0 500 1000 1500 2000 2500 3000-200

-150

-100

-50

0

50

100

150

200

distance [m]

1 pi

xel l

ow

d

ista

nce

erro

r [m

]

1

pixe

l hig

h

0 500 1000 1500 2000 2500 3000-6

-4

-2

0

2

4

6

8

distance [m]

1 pi

xel l

ow

d

ista

nce

erro

r [%

]

1

pixe

l hig

h

-3000 -2000 -1000 0 1000 2000 3000 4000-3000

-2000

-1000

0

1000

2000

3000Blow positions relative to ship

Distance [m]

Dis

tanc

e [m

]

500

1000

1500

2000

2500

3000

250 750 1250 1750 2250 2750 3250 3750 4250 4750 52500

20

40

60

distance [m]

# of

blo

ws

dete

cted

0 1000 2000 3000 4000 5000 60000

5

10

distance [m]

freq.

of b

low

per

km

2

Trac

king

s

Type Resolution (°/ reticule) Field of view

very best cond. (Big Eye) 0.01°

at sea „…worse than 0.1°…“

Fujinon 7 x 50 0.286° 10.0° x 10.0°

Fujinon 25 x 150 0.08° 2.7° x 2.7°

Leviathan 0.057° 2.6° x 2.6°

FIRST Navy 0.0313° 360.0° x 18.0°

Absolute error of dist. calculation Relative error of dist. calculation

The MAPS Project Goal: Develop operational automatic detection system to assist mitigation and research Step 1: Generate and handle thermal image video stream Step 2: Develop real-time automatic detector for blow signatures Step 3: Determine system performance for different environmental conditions / species

750 m

250 m

500 m

1000 m

Horizon (adjustable)events (tracked)

latest event (tracked)

Results Proof of concept established Image quality excellent, both night and day Long term stability of image acquisition system (FIRSTNavy sensor hardware) currently insufficient, debuggingin progress Graphical user interface (Tashtego) operational 5 weeks (~837 hrs) operation in Greenland Sea andSouthern Ocean 4.7 TB data archived for retrospective analysis anddevelopment of automatic pattern recognition algorithm ~500 blows detected by retrospective visual screeningof IR video Blow detection in waters as warm as up to 10°C Polar detection range ≈3 km (< 6km) First automatic algorithm detects 19 of 20 blows

N reticulesevent

horizonN pixelblow

horizon (19388 m)

Distance calculations after Lerczak, J. A. and R. C. Hobbs (1998): r = r (pixel between horizon and event, instrument height, resolution)

FIRST Navy thermal image Leviathan Big Eye

FIRST Navy sensor in RV Polarstern’s crows nest

FIRST Navy IR thermal imager by RDE, Bremen- Rotating, cooled (80°K) line sensor- Field of view: 360° (horizontal) x 18°(vertical)- Frame rate: 5 Hz (0.2s resolution)- Thermal resolution: 0.0004°K- Image resolution: 7200 (horizontal) x 576 ..(vertical) pixel- Data rate: 3.5 Tbyte/dayFully stabilized platform (gimbal) (± 12° roll/pitch)- Mounted in crow’s nest, 29 m height- Effective field of view: 300° (horizontal)

from about 100 m from ship to horizon

-3000 -2000 -1000 0 1000 2000-3000

-2000

-1000

0

1000

2000

3000

500

1000

1500

2000

25003000

Humpback whales (2010-02-08)

©AWI: 20100208-182118-2

166° 25' W 166° 23' W 166° 21' W 166° 19' W

65° 31' S

65° 30' S

65° 29' S

0 10 20 30 40time since first detection [min]

A group of at least 3 humpback whales at 1 km to > 3 km distancePeriod analyzed: 18:00 – 18:54; First detection: 18:21:18; Last detection: 18:52:53 Time between synchronized dives: about 10 min Apparently prolonged period near surface ( 15 min) during ship’s passage No overt escape response

1852 m

Blow positions relative to ship

Minke Whale (2010-02-13)Short track of a single minke whale crossing in front of ship’s bowPeriod analyzed: 01:31 – 01:38; First detection: 01:32:11; Last detection: 01:37:01 Relatively constant inter-blow interval throughout observations: 36 ± 13 s change of swim direction and speed from “along” (2.2 ms-1) to “away” (6.4 ms-1)

©AWI: 20100213-013211-0

159° 48' W 159° 47' W 159° 46' W 159° 45' W

72° 18' S

0 1 2 3 4 5time since first detection [min]

-1000 -500 0 500 1000

-1000

-500

0

500

1000

500

1000

Blow positions relative to ship

1852 m

zoom/track via mouse click