Tracking Migratory BirdsAround Large Structures

Presented by: Arik Brooks and Nicholas Patrick

Advisors: Dr. Huggins, Dr. Schertz, and Dr. Stewart

Senior Design Project 2003-2004Bradley University

Department of Electrical and Computer Engineering

Project Background

• Every year, many birds are killed when their migration path takes them near tall structures on overcast nights.

• One widely accepted theory on why this happens is that the birds do not want to leave the lighted area near a structure and end up running into it.

• Wildlife biologists would like to study this phenomenon.

Outline

• Project summary

• Previous Work

• Detailed description

• System block diagram

• Subsystems

• Results

Outline

• Test Plan

• Datasheet

• Conclusions

• Suggestions for future work

• Questions

Project Summary

• The purpose of this project is to implement a system to track the flight paths of birds in real-time via stereoscopic imaging.

• The desired system output is a display depicting a 3-D representation of the trajectories of the birds, and data relating to the trajectories.

Previous Work

• 2003 seniors Brian Crombie and Matt Zivney

• Results: basic object position location in a laboratory environment with major limitations.

• The groundwork laid out in their project (algorithms, design equations, software organization, etc.) was used as a starting point for our system.

Detailed Description

System Block Diagram

System

Hardware Block Diagram

Subsystems

• Cameras

• Frame Grabber

• PC’s/Network

Camera Subsystem

• Includes two cameras mounted in parallel a known distance apart allowing objects to be located in space.

• Inputs– Photons -- Images collected by the cameras– Synchronization -- Internal line lock

• Outputs– Data -- Image data transmitted to the frame

grabber• Operation in System

– The cameras capture images at a rate dictated by the speed of the preprocessing algorithm

Frame Grabber Subsystem

• The frame grabber simultaneously captures images from both cameras and supplies the digitized image data to the PC.

• Inputs– Data -- Image data (NTSC format) from the cameras– Setup -- Information from the PC

• Outputs– Image Data to PC

• Operation in System– The frame grabber operates at a rate dictated by the

speed of the preprocessing algorithm

PC’s/Network Subsystem

• Two PC’s are networked together to divide computation between the preprocessing and trajectory calculation computers.

• Inputs– Image Data -- Arrays of intensity information– Calibration Input -- Calibration data for the cameras being

used• Outputs

– Display – GUI showing trajectories plotted in a three dimensional representation

– Statistics -- Pertinent data calculated from bird trajectories – Raw Data -- Data file containing all preprocessed data

• Operation in System– The PC’s and network operate continuously

Results

Preprocessing Software

Streamlined Preprocessing in C++

• Implement faster centroid location code.– Perimeter search vs. pixel-by-pixel search

• Improve background subtraction algorithm:

– Fixed number of frames averaged for background to 256

– Current frame added using shift operations instead of multiplies/divides

– Stored background is 16 bits:• upper 8 bits are image data

• lower 8 bits for accumulating round-off error

ageCurrent_im8)d(BackgrounBackgroundBackground

Streamlined Preprocessing in C++

• Improve background subtraction algorithm:

– Speed Improvements (640x480, threshold image, do not find objects)

• Old -- 10.6 Frames per Second

• New – 15.9 Frames per Second

– Updating average every 60 frames• Without find object function -- 24 Frames

per Second

• With find object function -- 18 Frames per Second

Preprocessing in C++

Trajectory Determination in MATLAB

• Code to correlate objects between 2 cameras and over time restructured

• Added predictive searching to significantly improve tracking ability

• Improved graphing techniques real-time operation

• Implemented GUI for easy user interface

Trajectory Determination comparison - two tennis balls swinging

• System from last year:

Trajectory Determination comparison - two tennis balls swinging

• Current algorithm:

Trajectory Determination Software

Trajectory Determination in MATLAB

• Predictive Search Method– Search for a new point within a

sphere defined by:• Center at the location (x,y,z)

predicted by the previous two points in the trajectory and the time taken between frame-grabs

• Radius determined by average bird velocity, time between frames, current velocity, and distance from the cameras

Trajectory Software GUI

Test Plan• There will be three primary test procedures that

will be performed to verify the system specifications:

– Location Accuracy– Max/Min Distance from Cameras– Max # Objects

Tennis Ball dispenser used in accuracy testing

Test Plan• Location Accuracy

– Capture data at known heights using a “stationary” object and balls dropped from the tennis ball dispenser. Compare theoretical to experimental.

• Max/Min Distance from Cameras– Repeat ‘Location Accuracy’ experiment at

extremes of range.

• Max # Objects– Nerf Guns!!!

T-Bird Accuracy Test #1

X (height m) Z (distance m) X (height m) Z (distance m) X (height m) Z (distance m)Desired -1.00 3.00 -1.00 6.00 -1.00 9.00Actual x x 0.93 6.23 -0.96 8.90Desired -0.50 3.00 -0.50 6.00 -0.50 9.00Actual -0.48 3.15 -0.40 5.81 -0.45 8.71Desired 0.00 3.00 0.00 6.00 0.00 9.00Actual 0.04 2.83 0.09 5.64 0.05 8.50Desired 0.50 3.00 0.50 6.00 0.50 9.00Actual 0.56 3.19 0.52 5.65 0.56 8.50Desired 1.00 3.00 1.00 6.00 1.00 9.00Actual x x 0.94 5.67 0.97 8.50

max deviation 0.04 0.19 0.10 0.36 0.06 0.50

X = 0 is at a height equal to the height of the lower camera, which is at 5 feet from the floor

3 meters 6 meters 9 meters

T-Bird Accuracy Test #2Predicted X Measured Time from drop Desired Measured

X (height m) based on drop time Frametime (ms) Using X measured Z (distance m) Z (distance m)Point 1 1.85 x 78 377.96 3.00 2.67Point 2 1.56 1.53 78 449.49 3.00 3.22Point 3 1.16 1.10 94 532.61 3.00 3.12max deviation

Predicted X Measured Time from drop Desired MeasuredX (height m) based on drop time Frametime (ms) Using X measured Z (distance m) Z (distance m)

Point 1 1.90 x 78 365.45 6.00 6.10Point 2 1.57 1.58 79 447.04 6.00 6.23Point 3 1.16 1.12 93 532.01 6.00 6.52max deviation

Predicted X Measured Time from drop Desired MeasuredX (height m) based on drop time Frametime (ms) Using X measured Z (distance m) Z (distance m)

Point 1 2.21 x 78 264.90 9.00 8.77Point 2 1.96 1.92 93 347.97 9.00 9.05Point 3 1.62 1.66 79 435.55 9.00 9.35Point 4 1.21 1.26 78 523.26 9.00 9.35max deviation

, based on Physics equations of motion

X = 0 is at floor level, 1.524 m (5 ft) was added to the value found by the system to produce thevalues recorded in the column "X (height)"

0.35 m0.05 m

3 meters

6 meters

9 meters

0.33 m0.06 m

0.04 m 0.52 m

T-Bird Demonstration

T-Bird Demonstration

Datasheet

• Average Migratory Bird Diameter: 0.152 m• Average Migratory Bird Speed: 8.9409 m/s• Max # of Objects Tracked Simultaneously: TBD• Max Distance from Cameras: 20 m• Min Distance from Cameras: 3 m• Max Location Error (theoretical): 0.375 m• Light Level Sensitivity:

– Lab Cameras: 0.22 Lux– Low Light Cameras: 0.0002 Lux

• Max Framerate: ~15 FPS• Total Volume of Space Observed: 606 m3

Separation of Cameras assumed for calculations: 0.5 m

Conclusions• Real-time tracking of multiple

objects was achieved in a laboratory setting.

Suggestions for Future Work

• Implement boom (mechanical system and controls)

• Obtain and integrate high end cameras

• Optimize code (analyze algorithms, streamline processes)

• Port MATLAB to C• Investigate feature detection

methods for improved target recognition

Tracking Migratory BirdsAround Large Structures

Questions?