The Visual Targeting TurretFinal Presentation

Jason Lam

John Lee

Jonathan Rothberg

Overview

Objectives and Motivation Design SpecificationsDesign Approach, Validation and Obstacles

HardwareMathematical ModelingImage ProcessingParameter IdentificationControl Design

Results / Demonstration AssessmentConclusions

Objectives

Visually track a moving target on the floor in various lighting conditions

Infrared spectrum Targeting via webcam and laser pointerMATLAB image processing

Target travels freely around the visual systemMust be within the camera’s range of view (180° Pan)

Shoot Target (RC Car)Target has laser detection capabilityTarget signals a hit with a buzzer and LED

Motivation

Defense systemsNeutralizing threats without risking human lifeNighttime operation

Civilian and commercial applicationsTraffic controlSecurity systems

EntertainmentAutomated Laser-Tag player

SpecificationsCar Velocity: 0.684788 m/s (weighted car), 1.3252 m/s (non-weighted car)Height

Tilt Axle: 0.92 m Tilt Axle to camera: 3.4 cmTilt Axle to laser pointer: 5.7 cm

Motion range: Tilt: 30.5° to 68.5° (from the horizontal plane)Pan: 180°

Degree of view:Pan: 18.05° from centerTilt: 27.5345° from center

Specifications (Controller)

Settling time = 0.17 sec Less than average image processing time

Overshoot < 1%Slight overshoot may behave as a “predictive”algorithm

Steady State Error = 0

Design Approach: HardwareLaser Detection Circuit

Infrared emitters and red laser photodiodes

Variable SensitivityStatus LED and buzzerLayout

Design Approach: Hardware

Mounting System to Pan-Tilt Assembly

Minimize inertiaRigid mount to maintain geometry

Obstacles: Hardware

Target:Target speedLight reflection

Mounting:FabricationPrecision

Obstacles: Hardware

Proposed Design Implemented Design

Obstacles: Hardware

Maintaining the geometry of implemented design

Design Approach: Mathematical Modeling

Coordinate System TranslationsNon-linear relationship between tilt and actual distanceDegrees per pixel is linear

Parallax IssuesLaser pointer is above the cameraConvergence point depends on distance from assemblyVirtual Centerpoint System

Tilt compensation

Design Approach: Image Processing

Maximize frames per secondMinimize distance car travels between framesProvide fast processing of images

Robust and reliableAccurately find moving car 90% - 100% of the time

Design Approach: Image Processing Design

Direct search of image’s pixel values Collect locations of pixel values above a specified threshold Mean of selected pixel locations determines centroid of target

Target detection accuracy and performanceDetect target based on colorDetect target in dim lighting conditionsCorrectly detect car in noisy environmentOmit slow MATLAB sub-routines

Design Approach: Image Processing

Streamlined algorithmFrames per second: 10Image processing time: 0.1 seconds on averageAccurately detects target in normal or dim lighting conditions

The elapsed time is 0.151 seconds

Obstacles: Image Processing

Adequate performanceSlow MATLAB functionsCamera performance under dim lighting conditions

Erroneously tracked laser pointer instead of actual target

Color processing included RedIR LED’s emit blue colorNeed to be only concerned with blue and green

Modeling and Verificationvia Friction Identification

LLcMLMLLMLLML mglNBNBBNBJNJCCVV

ττ −Θ+=Θ′++Θ′++Θ′′+ )sin(*)sgn()*(*)*(*)*( 2

MLLL NaaI τ=Θ′+Θ′+Θ′′ )sgn(*** 21

12 /))0699.0(( aaVNL −=Θ′

Pan Friction Identification

Velocity vs. Time

Pan Negative Range:-2.6 V to -1.85 V (impulse of -3 V)

Pan Positive Range:

2.4 V to 1.75 V (impulse of 3 V)

Increments of 0.05 V

0 5 10 15 20 25 30-40

-30

-20

-10

0

10

20

30

40Velocity vs. Time Family of Curves

Time (sec)

Vel

ocity

(rad

/sec

)

Pan Friction Identification

Viscous Friction:Positive:0.0010 Nm*s/rad

Negative:0.0013 Nm*s/rad

Coulomb Friction:Positive:0.1341 Nm

Negative:-0.1345 Nm

-40 -30 -20 -10 0 10 20 30 40-0.2

-0.15

-0.1

-0.05

0

0.05

0.1

0.15

0.2Friction Identification Data

Steady State Theta Dot (rad/sec)

App

lied

Torq

ue (N

-m)

Tilt Friction Identification

Velocity vs. Time

Tilt Negative Range:-2.3 V to -1.5 V (impulse of -2.5 V)

Tilt Positive Range:

1.9 V to 1.4 V (impulse of 2.1 V)

Increments of 0.05 V

0 5 10 15 20 25 30-40

-30

-20

-10

0

10

20

30

40Velocity vs. Time Family of Curves

Time (sec)

Vel

ocity

(rad

/sec

)

Tilt Friction Identification

Viscous Friction:Positive:0.0008 Nm*s/rad

Negative:0.0014 Nm*s/rad

Coulomb Friction:Positive:0.1030 Nm

Negative:-0.1159 Nm

-40 -30 -20 -10 0 10 20 30 40-0.2

-0.15

-0.1

-0.05

0

0.05

0.1

0.15Friction Identification Data

Steady State Theta Dot (rad/sec)

App

lied

Torq

ue (N

-m)

Modeling and Verificationvia Friction Identification

Modeling and Verificationvia Friction Identification

Modeling and Verificationvia Parameter Identification

Modeling and Verificationvia Parameter Identification

Modeling and Verificationvia Parameter Identification

0 5 10 15 20 25 30-50

-40

-30

-20

-10

0

10

20

30

40

50

Time (sec)

Vel

ocity

(rad

/sec

)

Open-Loop Response (Chirp) - Actual and Simulated

Simulated

Actual

Controller Design

Pan Axis

Mo = 0.3906tp = 0.1850tr = 0.0680ts = 0.1160ess = 0.3906

0 0.5 1 1.5 2 2.5 3 3.50

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

time (s)

Vol

tage

(V)

Closed loop system response to pi/4 step input

actualdesired

Controller Design

Tilt Axis

Mo = []tp = 2.3950tr = 0.0868ts = 0.1750ess = 0.3906

0 500 1000 1500 2000 2500 3000 35000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

time (s)

Vol

tage

(V)

Closed loop system response to a pi/4 step input - tilt

actualdesired

Response to a Moving Target Demonstration Run

0 1 2 3 4 5 6 7 8-1.6

-1.4

-1.2

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

time (s)

posi

tion

(thet

a)

Desired vs Actual Response to a Demo Run - Pan

actualdesired

0 1 2 3 4 5 6 7 8-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4Desired vs. Actual Response to a Demo Run - Tilt

posi

tion

(thet

a)

time (s)

actualdesired

The Current vs. Past System Implementation

CurrentAble to achieve more than 5 frames/sec

PastTarget needs to stop between iterations to allow the camera to capture the current target position

The Difference – memory storageNew implementation integrates system into one Simulink model, excluding the image processing functionOld implementation runs the system in two models in order to store telemetry values

New System Implementation

6pan angle input

5ti l t angle input

4pan thetadot

3ti lt thetadot

2current pan angle

1current ti lt angle

127

yLoc on screen

160

xLoc on screen

In1 Out1

ti l t PID controller

In1 Out1

pan PID controller

In1 Out1

hold pan angle

1

flag

z

1

Unit Delay1

z

1

Unit Delay

PCIM-DAS1602/16ComputerBoardsAnalog Output

1

2

PCIM-DAS1602 16

PCI-QUAD04Comp. BoardsInc. Encoder

Count

PCI-QUAD04 1

PCI-QUAD04Comp. BoardsInc. Encoder

Count

PCI-QUAD04

1/0.001

Gain3

2*pi/(1024*4)

Gain2

1/0.001

Gain1

2*pi/(1024*4)

Gain

tilt_angle

x_px

y _px

pan_angle

desired_tilt

desired_pan

fcn

Desired Pan-Tilt Angles for (x, y) px on Screen

In1 Out1

hold ti l t angle

Old Implementation – part 1

6tilt angle input

5pan angle input

4Out5

3Out4

2current pan angle

1current ti lt angleIn1 Out1

tilt PID controller

In1 Out1

pan PID controller

desired ti lt angle difference

desired pan angle difference

z

1

Unit Delay1

z

1

Unit Delay

PCIM-DAS1602/16ComputerBoardsAnalog Output

1

2

PCIM-DAS1602 16

PCI-QUAD04Comp. BoardsInc. Encoder

Count

PCI-QUAD04 1

PCI-QUAD04Comp. BoardsInc. Encoder

Count

PCI-QUAD04

1/0.001

Gain3

2*pi/(1024*4)

Gain2

1/0.001

Gain1

2*pi/(1024*4)

Gain

Old Implementation – part 2

The old tilt angle is stored in MATLABThe desired pan and tilt is dependent on the current tilt angle, and it is necessary to stop (tg) before that value can be stored

2Out2

1Out1

yLoc

y px

xLoc

x px

old_ti lt

old ti l t angle

0

object distance

0

desired tilt rad

0

desired pan rad

current_til t

current ti l t angle

current_pan

current pan angle

old_tilt

tilt_angle

x_px

y _px

pan_angle

desired_tilt_dif f

desired_pan_dif f

camera_dist

fcn

Desired Pan-Tilt Angles for (x, y) px on Screen

Results / Demonstration

Stationary Demo

Results / Demonstration

Stationary Demo: Camera screenshots

Results / Demonstration

Initial System Implementation (Slow)

Results / Demonstration

Initial System Implementation (Slow): Camera Screenshots

Results / Demonstration

Final System Implementation (Fast)

Assessment

The target is no longer necessary to be weighted down for the system to trackController and image processing is robust enough to hit the targetThe mathematical model accurate enough to hit target within 2 screen captures

Self corrects to the hit the target

Future Enhancements

A true predictive algorithmAchieve a more continuous camera feedbackEnabling laser to shoot pulses