Introduction To Tracking

Introduction To TrackingMario Haddad2What is Tracking?Estimating pose (state) Possible from a variety of measured sensorsElectricalMechanicalInertialOpticalAcousticMagnetic

DYNAMIC SCENE ANALYSIS

Typical ApplicationsMotion detection. Often from a static camera.

Object localization.

Three-dimensional shape from motion.

Object tracking. 1)Common in surveillance systems. Often performed on the pixel level only (due to speed constraints).

2) Focuses attention to a region of interest in the image.

3) Called also structure from motion. Similar problems as in stereo vision.

4) A sparse set of features is often tracked, e.g., corner points.

4Example Application

TRACKING , , " " .. 5Object Tracking DefinitionObject tracking is the problem of determining (estimating) the positions and other relevant information of moving objects in image sequences.Difficulties In Reliable Object TrackingRapid appearance changes caused by image noise,illumination changes, non-rigid motion, ... Non-stable backgroundInteraction between multiple objects...

Difficulties In Reliable Object TrackingRobust Density Comparison for Visual Tracking (BMVC 2009)Difficult, but not impossible!Difficulties In Reliable Object Tracking

9Motion EstimationBlock Matching MethodFor a given region in one frame, find the corresponding region in the next frame by finding the maximum correlation score (or other block matching criteria) in a search region

Block Matching Method

Block Matching Method

13

(a) (b)Optical Flow Motion Field14(a)A smooth sphere is rotating under constant illumination. Thus the optical flow field is zero, but the motion field is not.(b)A fixed sphere is illuminated by a moving sourcethe shading of the image changes. Thus the motion field is zero, but the optical flow field is not.

Visible Motion and True MotionOPTIC FLOW - apparent motion of the same (similar) intensity patternsGenerally, optical flow corresponds to the motion field, but not always:

Motion is going right but optic flow is going up15Local Features for TrackingIf strong derivatives are observed in two orthogonal directions then we can hope that this point is more likely to be unique. Many trackable features are called corners. Harris Corner Detection !

Intuitively, cornersnot edgesare the points that contain enough information to be picked out from one frame to the next.

16Aperture Problem17The Aperture ProblemDifferent motions classified as similar source: Ran EshelThe Aperture ProblemSimilar motions classified as different source: Ran EshelTracking MethodsMean-Shift

The mean-shift algorithm is an efficient approach to tracking objects whose appearance is defined by histograms.(not limited to only color)Motivation

Motivation to track non-rigid objects, (like a walking person), it is hard to specifyan explicit 2D parametric motion model.

Appearances of non-rigid objects can sometimes be modeled with color distributionsMean Shift TheoryDistribution of identical billiard ballsRegion ofinterestCenter ofmassMean ShiftvectorObjective : Find the densest regionStolen from: www.wisdom.weizmann.ac.il/~deniss/vision_spring04/files/mean_shift/mean_shift.ppt Intuitive Description24Distribution of identical billiard ballsRegion ofinterestCenter ofmassMean ShiftvectorObjective : Find the densest regionStolen from: www.wisdom.weizmann.ac.il/~deniss/vision_spring04/files/mean_shift/mean_shift.ppt Intuitive Description25Distribution of identical billiard ballsRegion ofinterestCenter ofmassMean ShiftvectorObjective : Find the densest regionStolen from: www.wisdom.weizmann.ac.il/~deniss/vision_spring04/files/mean_shift/mean_shift.ppt Intuitive Description26Distribution of identical billiard ballsRegion ofinterestCenter ofmassMean ShiftvectorObjective : Find the densest regionStolen from: www.wisdom.weizmann.ac.il/~deniss/vision_spring04/files/mean_shift/mean_shift.ppt Intuitive Description27Distribution of identical billiard ballsRegion ofinterestCenter ofmassMean ShiftvectorObjective : Find the densest regionStolen from: www.wisdom.weizmann.ac.il/~deniss/vision_spring04/files/mean_shift/mean_shift.ppt Intuitive Description28Distribution of identical billiard ballsRegion ofinterestCenter ofmassMean ShiftvectorObjective : Find the densest regionIntuitive Description29Distribution of identical billiard ballsRegion ofinterestCenter ofmassObjective : Find the densest regionStolen from: www.wisdom.weizmann.ac.il/~deniss/vision_spring04/files/mean_shift/mean_shift.ppt Intuitive Description30Mean Shift VectorGiven:Data points and approximate location of the mean of this data:

Task:Estimate the exact location of the mean of the data by determining the shift vector from the initial mean.

We keep doing this iteratively until we do not have to move any more. Until the shift vector is zero.31

Mean Shift VectorNx = the number of pointsWe find the mean by adding up all the points, which are represented by a two dimensional vector (x,y) and dividing by the total number of points.Then we substract the point where we started, so we get a vector.The mean shift vector always points to the direction of the densest point of data points32A Quick PDF DefinitionAprobability density function(pdf), is a functionthat describes the relative likelihood for this random variable to take on a given value. . .33Mean-Shift Object TrackingTarget Representation

Choose a reference target modelQuantized Color SpaceChoose a feature spaceRepresent the model by its PDF in the feature space

Stolen from: www.cs.wustl.edu/~pless/559/lectures/lecture22_tracking.ppt

Mean-Shift Object TrackingPDF RepresentationTarget Model(centered at 0)Target Candidate(centered at y)

SimilarityFunction:Q is the target histogram,P is the object histogram (depends on location y)Stolen from: www.cs.wustl.edu/~pless/559/lectures/lecture22_tracking.ppt

Mean-Shift Object TrackingTarget Localization AlgorithmStart from the position of the model in the current frame

Search in the models neighborhood in next frame

Find best candidate by maximizing a similarity func.Stolen from: www.cs.wustl.edu/~pless/559/lectures/lecture22_tracking.ppt Mean ShiftMean-Shift in tracking task: track the motion of a cluster of interesting features. 1. choose the feature distribution to represent an object (e.g., color + texture), 2. start the mean-shift window over the feature distribution generated by the object 3. finally compute the chosen feature distribution over the next video frame.Mean ShiftStarting from the current window location, the mean-shift algorithm will find the new peak or mode of the feature distribution, which (presumably) is centered over the object that produced the color and texture in the first place.

In this way, the mean-shift window tracks the movement of the object frame by frame.Examples

Examples

Other Mean Shift ApplicationsEdge Preserving Smoothing

Segmentation

Contour Detection

Rudolf Emil KalmanBorn in 1930 in HungaryBS and MS from MITPhD 1957 from ColumbiaFilter developed in 1960-61Now retired

Kalman FilterTHE KALMAN FILTER IS OVER 50 YEARS OLD BUT IS STILL ONE OF THE MOST IMPORTANT AND COMMON DATA FUSION ALGORITHMS IN USE TODAY.

45Kalman FilterThe Kalman filter operatesrecursivelyon streams of noisy input data to produce a statistically optimalestimateof the underlyingsystem state.Noisy data in hopefully less noisy data out, , , , . .46Motivation

"" . .

47Tracking objects (e.g., missiles, faces, heads, hands)NavigationMany computer vision applications Stabilizing depth measurements Feature tracking Cluster tracking Fusing data from radar, laser scanner andstereo-cameras for depth and velocity measurements Many moreKalman Filter ApplicationsIntuitionRobotOdometer GPSSand

We may encounter:Wheel spinGPS inaccuracy

GPSOdometerPrevious state ... PREVIOUS STATE, , . . ODOMETER GPS ODOMETER , . GPS GPS ODOMETER.49Kalman Filter

Not perfectly sure. Why ? . 100 .50Kalman FilterKalman filter finds the most optimum averaging factor for each consequent state.

somehow remembers a little bit about the past states. 51Kalman Filter

State Prediction:Measurement Prediction: X K .52Two groups of the equations for the Kalman filter:Time update equations (Prediction) Measurement update equations. (Correction)

The time update equations are responsible for projecting forward (in time) the current state and error covariance estimates to obtain the a priori estimates for the next time step.

The measurement update equations are responsible for the feedbacki.e. for incorporating a new measurement into the a priori estimate to obtain an improved a posteriori estimate.Kalman Filter : , (, ) . , ( ). . . . , , , .53Brace Yourselves..

PredictPredict the state ahead:

Predict the error covariance ahead:UpdateUpdate the state estimate:

Update the error covariance:

where Kalman gain Kt is:

55

Kalman Filter ? 55Chart40.10.050.250.30.150.030.12

colorProbability

Sheet110.120.0530.25.0.3.0.15.0.03m0.12

Sheet1

colorProbability

Sheet2

Sheet3

Chart70.060.10.20.250.170.050.17

colorProbability

Sheet110.0620.130.2.0.25.0.17.0.05m0.17

Sheet1

colorProbability

Sheet2

Sheet3

Chart50.10.050.250.30.150.030.12

colorProbability

Sheet110.120.0530.25.0.3.0.15.0.03m0.12

Sheet10000000

colorProbability

Sheet2

Sheet3