Upload
alexia-allen
View
215
Download
0
Tags:
Embed Size (px)
Citation preview
Vehicle Segmentation and Tracking From a Vehicle Segmentation and Tracking From a Low-Angle Off-Axis CameraLow-Angle Off-Axis Camera
Vehicle Segmentation and Tracking From a Vehicle Segmentation and Tracking From a Low-Angle Off-Axis CameraLow-Angle Off-Axis Camera
Neeraj K. KanhereNeeraj K. Kanhere
Committee members
Dr. Stanley BirchfieldDr. Robert SchalkoffDr. Wayne Sarasua
Clemson UniversityClemson University
July 14th 2005
Why detect and track vehicles ?Why detect and track vehicles ? Intelligent Transportation Systems (ITS)Intelligent Transportation Systems (ITS) Data collection for transportation engineering applicationsData collection for transportation engineering applications Because it's a challanging problem!Because it's a challanging problem!
Vehicle TrackingVehicle Tracking
Loop detectorsLoop detectors Tracking using VisionTracking using Vision
Low per unit cost Field experience
No traffic disruption Wide area detection Rich in information
No tracking Maintenance difficult
Computationally demanding
Expensive
Available Commercial SystemsAvailable Commercial Systems
AUTOSCOPE (Image Sensing Systems)AUTOSCOPE (Image Sensing Systems)
Has been around for more than a decadeHas been around for more than a decadeDedicated HardwareDedicated HardwareReliable operationReliable operationGood accuracy with favorable camera Good accuracy with favorable camera placementplacement
VANTAGE (Iteris)VANTAGE (Iteris)
New in marketNew in marketAccuracy has been found to be lower than AutoscopeAccuracy has been found to be lower than Autoscope
Related ResearchRelated Research
Region/Contour BasedRegion/Contour BasedComputationally efficientComputationally efficientGood results when vehicles are well separatedGood results when vehicles are well separated
3D Model Based3D Model BasedLarge number of models needed for different vehicle Large number of models needed for different vehicle typestypesLimited experimental resultsLimited experimental results
Markov Random FieldMarkov Random FieldGood results on low angle sequencesGood results on low angle sequencesAccuracy drops by 50% when sequence is processed Accuracy drops by 50% when sequence is processed in true orderin true order
Feature Tracking BasedFeature Tracking BasedHandles partial occlusions Handles partial occlusions Good accuracy for free flowing as well as Good accuracy for free flowing as well as congested traffic conditionscongested traffic conditions
Factors To be ConsideredFactors To be ConsideredHigh angleHigh angle Low angleLow angle
Planar motion assumption
Well-separated vehicles
Relatively easy
More depth variation
Occlusions
A difficult problem
Overview of the ApproachOverview of the Approach
Offline CalibrationBackground model
Frame-Block #1
Frame-Block #3
Frame-Block #2 Feature Tracking
Estimation of 3-D LocationEstimation of 3-D Location
GroupingGrouping
segmented #1 segmented #2 segmented #3
Counts,Counts,Speeds and Speeds and
ClassificationClassification
Counts,Counts,Speeds and Speeds and
ClassificationClassification
Block Correspondence Block Correspondence and Post Processingand Post Processing
Processing a Frame-BlockProcessing a Frame-Block
Multiple frames are needed for motion information Tradeoff between number of features and amount of motion Typically 5-15 frames yield good results
Block # nBlock # n+1
frames
#features in
block
#features in
block#frames in
block#frames in
block
Overlap
Background ModelBackground Model
Time Domain Median filteringTime Domain Median filtering
For each pixel, values observed over timeFor each pixel, values observed over time
Median value among observationsMedian value among observations
Simple and effective for the sequences consideredSimple and effective for the sequences considered
Adaptive algorithm required for long term modelingAdaptive algorithm required for long term modeling
Frame DifferencingFrame Differencing
Partially occluded vehicles appear as single blobPartially occluded vehicles appear as single blob
Effectively segments well-separated vehiclesEffectively segments well-separated vehicles
Goal is to get filled connected componentsGoal is to get filled connected components
`
Offline CalibrationOffline Calibration
Required for estimation of world coordinatesRequired for estimation of world coordinates
Provides geometric information about the sceneProvides geometric information about the scene
Involves estimating 11 unknown parameters Involves estimating 11 unknown parameters
Needs atleast six world-image correspondancesNeeds atleast six world-image correspondances
Control pointsControl points
Calibration ProcessCalibration Process
Using scene features to estimate correspondences Using scene features to estimate correspondences
Standard lane width (e.g. 12 feet on an Interstate)Standard lane width (e.g. 12 feet on an Interstate)
Vehicle class dimensions (truck length of 70 feet)Vehicle class dimensions (truck length of 70 feet)
Relies on human judgment and prone to errorsRelies on human judgment and prone to errors
Approximate calibration is good enoughApproximate calibration is good enough
Estimation using Single FrameEstimation using Single Frame
Box-model for vehiclesBox-model for vehicles
Road projection using Road projection using
foreground maskforeground mask
Works for orthogonal surfacesWorks for orthogonal surfaces
camera vehicle
Road plane
Selecting Stable FeaturesSelecting Stable Features
Shadows, partial occlusions will result into wrong estimatesShadows, partial occlusions will result into wrong estimates
Planar motion assumption is violated more for features higher upPlanar motion assumption is violated more for features higher up
Select stable features, which are closer to roadSelect stable features, which are closer to road
Use stable features to re-estimate world coordinates of other featuresUse stable features to re-estimate world coordinates of other features
Estimation Using MotionEstimation Using Motion
➢ Estimate coordinates with respect to each stable featureEstimate coordinates with respect to each stable feature
➢ Choose coordinates which minimized weighted sum of euclidean Choose coordinates which minimized weighted sum of euclidean
distance and trajectory errordistance and trajectory error
Rigid body under translationRigid body under translation
Estimate coordinates with respect to each stable featureEstimate coordinates with respect to each stable feature
Select the coordinates minimizing weighted sum of Euclidean Select the coordinates minimizing weighted sum of Euclidean distance and trajectory errordistance and trajectory error
• Coordinates of P are unknown• Coordinates of Q are known• R and H denote backprojections • 0 : first frame of the block• t : last frame of the block• Δ: Translation of corresponding point
Affinity MatrixAffinity Matrix
Each element represents the similarity between corresponding featuresEach element represents the similarity between corresponding features
Three quantities contribute to the affinity matrixThree quantities contribute to the affinity matrix
Euclidean distance (AEuclidean distance (ADD), Trajectory Error (A), Trajectory Error (AEE) and Background- ) and Background-
Content (AContent (ABB))
Normalized Cut is used for segmentation
Number of Cuts is not known
Incremental CutsIncremental Cuts
We apply normalized cut to initial We apply normalized cut to initial AA with increasing number of cuts with increasing number of cuts
For each successive cut, segmented groups are analyzed till valid For each successive cut, segmented groups are analyzed till valid
groups are foundgroups are found
Valid Group: meeting dimensional criteriaValid Group: meeting dimensional criteria
Elements corresponding to valid groups are removed from Elements corresponding to valid groups are removed from A A and and
process repeated starting from single cutprocess repeated starting from single cut
Avoids specifying a threshold for the number of cuts Avoids specifying a threshold for the number of cuts
Correspondence Over BlocksCorrespondence Over Blocks
Formulated as a problem of finding maximum wieght graphFormulated as a problem of finding maximum wieght graph
Nodes represent segmented groupsNodes represent segmented groups
Edge weights represent number features common over two blocksEdge weights represent number features common over two blocks
an: groups in block N
bn: groups in block N+1
ResultsResults
ResultsResults
ConclusionConclusion
A novel approach based on feature point trackingA novel approach based on feature point tracking
Key part of the technique is estimation of 3-D world Key part of the technique is estimation of 3-D world
coordinates coordinates
Results demonstrate the ability to correctly segment vehicles Results demonstrate the ability to correctly segment vehicles
under severe partial occlusionsunder severe partial occlusions
Handling shadows explicitly Handling shadows explicitly
Improving processing speedImproving processing speed
Robust block-correspondance Robust block-correspondance
Future Work
Questions ?Questions ?
Thank You ! Thank You !