Stereo Matching - Yonsei

E-mail: hogijung@hanyang.ac.krhttp://web.yonsei.ac.kr/hgjung

Stereo MatchingStereo Matching

Stereo Vision [1]Stereo Vision [1]

Reduction of Searching by Epipolar Constraint [1]Reduction of Searching by Epipolar Constraint [1]

Photometric Constraint [1]Photometric Constraint [1]

Same world point has same intensity in both images.Same world point has same intensity in both images.– True for Lambertian surfaces

• A Lambertian surface has a brightness that is independent of viewing angle

– Violations:• Noise• Specularity• Non-Lambertian materials• Pixels that contain multiple surfaces

Photometric Constraint [1]Photometric Constraint [1]

For each epipolar line

For each pixel in the left image

• compare with every pixel on same epipolar line in right image

• pick pixel with minimum match costThis leaves too much ambiguity, so:

Improvement: match windowswindows

Correspondence Using Correlation [1]Correspondence Using Correlation [1]

Sum of Squared Difference (SSD) [1]Sum of Squared Difference (SSD) [1]

Image Normalization [1]Image Normalization [1]

Images as Vectors [1]Images as Vectors [1]

Image Metrics [1]Image Metrics [1]

Stereo Result [1]Stereo Result [1]

Left Disparity Map

Images courtesy of Point Grey Research

Window Size [1]Window Size [1]

Better results with adaptive window• T. Kanade and M. Okutomi, A Stereo Matching Algorithm with an Adaptive Window:

Theory and Experiment,, Proc. International Conference on Robotics and Automation, 1991.

• D. Scharstein and R. Szeliski. Stereo matching with nonlinear diffusion. International Journal of Computer Vision, 28(2):155-174, July 1998.

Ordering Constraint [3]Ordering Constraint [3]

Smooth Surface Problem [3]Smooth Surface Problem [3]

Occlusion [1]Occlusion [1]

Left Occlusion Right Occlusion

Search over Correspondence [1]Search over Correspondence [1]

Stereo Matching with Dynamic Programming [1]Stereo Matching with Dynamic Programming [1]

Dynamic Programming [1]Dynamic Programming [1]

Pseudo-code describing how to calculate the optimal match

Pseudo-code describing how to reconstruct the optimal path

Local errors may be propagated along a scan-line and no inter scan-line consistency is enforced.

Correspondence by Feature [3]Correspondence by Feature [3]

SegmentSegment--Based Stereo Matching [3]Based Stereo Matching [3]

AssumptionAssumption• Depth discontinuity tend to correlate well with color edges

• Disparity variation within a segment is small

• Approximating the scene with piece-wise planar surfaces

• Plane equation is fitted in each segment based on initial disparity estimation obtained SSD or Correlation

• Global matching criteria: if a depth map is good, warping the reference image to the other view according to this depth will render an image that matches the real view

• Optimization by iterative neighborhood depth hypothesizing

Hai Tao and Harpreet W. Sawhney

Network of Constraints [2]Network of Constraints [2]

Vertex Node

Edge Node

Face Node

Vertex Node

Edge Node

Face Node

Vertex Node

Edge Node

Face Node

DirectionsDirectionsDirections

Network of Constraints [2]Network of Constraints [2]

Smoothing by MRF [2]Smoothing by MRF [2]

Stereo Testing and Comparison [1]Stereo Testing and Comparison [1]

Window-based matching(best window size)

Ground truth

State of the art method Ground truthBoykov

et al., Fast Approximate Energy Minimization via Graph Cuts, International Conference on Computer Vision, September 1999.

Intermediate View Reconstruction [1]Intermediate View Reconstruction [1]

Right ImageLeft ImageDisparity

Intermediate View Reconstruction [1]Intermediate View Reconstruction [1]

Summary of Different Stereo MethodsSummary of Different Stereo Methods

ReferencesReferences

1. David Lowe, “Stereo,” UBC(Univ. of British Columbia) Lecture Material of Computer Vision (CPSC 425), Spring 2007.

2. Sebastian Thrun, Rick Szeliski, Hendrik Dahlkamp and Dan Morris, “Stereo 2,” Stanford Lecture Material of Computer Vision (CS 223B), Winter 2005.

3. Chandra Kambhamettu, “Multiple Views1” and “Multiple View2,” Univ. of Delawave Lecture Material of Computer Vision (CISC 4/689), Spring 2007.

4. J. Diebel and S. Thrun, “An Application of Markov Random Fields to Range Sensing,” Proc. Neural Information Processing Systems (NIPS), Cmbridge, MA, 2005. MIT Press.

Stereo Matching - Yonsei

Documents

Efficient binocular stereo correspondence matching with 1

Stereo Matching Using Loopy Belief Propagation

Computing the Stereo Matching Cost with a …campar.in.tum.de/.../Markus_Herb_Paper_Stereo_Matching_CNN.pdfComputing the Stereo Matching Cost with a Convolutional Neural Network Seminar

Multiresolution stereo image matching using complex wavelets

Stereo Matching Low-Textured Survey

Using Geometric Constraints for Matching Disparate Stereo

SCV-STEREO: LEARNING STEREO MATCHING FROM A …

Segment-based Stereo Matching Using Graph Cuts

1 DENSE STEREO MATCHING USING MACHINE LEARNINGcs229.stanford.edu/...DenseStereoMatchingUsingMachineLearning.pdf · DENSE STEREO MATCHING USING MACHINE LEARNING ... sum of absolute

Single View Stereo Matching · 2018-03-12 · passive stereo vision including stereo matching[17,25], structure from motion [35], photometric stereo [5] and depth cue fusion [31],

GRAPH· BASED MATCHING OF STEREO IMAGE FEATURES

Lecture 8 Feature Matching and Stereo Vision

Assignment 4: Stereo Matching

Stereo Matching by Joint Energy Minimization

Learning and Feature Selection in Stereo Matching

Pyramid Stereo Matching Network · pyramid stereo matching network for depth estimation. 3. Pyramid Stereo Matching Network We present PSMNet, which consists of an SPP [9,32] module

Stereo Matching Information Permeability For Stereo Matching – Cevahir Cigla and A.Aydın Alatan – Signal Processing: Image Communication, 2013 Radiometric

Gradient-learned Models for Stereo Matching CS231A Project … · block-matching step of a stereo algorithm. That is, instead 1. Matching Region Aggregation Neighborhood Propagation

Robust Stereo Matching with Surface Normal Prediction

Bilateral Grid Learning for Stereo Matching Networks