3D Multi-view Reconstruction Young Min Kim Karen Zhu CS 223B March 17, 2008

3D Multi-view 3D Multi-view ReconstructionReconstruction

Young Min KimYoung Min KimKaren ZhuKaren ZhuCS 223BCS 223B

March 17, 2008March 17, 2008

OutlineOutline

ProblemProblem

Data SetData Set

MRFMRF

Noise ReductionNoise Reduction

Multi-view ReconstructionMulti-view Reconstruction

ConclusionConclusion

ProblemProblem

Create broad-view high-resolution 3D viewCreate broad-view high-resolution 3D view

3D View

Normal Camera

Depth Camera

Data SetData Set

MRFMRF

Single view super-resolution reconstructionSingle view super-resolution reconstructionObjective function: E=Ed+EcObjective function: E=Ed+Ec Ed: Similarity between the up-sampled depth and the Ed: Similarity between the up-sampled depth and the

depth sensor measurementdepth sensor measurement Ec: Regions with similar color have similar depthEc: Regions with similar color have similar depth

mrfDepthSmooth code from Stephen Gould mrfDepthSmooth code from Stephen Gould

[1] James Diebel, Sebastian Thrun, “An Application of Markov Random Fields [1] James Diebel, Sebastian Thrun, “An Application of Markov Random Fields to Range Sensing”, to Range Sensing”, Proceedings of Conference on Neural Information Proceedings of Conference on Neural Information

Processing Systems (NIPS)Processing Systems (NIPS), MIT Press, Cambridge, MA, 2005., MIT Press, Cambridge, MA, 2005.

MRF: ResultMRF: Result

Original depth mapOriginal depth map MRF resultMRF result

Noise ReductionNoise Reduction

Single-view ImprovementSingle-view Improvement Median filteringMedian filtering Occlusion boundary removalOcclusion boundary removal

Original depth image Median filtered depth image

Original MRF

Median filtered Occlusion boundary removed

Multi-view ReconstructionMulti-view Reconstruction

Problem: misalignment

Multi-view ReconstructionMulti-view Reconstruction

New Objective New Objective function using multi-function using multi-view information: view information: E=Ed+Ec+EmE=Ed+Ec+Em

Em: similarity Em: similarity between depth in two between depth in two different viewdifferent view

Multi-view ReconstructionMulti-view Reconstruction

Multi-view Reconstruction: ResultMulti-view Reconstruction: Result

ConclusionConclusion

Median filter is effective in removing Median filter is effective in removing sensor noisesensor noise

Removing occlusion boundary reduce Removing occlusion boundary reduce noise due to motionnoise due to motion

By using information from multi-view, By using information from multi-view, depth images are better aligneddepth images are better aligned