Kinect Like 3D Camera Slides

7/27/2019 Kinect Like 3D Camera Slides

1/37

STEVEN ARROYO

RAJOSHI BISWASPETER HOKANSON

HANA WANG


2/37

Project Goals

Create a structured light programthat generates a 3D point cloud(x,y,z coordinates)

Create embedded system capableof running our structured lightprogram

Display point cloud results usingopen source 3D modeling software


3/37

What is structured lighting?

An encoded set of pixels projectedonto a scene

The deformation created by a

surface can be used to calculatedepth and surface information of theobject

3D-Point Cloud


4/37

What does SL Solve?

Eliminates need for correspondencesearch with scan lines used in Stereo

Texture less objects

Occlusions/patterns (repetitions)

Higher resolution than Kinect


5/37

Portable Structured Light

3D Scanner


6/37

TI DLP Pico Projector

Resolution : HVGA [480 x 320]

Brightness: 7 lumens

Dimensions: 44 x 67 x 14 mm

Low Power consumption Sync Signal Output for camera

Allows up to 2400Hz SL patterns


7/37

VGA Camera Board

# of Pixels: VGA (640 x 480 pixles

Hardware compatible with Beagleboard

Drivers included in Angstrom distribution

2x telephoto lens used to help match projectorfocal length


8/37

BeagleBoard-xM


9/37

N

arcissus

Too

l 3

1

2

http://narcissus.angstrom-distribution.org/


10/37

OpenCV Setup

Pkg-config program providescompiler and linker flags

pkg-config --cflags opencv

pkg-config -- libs opencv

Build with native toolchain


11/37

Demo!


12/37

When would you not use SL?

In bright/noisy environments

With Non-Lambertian/reflective/dark objects


13/37

When would you use SL?

3D object scanning for use in:

Reverse engineering

Precision shape measurement for

production control Classification of grinding materials

Volume measurement

Forensic science inspections Skin surface measurement

Body shape measurement


14/37

Temporally encoded Structured Light

Temporallyencoded pattern

High Resolution

Static Scenes

Gray Code

Low decodingerror

Reversible

L. Zhang, B. Curless, and S. M. Seitz. Rapid Shape Acquisition Using Color Structured Light and Multi-pass DynamicProgramming.3DPVT2002
http://grail.cs.washington.edu/projects/moscan/http://grail.cs.washington.edu/projects/moscan/http://grail.cs.washington.edu/projects/moscan/http://grail.cs.washington.edu/projects/moscan/http://grail.cs.washington.edu/projects/moscan/http://grail.cs.washington.edu/projects/moscan/


15/37

Simplest Case: Parallel images Image planes of cameras

are parallel to each other

and to the baseline

Camera centers are at

same height

Focal lengths are the same

Then, epipolar lines fall

along the horizontal scan

lines of the images

Ashok VeeraraghavanLect 16


16/37

Depth from disparity

f

x x

Baseline

B

z

O O

X

f

Depth is inversely proportional to disparity!

Ashok VeeraraghavanLect 16


17/37

Decoding the Gray Code

Find difference between positive and

negative projected patterns

Apply a threshold to the difference to

remove unlit pixels

Positive values are ones, negative

values are zeros


18/37


19/37

Example: A Tale of Two Pixels

Column 392

Column 335


20/37

01



21/37


0

1

We also project the patterns inverse!


22/37


01

11


23/37


010

110


24/37


0100

1101


25/37


01000

11010


26/37


010000

110100


27/37


0100001

1101001


28/37


01000011

11010011


29/37


010000111

110100111


30/37

Yellow Pixel Decoding

Gray Code: 110100111

Binary: 100111010

Projector column: 314

Camera column: 335 Disparity: 335314 = 21

Unscaled results:

x = Camera column = 335

y = Camera row z = (Disparity)-1 = 0.0476

Red Pixel Decoding

Gray Code: 010000111

Binary: 011111010

Projector column: 250

Camera column: 392 Disparity = 392250 = 142

Unscaled results:

x = Camera column = 392

y = Camera row z = (Disparity)-1 = 0.007042

Decoding the Example


31/37

ProcessC Implementation for Image Reconstruction

Project 18 patterns, with odd images being

inverted patterns of the previous invertedimage, capture deformed images

Convert the images to grayscale

Convert gray code to binary and get thedecimal value = column number of theprojector

Calculate disparity, depth and plot the 3Dpoint cloud

For each pixel, compare the pixel value inthe even and odd image. The min value isset as 0 and the max value is set as 1. Ifthey are the similar, the pixel is discarded.


32/37

Results with OpenGL

Not yet compatible with Angstrom on Beagleboard


33/37

Results from Meshlab


34/37

Results from Meshlab


35/37

Issues from Last Time

The difference in the focal length and resolutionsof the camera and projector may introduceerrors in the calculation and the resulting imagewill be skewed.

The homography is currently unknown. Therefore,the epipolar lines are not horizontal.

The camera and projector are not calibrated yetand hence the scaling is not accurate without

the knowledge of focal length. Also the imagesare still distorted.

Need to eliminate redundant camera pixels incaptured images due to high camera resolution


36/37

Issues from Last Time

The difference in the focal length and resolutions of thecamera and projector may introduce errors in thecalculation and the resulting image will be skewed.

2x telephoto lens cuts the difference by half: focal length

Resolutions are normalized before calculating disparity

The homography is currently unknown. Therefore, theepipolar lines are not horizontal.

By placing the camera in parallel, we force the epipolarlines to be horizontal.

The camera and projector are not calibrated so images arenot to scale.

Project scope does not include recreating scale models


37/37

Live Results!

Documents

Kinect Like 3D Camera Slides