A Multi-Spectral Structured Light 3D Imaging SystemMATTHEW BEARDMOREMATTHEW BOWEN

Origins of our Project• Freshman Imaging Project

2011• Tasked with creating a 3D

imaging system over three quarters

• Presented at ImagineRIT 2012

• Contour distances along a person’s face gives information about the structure of that person’s trachea

• Uses technique known as Structured Light to scan subjects

A brief primer on digital imaging

39

56

45

75

62

99

64

101

228 178 106 193

183 143 84 162

Source: Digital Photography Presentation (Jeff Pelz, Joe Pow)

How are digital images captured?• Light rays pass in through

an aperture

• Those rays are focused onto the sensor by a lens

• The sensor segments the light into individual boxes, known as pixels

• Each pixel interprets the intensity of the light striking it as a numerical value

A closeup of an imaging sensorSource: Digital Photography Presentation (Jeff Pelz, Joe Pow)

Individual pixel element

What is Structured Light?• A 3D scanning technique• Involves projecting a series of

known patterns onto a subject

• A camera interprets the distortions in the patterns and calculates depth

• The series of patterns creates a temporal code for each pixel• Each projected pixel is

uniquely identified by this code

What is Structured Light? (continued)• The camera can detect where

each projected pixel falls upon the subject• Interprets each pattern as a part

of a temporal code

• Combines each part at the end of the scan to reconstruct the temporal code

• Depth can then be calculated• Triangulation between camera

and projector projector

camera

projectedlight ray

reflectedlight ray

object beingscanned

p

Diagram source: Structured Light: The Mathematics of 3D Triangulation Presentation (Gabriel Taubin, Douglas Lanman)

ImagineRIT 2012 Prototype

360° Scanning – Obstacles• Simultaneous scanning with black and white can cause

the scanners to interfere with each other

Left projector only Right projector only Both projectors simultaneously

360° Scanning – Utilizing multiple spectra

Left projector displaying greenRight projector displaying red Camera with green color filter Camera with red color filter

• Instead of black and white scanning, different colors can be assigned to each camera-projector pair and isolated using color filters

Our goals• Extend the original FIP2011 prototype to four cameras

and projectors• Allows for 360° + overhead scanning of subject (2π steradians)

• Utilize red, green, and blue portions of the visible spectrum• Allows for simultaneous projection, keeping scan times as short

as possible

• Decrease the overall scan time of our system• Assists in the scanning of subjects, as subjects will move less

First attempt• Three color cameras, each with a Bayer filter on the

sensor

• Separate red, green, and blue channels into intensity maps (grayscale)• No color filters

• Suboptimal quality – projectors not displaying precise colors• E.g. displaying green would cause projector to display a small

amount of red

Use of color filters• Instead of raw Bayer filter, color filters were placed in

front of camera and projector• Restricts projector output and camera input to only desired

wavelengths

• Significantly improved scanning results with three camera-projector pairs• Very little interference between scanners

Adding an overhead scanner• Red, green, and blue make up primary colors of light• What color should the fourth scanner use?

• Yellow was chosen due to its distance away from any primary color relative to other secondary colors

• Significant interference due to breadth of wavelengths that the red and green filters cover• Yellow wavelength overlaps too much with red and green to be

viable

Summer 2012 Prototype

Results

•PUT PHOTO HERE

Status of goals• Successfully implemented a multi-spectral scanner that

uses red, green, and blue to achieve 360° scanning

• Scan time halved from ~8 seconds to ~4 seconds

• Addition of fourth color – yellow – not successful• Colors filters not narrow enough to be useful for four scanners

• To achieve the goal of scanning a subject with the overhead view, the last camera-projector pair would not be able to scan simultaneously

What we’ve learned • Matching the output of the projector to color filters is

difficult• Narrow matches are optimal

• Grayscale cameras paired with color filters provide optimal contrast and resolution• As opposed to the color (Bayer) cameras used in our project

Acknowledgements• Chester F. Carlson Center for Imaging Science

• Joe Pow, Advisor

• Maria Helguera, Advisor

• Stefi Baum

• Class of Freshman Imaging Project 2011

• Gabriel Taubin and Douglas Lanman, Brown University