Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
3D Vision: Multiple Ways of Getting Data and Which is
Best for My Application
Jim AndersonSenior Vision Systems Specialist
SICK,Inc
Why 3D?
Data Types
• 2D intensity • 2D array of brightness/color pixels
• 2.5 D range • 2D array of range/height pixels
• Single view-point information
• Depth Map / Distance Map
• 3D surface range data• Surface coordinates [x,y,z]
• Point cloud data
• 3D "voxel" • A volume [x,y,z] of densities
• e.g., CT scan
Map of 3D
3D imaging
Passive Active
Stereo Time-of-flightStructured Light
Laser Triangulation
InterferometryLightfield
Shading CW PulsedBinary Coded Phase Coded
Focus
Triangulation
Time-of-flight
Interferometry
Base Technologies:
Map of 3D
3D imaging
Passive Active
Stereo Time-of-flightStructured Light
Laser Triangulation
InterferometryLightfield
Shading CW PulsedBinary Coded Phase Coded
Focus
Active Triangulation - Where is the light
Map of 3D
3D imaging
Passive Active
Stereo Time-of-flightStructured Light
Laser Triangulation
InterferometryLightfield
Shading CW PulsedBinary Coded Phase Coded
Focus
Time-of-flight- When is the light
Map of 3D
3D imaging
Passive Active
Stereo Time-of-flightStructured Light
Laser Triangulation
Lightfield
Shading CW PulsedBinary Coded Phase Coded
Focus
Interferometry- How is the light
Interferometry
3D Imaging Methods
• Triangulation• Stereo
• Structured light
• Sheet-of-light
• Projected patterns
• Time-of-flight
• Misc.• Shading
• Focus
• Light field
• Interferometry
Triangulation Principle
B
a
b
Pg
g = 180-a-b
L1
L2L1 = B*sinb /sing
Robustness:- Large B- Large g
Laser Line Triangulation
Camera view Sensor Image
3D profile
Laser Triangulation Products
• Product examples
• Algorithm support in vision SW packages
• SICK Ranger/Ruler/Trispector - Proprietary CMOS sensor, multi scanning/color
• Automation Technology - Fast CMOS sensors and FPGA processing
• Photonfocus - Fast CMOS sensors+ Lin-Log response
Booth #1655
Booth #2552
Laser Triangulation Conclusions
• Benefits• “Micrometer to mm” resolution scalability• Fast and robust• With Moving objects -> No additional scanning needed
• Limitations• Occlusion (shadow effects)• Laser speckles• Not suitable for large outdoor applications (~ > 1 m FOV)• Not snapshot
• Typical applications have linear object motion :• Log/board/veneer wood inspection• Electrical components / solder paste• Food and packaging
Stereo Imaging
3D imaging
Passive Active
Stereo Time-of-flightStructured Light
Laser Triangulation
InterferometryLightfield
Shading CW PulsedBinary Coded Phase Coded
Focus
Stereo Imaging
B
a
b
P(x,y)
L1
L2
Where to Match?
• Lens centers and rays create a plane – Epipolar plane• Epipolar plane intersects sensor plane on a line
• Match Along a line in a plane defined by Baseline & Ray
• This is the Epipolar line
Bb
Structure Comparison
No structure Active structure
Stereo Products
• IDS - Ensenso with “noise” illumination
• Flir (Point Grey) - 2/3 cameras
• Chromasens – line scan color
• Most vision SW packages
• And many others…
Booth #2629
Coded Structured Light
3D imaging
Passive Active
Stereo Time-of-flightStructured Light
Laser Triangulation
InterferometryLightfield
Shading CW PulsedBinary Coded Phase Coded
Focus
Coded Structured Light
• Generally called Digital Fringe projection or often “structured light”
• Light modulation :• Binary [Gray coded]
• Continuous phase shift - “sinus”
• Pseudo random pattern
Phase Coded
Shift 0
Shift 120 degrees
Shift 240 degrees
Intensity 3 unknown:
I(x,y,t) = I(x,y)+ I’(x,y)*cos(j(x,y,t))
Analytical expression in each pixel
-> range, modulation, background
0 360
120 480
240 600
More common:
4 patterns with 90 degree separation
-> Simpler math & more robust
General Triangulation
3D imaging
Passive Active
Stereo Time-of-flightStructured Light
Laser Triangulation
InterferometryLightfield
Shading CW PulsedBinary Coded Phase Coded
Focus
Time-of-flight
3D imaging
Passive Active
Stereo Time-of-flightStructured Light
Laser Triangulation
InterferometryLightfield
Shading CW PulsedBinary Coded Phase Coded
Focus
TOF-CW
• By knowing the time or phase shift of signal arrival relatively to the initial signal, the distance (depth information) between the device and target can be extracted
3D Compact Systems| Confidential 23
TOF- CW
• Basic principle: “3D Time-of-Flight” means measuring the time-of-flight of a light signal between the device and the target for each point of the image
3D Compact Systems| Confidential 24
Kinect One
• 512x424 @30 Hz
• Multi frequency CW
• Multi-exposure HDR
• SDK available• Not industrial…
See : IEEE Journal of Solid State Circuits 50 (1), 2015
TOF Pulsed
3D imaging
Passive Active
Stereo Time-of-flightStructured Light
Laser Triangulation
InterferometryLightfield
Shading CW PulsedBinary Coded Phase Coded
Focus
Pulsed TOF Shutter Principle
Relationship between Gated and Full gives range
X-Y resolution today ~Megapixel, but Z resolution not as good as for CW.
Emitted pulses
Reflectedpulses
Shutter
Full Gated : ”Large”
Gated :”Small”
Near Distance Far Distance
TOF Pulsed
Time-of-flight
• Pulsed
• Send a light pulse – measure the time until it comes back
• Light speed 0.3 Gm/s … at 1 m it comes back after ~7 ns
• Measure “indirect” delay time
• CW - Continuous Wave
• Modulated continuous illumination• Phase shift ~distance
• Used in most TOF imager arrays
• Low resolution due to complex pixels
• ~ a few mm-cm depth resolution
Technology Comparison
Ph
ase
Co
ded
3D
TOF
3D
Active
Stereo 3
DLaser Trian
gulatio
n 3
D
Map of 3D
3D imaging
Passive Active
Stereo Time-of-flightStructured Light
Laser Triangulation
InterferometryLightfield
Shading CW PulsedBinary Coded Phase Coded
Focus
Shape from Shading
• Gives shape information, but not real distance• Shade from different directions of illumination gives surface orientation information
• Integrating the orientation gives depth variations
• Limitations• Only surface orientation, no actual depth
• No discontinuities allowed
Light-Field 3D
• Micro lens array used to create "4D" light-field image on standard image sensor
• 2D direction "subpixels" in each 2D "pixel"
Light-Field 3D
• Processing of light-field image• Refocussing
• 3D calculation
• Cameras – Raytrix• AIT Multi-line linescan
• Features• "No occlusion"
• Limitations• Depth accuracy
"lens aperture triangulation"
• Special cameras
• Complex processing
Light-Field
Depth from Focus
• Grab a sequence of images focused from A to B
• Scan through the stack and find where local focus is maximized• That gives the range
• Features• No occlusion
• No structured illumination needed
• Limitations• Slow
• Needs structure to estimate focus
• Pixel regions needed to estimate focus
• Poor accuracy
• “Triangulation using lens aperture”
A
B
3D Technology Overview
Distance / FOV size
Z Resolution / Accuracy
Time Of Flight
Interfero-metry Coded
StructuredLight Laser
Triangulation
StereoTriangulation
3D Applications
• Where are these actually being used?
Train Inspection
Logistics with TOF
• Measure volume and size of box onpallet or conveyor
Logistics with TOF
Collision Avoidance- Stereo
Analog Audio Reconstruction
Contact Info
Jim AndersonSenior Vision Solution Specialist
SICK, Inc.
6900 West 110th Street
Bloomington, MN 55438
Telephone: 800-325-SICK(7425)
Email: [email protected]
www.SICK.com