Non Conventional Imaging Systems : Application to 3D ... fileNon Conventional Imaging Systems :...

Non Conventional Imaging Systems :

Application to 3D scanning of Transparent Objects

Fabrice MERIAUDEAU

Laboratory Le2i UMR 5158

Jakarta, Indonesia

06- 01-2012

Contributors:

M. Ferraton, R. Rantoson, G. Eren, L. Sanchez-Sécades (PhD Students)

C. Stolz, O. Aubreton, F. Truchetet, D. Fofi, R. Seulin (Academic collaborators)

3D Digitization

Contact

Touching Probe

Non Contact

Reflection Transmission Emission

Outlines

Introduction

State of the Art

Scanning From Heating

Background

Method

Application to Glass

Implementation & Experimental Results

Shape from polarization

Polarization Imaging

Applications

« Shape from polarization »

Experimental System and result

Shape from UV

Background

Implemenation and results

Conclusion

Outlines

Introduction

State of the Art

Background

Method

Applications

Shape from UV

Background

Conclusion

Passive 3D Scanning Stereo Vision Shape From Focus

Active 3D Scanning

Laser Triangulation Pattern Projection Time of Flight

Manual Scanning

• Long

• Tedious

• Highly skilled operator

• Results

=> tied to operator’s expertise

Design Solution for 3D Digitization

◦ Results => Independent of User Skills

◦ Automatic & Fast Acquisition

◦ Optimized View Planning

for Maximal Surface Coverage

◦ Post-Processing of Delivered 3D Model

Object 3D Model

Manual

Teaching

Model Based

Non Model Based

Automation Robot CMM Arm => Internal 7-axis articulated arm with an external skeleton driven by electromotors Drives laser line scanner => programmed motion path

Automation => Manual Teaching

http://www.metris.com

Automation Fringe Projection Scanner Head Mounted on Industrial Robot Arm

Automation => Manual Teaching

http://www.steinbichler.de

Model Based – Offline View Planning

Sensor & Robot Modeling

1 Face = 1 ViewPoint

Visibility Study : Binary Table

Optimization : Set Covering Problem

Automation => Model Based

Non Model Based – Online View

Planning Surface

•Mass Vector Chains (MVC)

•Sum of Surface Normal’s

•Weighted by Surface‘s Area

Automation => Non Model Based

Planning Surface

Components

Fringes Projection

3D scanner head

Robot Turntable

Automation => Implementation

3D Digitization Cell

Automation => Implementation

Types of Surfaces

Diffuse (Lambertian) Glossy Specular

Translucent Transparent

Type de surface

19 Diffuse Surface Specular Surface Transparent Surface

Type de surface

20 Diffuse Surface Specular Surface Transparent Surface

Problem to be solved

« Laser scanning »

21 Surface transparente

Outlines

Introduction

State of the Art

Background

Method

Applications

Shape from UV

Background

Conclusion

➡ New methods:

➡“Scanning From Heating”

➡Shape from polarization

➡Shape from UV

‣ Capable of scanning different transparent materials and type of surfaces

Increasing demand for three-dimensional (3D) applications

◦ object modeling, preservation of historic artifacts, quality control,...

Transparent objects have not received much attention

High demand for in-line 3D quality control of transparent products

For complex geometric forms: touch probe scanners

◦ too slow for inline inspection

◦ statistical sampling

INTRODUCTION

State of the Art Methods

Structured LightS. Hata, Y. Saitoh, S. Kumamura, and

K. Kaida, “Shape extraction of

transparent object using genetic

algorithm," in Pattern Recognition,

Proceedings of the 13th International

Conference on, vol. 4, 1996.

Shape From MotionM. Ben-Ezra and S. Nayar, "What

does motion reveal about

transparency?" in Proc. IEEE Int'l

Conf. Computer Vision, 2003, pp.

1025-1032.

Optical FlowS. Agarwal, S. Mallick, D. Kriegman,

and S. Belongie, "On refractive optical

flow," in Proc. ECCV'04, 2004, pp.

483-494.

Structured LightS. Hata, Y. Saitoh, S. Kumamura, and

K. Kaida, “Shape extraction of

transparent object using genetic

algorithm," in Pattern Recognition,

Proceedings of the 13th International

Conference on, vol. 4, 1996.

Shape From MotionM. Ben-Ezra and S. Nayar, "What

does motion reveal about

transparency?" in Proc. IEEE Int'l

Conf. Computer Vision, 2003, pp.

1025-1032.

Optical FlowS. Agarwal, S. Mallick, D. Kriegman,

and S. Belongie, "On refractive optical

flow," in Proc. ECCV'04, 2004, pp.

483-494.

State of the Art Methods

FluorescenceM. B. Hullin, M. Fuchs, I. Ihrke, H.-P.

Seidel, and H. P. A. Lensch,

"Fluorescent immersion range

scanning," ACM Trans. Graph., vol.

27, no. 3, pp. 1-10, 2008.

PolarizationD. Miyazaki and K. Ikeuchi, "Inverse

polarization raytracing: estimating

surface shapes of transparent

objects," in IEEE Computer Society

Conference on Computer Vision and

Pattern Recognition, vol. 2, 2005, p.

Shape From

DistortionM. Tarini, H. Lensch, M. Goesele, and

H. Seidel, "3d acquisition of mirroring

objects using striped patterns,"

Graphical Models, vol. 67, no. 4, pp.

233-259, 2005.

FluorescenceM. B. Hullin, M. Fuchs, I. Ihrke, H.-P.

Seidel, and H. P. A. Lensch,

"Fluorescent immersion range

scanning," ACM Trans. Graph., vol.

27, no. 3, pp. 1-10, 2008.

PolarizationD. Miyazaki and K. Ikeuchi, "Inverse

polarization raytracing: estimating

surface shapes of transparent

objects," in IEEE Computer Society

Conference on Computer Vision and

Pattern Recognition, vol. 2, 2005, p.

Shape From

DistortionM. Tarini, H. Lensch, M. Goesele, and

H. Seidel, "3d acquisition of mirroring

objects using striped patterns,"

Graphical Models, vol. 67, no. 4, pp.

233-259, 2005.

Outlines

Introduction

State of the Art

Background

Method

Applications

Shape from UV

Background

Conclusion

Transparency

Human flesh is transparent to X-Ray

while bone is not

Electromagnetic Spectrum

Transparency - Example Case of

Transparent glass bottle in front of an infrared heat sourceand Image

taken with a long wave infrared camera

Evolution of the refraction and absorption index

of glass depending on the wavelength

Glass is Opaque in UV and in IR

We have developed methods making use of this property

Thermal Radiation

Energy distribution of a blackbody Thermal Image of an house

Thermal Radiation

*>47.7°C

*<-11.6°C

Application to Glass - Emissivity

Emissivity of a glass sphere

๏ G. Gaussorgues and S. Chomet, Infrared thermography. Kluwer Academic Publishers,

A new approach:

Scanning From Heating G. Eren, F. Meriaudeau and al., Optics Express 2009.

For the method to work properly:

The object surface should be opaque to the laser source i.e. the laser should be absorbed by the

surface

The object has to emit Once the surface is heated, the thermal emission should be omnidirectional

so that the thermal camera can capture accurately the heated point on different curvatures of the

surface

Homogenous, Isotropic Uniform in structure and composition

Physical properties of the material are independent of direction

Application to Glass - Heating

P0, power of the CO2 laser beamr, radius of the CO2 laser beam∂(z), impulse function

๏ Jiao, J., Wang, X.: A numerical simulation of machining glass by

CO2-laser beams. Optics and Laser Technology 40(2) (2008) 297-301

Application to Glass - Heating

Model • To bring the surface form 20 to 80 degrees with v=10mm/s, r=1.5mm

➡Laser power : 3W

Experimental results Experimental results compared to the heating model

Application to Glass - Selection of

the Camera

Transmission of light as a percentage in the infrared

domain

Transmission of light as a percentage in the

infrared domain of commonly used glasses

The Scanner

Back View

Camera Calibration

Custom calibration plate

Calibration image set

Results - Glass Plate

Results obtained on a 10x5 cm glass plate

• The reconstruction is compared to a perfect plane

• Average deviation is 150 µm

Results - Glass Cup

Transparent glass cup

3D reconstruction

by SFH Scanner 3D reconstruction (after being powdered)BY Minolta 3D Laser Scanner

Comparison of the reconstructions

(average deviation is 210 µm)

Results - Automotive Glass

Automotive Glass

Reconstruction by a Touch Probe Scanner Comparison of the reconstructions

Results - Plastic

Transparent plastic bottle 3D reconstruction by SFH Scanner

Comparison to Minolta VI-910

Line Projection : an extension

Experimental Setup

Meriaudeau and al., IEEE TIM, 2010

Object Reconstruction Object is Powdered

for Comparison

Comparison to Minolta VI-910

Specular object: an extension, A. Bajard, PhD Thesis, 2009/2012

Outlines

Introduction

State of the Art

Background

Method

Applications

Shape from UV

Background

Conclusion

Outlines

Introduction

State of the Art

Background

Method

Applications

Shape from UV

Background

Conclusion

Electromagnetic Plane waves

Non polarized wave : random phase

Polarized wave :

◦ Linear polarization

◦ Circular polarisation

rktieEE

rktavec 12: shiftphase

mwithm

Stokes Vector :

Total Intensity :

Degree of Polarization :

Angle of polarization :

0 ssss

sin2sin

cos2sin

Mueller Matrix : describes the effects of

linear optical systems (polarizers, wave

plates, reflecting surfaces…)

Example : Linear polarizer inclined with an

angle α

02sin2sin2cos2sin

02sin2cos2cos2cos

02sin2cos1

Polarization imaging

Polarization imaging in industrial vision:

◦ Avoid specular reflections

◦ Distinguish dielectrics from metallics (Wolff)

◦ …

> Shape from polarization:

Polarization Images Information on the surface normals

Physical principle: after reflection on a surface, an unpolarized light

wave becomes partially linearly polarized.

Studying the state of polarization of the reflected light enables to get

information on the surface normals (Fresnel coefficients).

Polarization imaging

Polarization state of a light wave:

◦ Goal: study the state of a partially linearly polarized light

No need to have a stokes polarimeter

Just a rotating polarizer in front of the camera

minmax

• Degree of polarization:

• Magnitude of the light: minmax III

• Angle of polarization: 3 parameters:

Partial Stokes’ polarimeters

Measure of S0, S1, S2

Set-up with a linear polarizer

Set-up with elliptical nematics liquid crystal

Set-up with LVCR

55 Wolff et al., 1997

Bigue & Cheney, 2007

Partial Stokes’ polarimeters

Polarization imaging applications

Active polarimetry

◦ Depolarization

56 Morel et al., 2006

Alouini, 2005

Study of the polarization state

of the reflected wave

Normals Determination

Normal field integration

so as to obtain the surface

Principle:

◦ Unpolarized light

◦ Reflective surface

sintan

costan

( , r ) ?

◦ Angle of polarization

◦ Degree of polarization

Fresnel reflection coefficients

Angle of polarization

)²(tan

)²(sin

The linearly polarized component is orthogonal to the plan of incidence

1st ambiguïty

Degree of polarization r

)²(tan

)²(sin

ti n sinsin

Fresnel reflection coefficients Snell-Descartes Law

Relation between the degree of

polarisation ρ and the zenithal angle θ

tansinsin

sintansin2

2nd ambiguïty

Previous approaches of the « Shape from

polarization » for transparent objects

63 Miyazaki et al., 2002

64 Miyazaki et al., 2004

Solutions

65 [Ferraton, 2009] Ferraton, M., Stolz, C. and Meriaudeau, F., "Optimization of a polarization imaging system for 3D measurements

of transparent objects", Optics Express, Optical Society of America, vol. 17 (23), pp. 21077-21082, 2009.

ρ = f(θ,n)

n = f(λ)

λ1 ρ1

λ2 ρ2

ρ = f(θ,λ)

Solutions

66 Ferraton & Meriaudeau, 2009

θB θ

ρλ2 - ρλ1 > 0 0° < θ1 < θB

Solutions

θB θ

ρλ1 ρλ2

ρλ2 - ρλ1 < 0 θB < θ2 < 90°

Solutions

θB θ

ρλ2 - ρλ1 < 0 ρλ2 - ρλ1 > 0

0° < θ1 < θB θB < θ2 < 90°

Outlines

Introduction

State of the Art

Background

Method

Applications

Shape from UV

Background

Conclusion

Experimental System

12 bit-depth Camera

Orthographic projection Rotating polarizer

Experimental system

0° 10°

Acquisition

Approximation

Normal extraction

sintan

costan

Time < 1sec

Liquid Crystal Variable Retarder (LCVR)

λ = 632.8 nm

Experimental system

Optical calibration

Rotating polarizer

Optical calibration

Angle (°) Tension (V)

0 1.565

-5 1.590

-10 1.620

-15 1.650

-20 1.675

-25 1.705

-30 1.745

-35 1.775

-40 1.805

-45 1.835

-50 1.865

-55 1.900

-60 1.935

-65 1.975

-70 2.010

-75 2.055

-80 2.095

-85 2.145

… …

Optical calibration

◦ Saliency operator

76 Walter, N., Aubreton, O. and Laligant, O., "Salient point characterization for low resolution meshes", IEEE International Conference

on Image Processing , pp. 1512-1515, 2008

Optical calibration

λB = 472 nm

nsité lu

Optical calibration

Optical Calibration Zenithal Angle Ambiguity

2222 1tansin

tansin2

2222 1tansin

tansin2

11ˆ 222 knn

Pseudo-index

3D Reconstruction

0° 10°

Acquisition

Approximation

Normal extraction

3D Reconstruction

Multispectral approach to relieve the ambiguity on the Zenithal

Active lighting approach to relieve the ambiguity on the azimutal

,03100.2 quadquadquad IIIsi

Morel et al., 2005

3D Reconstruction

Active lighting approach to relieve the ambiguity on the azimutal

,03100.2 quadquadquad IIIsi

Morel et al., 2005

3D Reconstruction

Normal Evaluation

Integration

sintan

costan

~,0,0,

qjvpjuvufvu

Frankot, R. and Chellappa, R., "A method for enforcing integrability in shape from shading algorithms", IEEE Transactions on

Pattern analysis and Machine Intelligence, vol. 10, pp. 439-451, 1988

yxgyxfyxf ,,, 0

kqypxyxg 0,0~0,0~,

Std: 0.070 mm , Mean. : -0.015 mm, Min : -0.226 mm, Max : 0.150 mm

Shape from polarisation in the IR (results from september 2011)

Expérimenta Set-upl: IR camera Flir 3µm - 5 µm, polariser ZnSe 1µm -15µm, « IR dome »

made of 4x14 resistors (12Ω et 0.25W) with a 9V power generator. One resistor 60° ~

maximal radiation around 8.7µm according to de Wien’s Law), a piece of glass of complexe

« Two ambiguities solved»

Azimutal angle Iquad in the IR

Shape from polarisation in the IR

Zénithal Angle complex refraction index for the glass

« Two ambiguities solved»

Camera Calibration Bouguet ‘s Toolbox (z0 ~ 300mm)

α=0°

α=0° α=90°

Without polariseur

Reconstruction par polarisation dans l’IR

Méthode de validation : Carte binaire

Black pixel(110,325) White Pixel (322,292)

70% of the points are useful

Implémentation et résultats dans l’IR

Polarisation angle Azimutal Angle

Degree of polarisation Zenithal Angle

Integration

Outlines

Introduction

State of the Art

Background

Method

Applications

Shape from UV

Background

Conclusion

Shape From UV

Principle:

Under the UV irradiations, transparent surface reemit fluorescence in the Visible

Optimization of the excitation wavelegnth

Absorbance

Emission Spectra

Principle : Strutured lighting system

Reconstruction by active triangulation

Calibration

Matching

Structure lighting: point or line (or pattern)

Sensors: one or two cameras

Shape From UV

Results

Internationale Conferences (2)

• 3D Reconstruction of Transparent Objects Exploiting Surface Fluorescence caused by

UV Irradiation, Rindra Rantoson, Christophe Stolz, David Fofi, Fabrice Meriaudeau, IEEE

International Conference on Image Processing (ICIP), Hong Kong, September 2010.

• Non Contact 3D Measurement Scheme for Transparent Objects using UV Structured

Light, Rindra Rantoson, David Fofi, Christophe Stolz, Fabrice Meriaudeau, IEEE International

Conference on Pattern Recognition (ICPR), Istanbul, Turkey, August 2010

Workshop (1)

• Triangulation par stéréovision basée sur l'exploitation des images de fluorescence d'une

surface transparente, Rindra Rantoson, Christophe Stolz, David Fofi, Fabrice Meriaudeau,

Journées imagerie optique non conventionnelle, Paris, France, GDR ISIS, 22 March 2010

Book (1)

• "Scanning from Heating" and "Shape from Fluorescence" two Non Conventional

Imaging Systems for 3D Digitization of transparent objects, Fabrice Mériaudeau, R.

Rantoson, G. Eren, L. Sanchez-Sécades, O.Aubreton, A. Bajard, D. Fofi, I. Mohammed, O.

Morel, C. Stolz, F. Truchetet, IGI Global, Novembre 2011

Active Triangulation by stereovision : Publications

Shape From UV

Active Triangulation by monocular vision

two experimental set-ups

Point structured lighting system Line structured lighting system

Shape From UV

Experimental set-up

Active Triangulation by monocular vision

Implementation and results : Expérimental Set-up

- UV Laser (266m, 10mW, spot elliptical shape of 2mm)

- CCD RGB (Guppy F-080C, 480x640, 1/3 inch, 8mm focal, 1.4 f-number

- Displacement table

Shape From UV

µ = 0.07mm, σ=0.07mm

µ = 0.08mm, σ=0.1mm

Shape From UV

µ = 0.08mm, σ= 0.09mm

Shape From UV

Experimental set-up

- A UV lasre (266m, 10mW, spot elliptique de 2mm)

- A semi-cylindrical « UV proof » lens

- CCD RGB (Guppy F-080C, 480x640, 1/3 inch, 8mm focal,1.4 f-number

- Displacement table

Line structured lighting system

Shape From UV

µ = 0.14mm

σ = 0.12mm

Shape From UV

Potential Extensions

Specular Surfaces

Optimizing the tracking

Light pre-heating of the surface

Modelisation of the shape of the UV spot or line

Shape From UV

Scanning from Heating

High accuracy

No a priori for the

object

Adapted for online

processes

High Cost

High accuracy

No a priori for the object

Adapted for online processes

Low cost

Pre/post-

processing needed

Multispectral « shape

from polarization »

High accuracy

Tedious Calibrations

Post-processing needed

Scanning from UV

Thank you for your attention

Terima Kasih

Non Conventional Imaging Systems :

Application to 3D scanning of Transparent Objects

Fabrice MERIAUDEAU

Laboratory Le2i UMR 5158

Jakarta, Indonesia

06- 01-2012

Contributors:

M. Ferraton, R. Rantoson, G. Eren, L. Sanchez-Sécades (PhD Students)

C. Stolz, O. Aubreton, F. Truchetet, D. Fofi, R. Seulin (Academic collaborators)

Non Conventional Imaging Systems : Application to 3D ... fileNon Conventional Imaging Systems :...

Documents

Tomographic Imaging with Cosmic Ray Muonsscienceandglobalsecurity.org/archive/sgs16morris.pdfTomographic Imaging with Cosmic Ray Muons 39 Conventional radiography takes advantage of

Imaging Techniques for the Lumbar Spine: Conventional Radiology

Prior Authorization Review Panel MCO Policy Submission A … · 2020-03-20 · conventional imaging (computed tomography, magnetic resonance imaging, or ultrasound); or . The use

Laboratoire LE2I – UMR CNRS 5158 – Université de Bourgogne

Conventional 3T brain MRI and diffusion tensor imaging in ... · DIAGNOSTIC NEURORADIOLOGY Conventional 3T brain MRI and diffusion tensor imaging in the diagnostic workup of early

Imaging Modalities: Clinical Reasoning and Key ......5 Key Principles of Diagnostic Imaging “Do no harm” Consider diagnostic yield Conventional radiographs generally first Use

Clinical Programme Diagnostic Imaging - Improve … · Radiography, CT & MRI: introduction to diagnostic imaging . modalities • Explain the principles of conventional ... adrenal

4D PET: Beyond Conventional Dynamic PET Imagingapplications.emro.who.int/imemrf/Iran_J_Nucl_Med/Iran_J_Nucl_Med... · 4D PET: Beyond Conventional Dynamic PET Imaging ... elaborated

TIK: a time domain continuous imaging testbed using conventional …aggregate.org/DIT/ei2017TIK.pdf · TIK: a time domain continuous imaging testbed using conventional still images

PROGRAMMING WITH INTEL IPP (INTEGRATED - Le2i

An Evaluation of Confocal Versus Conventional Imaging of

Conventional Nuclear and PET Imaging of Suspected MSK infections

Digital Signal Processing Methodologies for Conventional Ultrasound Imaging

Editor: DrMohammad Sadrkabir. Magnifying Narrowband Imaging Is More Accurate Than Conventional White-Light Imaging in Diagnosis of Gastric Mucosal Cancer

New dimensions in endodontic imaging: part 1. Conventional ... · Conventional and alternative radiographic systems ... dental radiography, ... part 1. Conventional and alternative

Vector Flow Imaging Compared with Conventional Doppler ... · vector flow imaging, transverse oscillation, flowrig, intraoperative cardiac ultrasound, ascending aorta Introduction

High-speed imaging of transient metabolic dynamics using two-photon …biophotonics.illinois.edu/sites/default/files/Optica-Fast... · 2018-10-22 · metabolic imaging, conventional

Chest Radiography Diagnostic value and … Radiography Diagnostic value and interpretation Imaging modalities Conventional X-ray & Tomography Computed tomography Radionuclide imaging

Conventional and diffusion-weighted magnetic resonance imaging

New dimensions in endodontic imaging: part 1. Conventional ... · REVIEW New dimensions in endodontic imaging: part 1. Conventional and alternative radiographic systems S. Patel1,2,