Upload
jacob-hahn
View
212
Download
0
Tags:
Embed Size (px)
Citation preview
V. Martin et al. 1 (18) WFDPVA, ENEA Frascati 28/03/12
3D Scene Calibration for Infrared Image Analysis3D Scene Calibration for Infrared Image Analysis
V. MartinV. Martin, V. Gervaise, V. Moncada, M.H. Aumeunier, M. Firdaouss, J.M. Travere (CEA), V. Gervaise, V. Moncada, M.H. Aumeunier, M. Firdaouss, J.M. Travere (CEA)S. Devaux (IPP), G. Arnoux (CCFE) and JET-EFDA contributorsS. Devaux (IPP), G. Arnoux (CCFE) and JET-EFDA contributors
Workshop on Fusion Data Processing Validation and Analysis, ENEA Frascati, 26-28 March 2012Workshop on Fusion Data Processing Validation and Analysis, ENEA Frascati, 26-28 March 2012
i
a ca
r mf
c ar ed h
i
a ca
r mf
c ar ed h
V. Martin et al. 2 (18) WFDPVA, ENEA Frascati 28/03/12
•Issue: a complex thermal scene
1. Wide angle views with high geometrical effects: depth of field and curvature
2. Many metallic materials (Be, W) with different and changing optical (reflectance) and thermal (emissivity) properties
•Objective: Match each pixel with the 3D
scene model of in-vessel components for:
1. getting the real geometry of the viewed objects
2. reliable linking between viewed objects and their related properties
•Applications
1. Image processing (event characterization)
2. IR data calibration: Tsurf = f(material emissivity)
3D IR Scene Calibration
JET #81313 KL7(images in DL)
Bulk W
Bulk Be
Be coated linconel
W coated
CFC
Bulk Be
Bulk Be
W coated CFC
V. Martin et al. 3 (18) WFDPVA, ENEA Frascati 28/03/12
IR Data Calibration Methodology
Image Stabilization
2D/3D Scene Model
Mapping
Image Processing
Image Correction
Camera
NUCDead pixel Map
Reference image 2D/3D scene models
Knowledge base of the thermal
scene
•Calibration chain
Registered & Calibrated
Image
V. Martin et al. 4 (18) WFDPVA, ENEA Frascati 28/03/12
Illustration of Motion in Images
• Camera vibrations lead to misalignments of ROIs (PFC RT protection) = false alarms or worth missed alarms
• Image stabilization is a mandatory step for analysis based on ΔT(t) estimation (e.g. heat flux computation)
V. Martin et al. 5 (18) WFDPVA, ENEA Frascati 28/03/12
•Important factors for method selection• Deformation type: planar (homothety), non-planar
• Target application: real-time processing, off-line analysis
• Data quality and variability: noise level, pixel intensity changes, image entropy
• Aimed precision level: pixel, sub-pixel
•Applications in tokamaks (non-exhaustive list)
Image Stabilization
Motion cause Deformation type
Target application Aimed precision level
Difficulty
Tore Supra
RF antenna IR
1. antenna positions
2. camera vibrations
1. non-planar
2. planar (Δx, Δy)
RT PFC protection pixel level changes of scene appearance
JET KL7
wide-angle IR
camera vibrations planar (Δx, Δy) offline analysis (heat load, disruptions…)
sub-pixel low image entropy
JET KL9
divertor tiles IR
sensor affected by magnetic field
planar (Δx, Δy) offline analysis (heat load, disruptions…)
sub-pixel low resolution, slow motion, aliasing
JET KL8
fast visible
intensifier affected by magnetic field
planar (Δx, Δy, Ө) physics (transient events)
pixel level noisy, low image entropy, aliasing
V. Martin et al. 6 (18) WFDPVA, ENEA Frascati 28/03/12
•Classical Methodology
1. Feature Detection• Local descriptors: Harris corners, MSER, codebooks, Gabor wavelets (see Craciunescu
talk), SIFT, SURF, FAST…
• Global descriptors: Tsallis entropy (see Murari talk), edge detectors…
• Fourier analysis: spectral magnitude & phase, pixel gradients, log-polar mapping…
2. Feature Matching• Spatial cross-correlation techniques: normalized cross-correlation, Hausdorff distance…
• Fourier domain: normalized cross-spectrum and its extensions
3. Transform Model Estimation
• Shape preserving mapping (rotation, translation and scaling only)
• Elastic mapping: warping techniques…
4. Image transformation
• 2D Interpolation: nearest neighboor, bilinear, bicubic…
Image Stabilization
See Zitova’s survey, Image and Vision Computing, vol. 21(2003), pp. 977-1000
V. Martin et al. 7 (18) WFDPVA, ENEA Frascati 28/03/12
Proposed Algorithm
1. Masked FFT-based image registration [1] Deterministic computing time
Accelerating hardware compatible algorithm (e.g. FFT on GPU) → real time
applications
Local analysis with dynamic intensity-based pixel masking (e.g. mask the
divertor bright region)
2. with sub-pixel precision [2] Slow drift compensation
3. and dynamic update of the reference image Robust to image intensity changes (context awareness)
Evaluation of registration quality over time
[1] D. Padfield, IEEE CVPR’10, pp. 2918-2925, 2010[2] M. Guizar-Sicairos et al., Opt. Lett., vol. 33, no. 2, pp. 156-158, 2008
V. Martin et al. 8 (18) WFDPVA, ENEA Frascati 28/03/12
Principle of Fourier-based Correlation
•Let Iref a reference image, It an image at time t and DFT the Discrete 2D Fourier
transform such as It ( x , y ) = Iref ( x-x0 , y-y0 )
•NCC is the Normalized Cross Correlation figure
(image) and the position of the peak gives the coordinates
of the translation ( x0 , y0 )
)(
(.,.)(.,.)
(.,.)(.,.)(.,.)
)(
)(
1-
21
21
2
1
NCC
NCC
t
ref
FDFTNCC
FF
FFF
IDFTF
IDFTF
Iref It
NCCyxyx,
00 maxarg, max (NCC(Iref, It))
NCC(Iref, It)
V. Martin et al. 9 (18) WFDPVA, ENEA Frascati 28/03/12
Sub-pixel Precision
)(
aliasing)-(anti 0)sfrequenciehigh (
)sfrequencie low(
)(
otherwise0
integeran is if),(),(
1-
.
UPNCC
UP
UPNCC
UPNCC
UPNCC
UPUPNCC
kkv
ku
UP
FDFTNCC
F
FF
NCCDFTF
NCCvuNCC
•Up-sample k times the DFT of NCC (trigonometric interpolation):
•The peak coordinates ( x0 , y0 ) give F the translation with 1/k pixel of precision:
UP
yxNCC
kyx
,00 maxarg
1,
V. Martin et al. 10 (18) WFDPVA, ENEA Frascati 28/03/12
Reference Image Updating
•Use the NCC value to trigger the update of Iref :tref IIthen
TtNCCTif
maxmin )(
High NCC confidence, no Iref
update needed
Tmin
Tmax
NCC failed, no Iref update
update Iref
V. Martin et al. 11 (18) WFDPVA, ENEA Frascati 28/03/12
Results
• JET #81313, KL7, 480x512 pixels, 50 Hz, 251 frames
k=1/4 pixel
V. Martin et al. 12 (18) WFDPVA, ENEA Frascati 28/03/12
Results
k=1/2 pixel
• JET #80827, KL7, 128x256 pixels, 540 Hz, 13425 frames
V. Martin et al. 13 (18) WFDPVA, ENEA Frascati 28/03/12
Results
k=1/16 pixel
• JET #82278, KL9B, 32x96 pixels, 6 kHz, 4828 frames
V. Martin et al. 14 (18) WFDPVA, ENEA Frascati 28/03/12
Computational Performance
• Influence of image size and precision level on frame rate
•GPU NVidiaTM GTX 580: 512 processor core units 32 threads per processor Up to 1 Tflops!256x256, k=1/4
→ 700 fps
GP
U/C
PU
per
form
ance
gai
n
Number of CPU cores
GPU is at least 15 times faster than
CPU (same generation)
• FFT 256x256 pixels: CPU v/s GPU performance
V. Martin et al. 15 (18) WFDPVA, ENEA Frascati 28/03/12
From 2D to 3D
•Challenge
– transform pixel coordinates into machine coordinates: (x, y) (r, θ, φ)
•Method– Ray-tracing method from simplified CAD files
V. Martin et al. 16 (18) WFDPVA, ENEA Frascati 28/03/12
3D Scene Model for Image Processing
S. Palazzo, A. Murari et al., RSI 81, 083505, 2010
V. Martin et al.
Blobs 1 & 2 must not be
merged!
1
2
1
2
mm
Z Map (depth)
2
12m
7m
V. Martin et al. 17 (18) WFDPVA, ENEA Frascati 28/03/12
Plasma ImagiNg data Understanding Platform (PINUP)
IR Data Calibration Methodology
•An integrated software for IR data registration & calibration
Image Stabilization
2D/3D Scene Model
Mapping
Image Processing
Image Correction
Camera
NUCDead pixel Map
Reference image 2D/3D scene models
Knowledge base of the thermal
scene
Registered & Calibrated
Image
Used for PFC protection
Used for temperature evaluation
Used for event triggering
Set sub-pixel precision factor
Set mask
Load/save translations
V. Martin et al. 18 (18) WFDPVA, ENEA Frascati 28/03/12
Conclusion
• Summary– Complex IR scenes require a new calibration approach including image
stabilization, 3D mapping for reliable data analysis.
– A robust and fast image stabilization algorithm with sub-pixel precision has been proposed.
– A first integration of 3D model for IR data analysis has been performed.
– An integrated software (PINUP) is available for users
• Outlook– Test of the stabilization algorithm on visible imaging data (JET KL8) with
rotation compensation
– Full integration of 3D scene models into PINUP
– Improvement of image processing algorithms (e.g. hot spot detection) with 3D information