A new fusion algorithm for shadow penetration using visible ...A new fusion algorithm for shadow penetration using visible and midwave infrared polarimetric images Daniel A. Lavigne

A new fusion algorithm for shadow penetration using visible and midwave infrared polarimetric images

Daniel A. LavigneDefence Research and Development Canada – Valcartier

July 27 2010

Mélanie BretonAerex Avionics Inc.

Defence Research andDevelopment Canada

Recherche et développementpour la défense Canada Canada

July 27, 2010

Topics

1. Introduction2 A new passive polarimetric imaging sensor suite2. A new passive polarimetric imaging sensor suite3. Target detection and shadow penetration using polarized

light4. Experimental measurements5. Conclusions & Future work

Defence R&D Canada • R & D pour la défense Canada

1. Introduction

• Electro-Optical (EO) imaging systems are frequently employed during surveillance operations and search & rescue missions to detect various targets in both the civilian and military communitiesin both the civilian and military communities.

• The exploitation of the polarization of light can provide additional information about the shape, roughness, and surface characteristics of some targets of i ( d bj )interest (man-made objects).

• It has been demonstrated* that the overall performance of target detection algorithms can be increased by exploiting the polarimetric signatures ofalgorithms can be increased by exploiting the polarimetric signatures of targets and discriminate them from cluttered backgrounds.

*D.A. Lavigne, M. Breton, G. Fournier, M. Pichette, V. Rivet, Development of performancei h i h d f l i i f d bj i imetrics to characterize the degree of polarization of man-made objects using passive

polarimetric images, SPIE Defense and Security Symposium 2009, Signal Processing,Sensor Fusion, and Target Recognition XVIII, Orlando (Florida), 13-17 April 2009.


1. Introduction

• A new passive polarimetric imaging sensor-suite was designed to collect polarimetric signatures simultaneously using four broadband cameras in the VIS/SWIR/MWIR/LWIR spectral bands: the Visible Infrared Passive SpectralVIS/SWIR/MWIR/LWIR spectral bands: the Visible Infrared Passive Spectral Polarimetric Imager for Contrast Enhancement (VIP SPICE).

• Hypothesis: It is possible to perform shadow penetration of targets’ main yp p p p gbody parts using the polarimetric signatures of some targets of interest in both the visible and the infrared spectral bands.

Ultimate goal: Detection of targets in shadow areas that are currently

• Methodology: - Acquire polarimetric signatures of man-made targets using the VIP SPICE sensor suite

Ultimate goal: Detection of targets in shadow areas that are currently undetectable by traditional EO imaging systems.

the VIP SPICE sensor suite.

- Assess the utility of combining information derived from Stokes vector parameters to perform shadow penetration.


2. A new passive polarimetric imaging sensor

• VIP SPICE: a broadband fully automated remotely controlled four sensor-suite*.

MWIR(3 5 )

Front panel Polarizers panel7-filters wheel

Visible(0.4 – 0.9 µm)

(3 – 5 µm)

LWIR(8 – 12 µm) SWIR

(1.0 – 1.9 µm)

Range Finder

GPS i

Visible30°


GPS receiver Pan &Tilt

*D.A. Lavigne, M. Breton, G. Fournier, M. Pichette, V. Rivet, A new passive polarimetric imaging system collecting polarization signatures in the visible and infrared bands, SPIE Defense and Security Symposium 2009, Infrared Imaging Systems; Design, Analysis, Modeling, and Testing XX, Orlando (Florida), 13-17 April 2009.

2. A new passive polarimetric imaging sensor

• 4 Sensors Embedded:• Visible (0.4 – 0.9 µm)

- CCD camera imager (Lumenera) - 1392 x 1040 pixels resolution. FOV = 5o- Linear polarizer (Edmund Optics)- A seven positions filter wheel

• LWIR (8 – 12 µm)- Uncooled microbolometer (Nytech) - 640 x 480 pixels resolution. FOV = 10o

Wire grid ZnSe Reflex polarizer (Analytical Corp )

• MWIR (3 – 5 µm)- A focal plane array (platinum silicide) from Mitsubishi

801 512 i l l ti FOV 12o

- Wire-grid ZnSe Reflex polarizer (Analytical Corp.)

• SWIR (1 – 1.9 µm)

- 801 x 512 pixels resolution. FOV = 12o- Wire-grid ZnSe polarizer (Medway Optics)


- A solid-state InGaAs imager (Sensors Unlimited)- 320 x 256 pixels resolution. FOV = 20o- Aluminum microwires on a glass substrate as a polarizer (Versalight)

3. Target detection and shadow penetrationThe Stokes vector

• The representation of the radiation backscattered by the targets is computed using the Stokes vector:

=

−+

=

= 10

22

22

SS

AAAA

QF yx

yxII Intensity of radiation

AA Amplitudes of the EW in mutually

32

sin2cos2

SS

AAAA

VU

F

yx

yx

γγ yx AA ,

Amplitudes of the EW in mutually perpendicular directions

γ Phase angle between and xA yA

• S0 is obtained by passing light waves through linear polarizers oriented at 0 degree.

• S1 is obtained by passing light waves through linear polarizers oriented at 90 degree. 1 y p g g g p g

• S2 indicates the excess of radiation in the +45o direction over that in the +135o direction relative to the plane of vision.

• S is the circularly polarized component


• S3 is the circularly polarized component.

3. Target detection and shadow penetrationPolarized lightg

• Stokes vector parameters computation:Visible band

I0 I90 I45 I135

S1 = I0 – I90 S2 = I45 – I135S0 = I0 + I90


3. Target detection and shadow penetrationPolarized lightg

• Computation of Performance Metrics:

• Degree of Linear Polarization (DoLP):

22

21

SSS

DoLP+

=

• Degree of Linear Polarization (DoLP):

0S

• Polarization angle (φ):

DoLP mapped in pseudo-color

= −

1

21tan21

SSϕ

1


3. Target detection and shadow penetration

• Main Hypothesis:• It is possible to perform shadow penetration of targets’ main body parts using the polarimetric signatures of some targets of interest in both the visible and the infrared spectral bands.

• The Approach:• Targets are detected from their background using the polarimetric and spectral surface characteristics of targets within the scene.g

• A false color rendering of the scene is generated using the information provided by:

- the total intensity (S0),- the degree of linear polarization (DoLP),- the polarization angle (φ) of the scene.



• The Process:

Image preprocessing Includes normalization and denoising using state-of-the-art linear filtersStep 1

Using multiple automated image co-registration algorithms*Step 2 Image co-registration

Calculate Common

Component (CC)Step 3 Local minimum operators are determined in each polarized image

{ }{ }00 ,,minmin SDoLPDoLPSCC φφ =∩∩=


*D.A. Lavigne, A fully automated image co-registration system, SPIE Defense and Security Symposium 2006, Signal Processing, Sensor Fusion, and Target Recognition XV, Vol. 6235, Orlando (Florida), 17-21 April 2006.


• The Process:

Step 4Polarized

Contributions computation

Unique contribution of each polarized image is calculated using modified Common Component information

( )βDoLPSSS ∩−= 00*0( )βDoLPSDoLPDoLP ∩−= 0*

( )βββ DoLPS ∩−= 0*

where is a measure of polarization* defined by:βwhere is a measure of polarization defined by: β

( ) ( )( )εεεε

εεεεεε ααφβ13545900

1

13545

1

9002IIIIIIII

++++−++−= with and 25.0=ε 16=α


*Duggin et al., Contrast enhancement using multiband polarization methods, SPIE Vol.3121, Polarization Analysis, Measurement and Remote Sensing IV, San Diego (CA), 30 July – 01 August 1997, pp.288-295.

( )13545900


• The Process:

Step 5Image

adjustments Adjusments of the polarized images: remove the computed polarized contributions for each original polarized image

**0

**0 β−−= DoLPSS 00 βDoLPSS

**0

** β−−= SDoLPDoLP*0

*** SDoLP −−= ββ

Step 6HSV – RGB conversion

Images are combined following the conversion HSV to RGB color maps

{ } → ******0 ,, βDoLPSRGBHSV


4. Experimental measurements

• Polarimetric signatures of man-made objects were collected using the VIP SPICE sensor suite during experiments conducted in Summer time in Western CanadaCanada.

• Numerous man-made objects have been used as targets of interest: metallic aluminum plates, SUVs, military vehicles, camouflage nets, etc.

Metallic plates Vehicles, tracks, camouflage nets


4. Experimental measurements

• Man-made objects placed within the scene generated shadow areas covering a portion of the objects acquired by the VIP SPICE sensor-suite on the ground and aboard a Genie boom.aboard a Genie boom.


4. Experimental measurementsFirst experiment

• Acquisition of an aluminum metallic plate in the Visible band

VisibleVIP SPICE sensorVIP SPICE sensor• 25 degrees off-nadir angle• 15 meters above ground



• Results for the aluminum plate (Visible band)S0 Computed Polarimetric Images mapped in pseudo-color

11

2

3

4 24

• Metallic plate is segmented from the background1. Shadowed background2. Vegetal background3. Shadowed region of the plate


g p4. Unshadowed region of the plate

4. Experimental measurementsSecond experiment

• Acquisition of a SUV in the MWIR spectral band (Ground-based)

MWIR


4. Experimental measurementsSecond experiment

• Results for the SUV (MWIR band)S0 Computed Polarimetric Images mapped in pseudo-color

• Tires are segmented in shadow areasP i h l i d bl b l h SUV


• Persistent thermal signature detectable below the SUV


• Post-processing: edge detection algorithm (e.g. Canny)

Visible



• Post-processing: edge detection algorithm (e.g. Canny)MWIR


5. Conclusions and Future work

• A new passive polarimetric imaging system was used to acquire polarimetric signatures of man-made objects in the VIS/SWIR/MWIR/LWIR spectral bands.

• Using the polarized light information gathered from the objects characteristics, the total intensity, the degree of linear polarization, and the phase of polarization have been computed by means of the Stokes vector parameters.

• Experimental measurements demonstrated the possibility to exploit the polarimetric response of man-made objects for shadow penetration purposes in the visible (VIS) and midwave (MWIR) infrared bands.( ) ( )

• Future work will involve the development of additional performance metrics to extract information embedded within the polarized signatures of targets of i Sh d i f ill b i h linterest. Shadow penetration performance will be assess using other spectral bands (SWIR/LWIR) and supplementary metrics in urban environments.


Thank you for your attention

Daniel A Lavigne

[email protected]

Daniel A. LavigneDefence Research and Development Canada


Documents

A new fusion algorithm for shadow penetration using visible ...A new fusion algorithm for shadow penetration using visible and midwave infrared polarimetric images Daniel A. Lavigne