SSN Description and Operation - NetX

SSN Description and Operation
Table of Contents
Application Note 1
1.0 Document 3
2.0 Requirements 5
4 FFC Operation Mode 16
5 NVFFC Procedure 17
6 Known bugs 19
Figure 1. “Help→About” window showing factory Quark versions. .......................................................... 5
Figure 2: FLIR GUI Setup Tab with SSN Enable/Presets ............................................................................ 8
Figure 3:Typical raw image of a LWIR camera without FFC correction applied nor SSN enabled. ......... 10
Figure 4: Same scene as Figure 3 but with the camera moving for several seconds (No FFC). ................ 11
Figure 5: Same image as Figure 3 with both FFC and SSN applied. .......................................................... 12
Figure 6: An image with a LWIR camera subjected to a thermal gradient. ................................................ 13
Figure 7: Scene with FFC and SSN applied, but a high motion preset is used for static scene. ................. 14
Figure 8: Scene with FFC and SSN applied and no motion preset. ............................................................ 15
Figure 9. FLIR Camera Controller GUI – Gain Calibration Tab ............................................................... 18
1.0 Document
100 1/23/2013 Initial Release for Main app 16.0.2.128, Firmware 14.02.00.118
110 5/22/2014 Update for Tau2.7 and Quark 2.0
1.2 Scope
The Quark and Tau cameras have traditionally required an FFC (Flat Field Correction) acquired against
and internal shutter or external flat field source to reduce fixed pattern image offset. The SSN (Silent
Shutterless Non-uniformity Correction) is an image processing algorithm that uses scene content to
augment the FFC. With SSN the FFC can be acquired much less frequently and in some circumstances
not at all. Camera settings and algorithm limitations are discussed and examples are presented in this
document.
1.3 SSN Firmware and Software Versions
The Tau 2.7 and Quark 2.0 releases and later have SSN included in the standard firmware/software
release. Quark 2.0 cameras will report Main app 16.20.15.12 and Tau 2.7 cameras will report Main app
15.20.15.9. Fast cameras will report Firmware X.12.14.12 while slow cameras will report Firmware
X.13.14.12 and 60Hz 640s will report Firmware X.22.14.12. Operation of fast and slow SSN versions are
identical with the exception that the slow version will take longer to acquire the first SSN correction at
power up.
Important note: Fast and slow camera have different builds and are not interchangeable. Do not attempt
to upgrade without confirmation of the correct version. The .pcf file upload process does not check
versions and loading the incorrect version will result in a non-operational camera which must be returned
to the factory.
102-PS241-100-15 Rev110 May 2014 Page 4 of 20
Here is a sample of some of the resources that can be found on the FLIR website:
Document Title Document
Number
Description
Quark Quick Start Guide 102-PS241-01 Quark Quick Start Guide for first-time use
FLIR Camera Controller GUI
102-PS242-02 Detailed Descriptions for functions and adjustments for
FLIR cameras using the FLIR Camera Controller GUI
Quark Product Specification 102-PS241-40 Product specification and feature description
Quark Electrical IDD 102-PS241-41 Written for Electrical Engineers to have all necessary
information to interface to a Tau 2 camera
Tau 2/Quark Software IDD 102-PS242-43 Written for Software Engineers to have all necessary
information for serial control of Tau 2 and Quark
Assorted Mechanical
Various There are drawings and 3D models for various camera
configurations for mechanical integration
Application Notes Various Written for Systems Engineers and general users of
advanced features such as Gain Calibration,
Supplemental FFC Calibration, NVFFC Calibration,
Camera Link, On-Screen Symbology, AGC/DDE
explanation, Camera Mounting, Spectral Response,
Optical Interface for lens design, and others.
There is also a large amount of information in the Frequently Asked Questions (FAQ) section on the
FLIR website: http://www.flir.com/cvs/cores/knowledgebase/. Additionally, a FLIR Applications
Engineer can be contacted at 888.747.FLIR (888.747.3547).
Quark 2.0
Tau 2.7
GUI version .111 or greater
The SSN is not available for generic part numbers received prior to Tau 2.7 and Quark 2.0. The camera
software revision can be verified in the FLIR Camera Controller GUI by clicking Help→About. The
software will be described as “Main app” as shown in Figure 1.
Figure 1. “Help→About” window showing factory Quark versions.
The FLIR Camera Controller GUI can be obtained at
http://www.flir.com/cvs/cores/resources/software/tau/. Refer to the FLIR Camera Controller GUI User’s
Guide for information on installing the FLIR Camera Controller GUI or connecting to the camera.
102-PS241-100-15 Rev110 May 2014 Page 6 of 20
3.0 SSN Description The Silent Shutterless NUC (SSN) algorithm is used to augment the FFC correction to produce a lower
fixed pattern spatial noise image. LWIR imaging cameras have a small amount of fixed pattern offset in
the image as a result of sensor manufacturing and image non-uniformities are common with all thermal
cameras. The traditional method to remove image non-uniformities is addition of an offset correction
map to the current image. FLIR cameras have a built in method to acquire and save offset correction
maps, but generation of the map requires the camera be viewing a uniform scene during correction
acquisition. Due to temperature drift the offset correction needs to be updated periodically, usually every
few minutes. This is typical of any LWIR camera without temperature stabilization. The SSN algorithm
uses normal imaging scene content to reduce fixed pattern non-uniformities so that an FFC can be valid
for long periods of time.
Quark 2.0 and Tau 2.7 shutterless cameras with SSN may work for many applications, but if the
application is not consistent with the uses described in this document (see 3.2 Operational profile) a
mechanical shutter will be necessary. Tau 2.7s that contain an internal mechanical shutter will meet a
wider range of applications.
The SSN algorithm creates a per-pixel Correction map to remove Spatial Pattern Noise (SPN) from image
data. This algorithm is similar to an older version SBNUC-SSN, but instead of correcting per frame, it
detects and corrects individual pixel motion and has enhancements to simplify correction memory
clearing in comparison to its predecessor. This algorithm depends on detection of motion so as to not
remove static scene information. Static scene information could be “burned” into the correction map if
motion is incorrectly stated when there is none. A motion limit exists in order to create a threshold for the
change in motion required to have this algorithm apply its corrections.
There are specific requirements for scene content and camera use case to maximize the effectiveness of
the SSN algorithm. This is a list of requirements with explanations.
1. SSN is very good at removing high spatial frequency artifacts, but it does not remove low spatial
frequency items like shading. Sensor thermal non-uniformity can cause image shading
characterized by bright to dark changes across the image in any pattern like top to bottom, side to
side or center to corner (Figure 4 or Figure 6). Although non-uniformities can be minimized with
proper head sink design and thermal management, ultimately image shading problems must be
corrected with an FFC. Saved corrections can be used with SSN to reduce the need for constant
FFC update.
2. SSN requires a moving image. Motion is defined as irradiance on pixels changing by more than a
set threshold. A static scene or a scene with small amounts of movement will result in no updates
to the SSN correction. A scene with a high amount of moving, and static portions will result in
the SSN algorithm correcting the portions that have movement. A static scene overlooking a
highway will have corrections where the cars are moving, but no corrections on the sides of the
road where there are no cars.
3. The SSN correction map applied to the image is updated constantly from the moving scene
information. The SSN control is linked to the FFC logic so the two corrections work together.
102-PS241-100-15 Rev110 May 2014 Page 7 of 20
An FFC event normally happens at startup, after a camera reset, or based on the FFC
time/temperature profile.
4. The SSN correction map is reset (start a new correction map) when the correction is
enable/disabled, an FFC event occurs. During normal camera operation the best way to start
acquisition of a new SSN map is to enable/disable the correction as described in section 3.2.
5. SSN has threshold limits to prevent overcorrection. If the raw image before SSN has large
amplitude high spatial frequency artifacts they will not be removed from the image (Figure 4). A
reasonably good pre-SSN image can be achieved by using the FFC. The FFC can be either a
newly acquired FFC or a pre saved non-volatile FFC (NVFFC).
3.1 SSN Presets
Most Tau camera configurations include a shutter, however Quark cameras are all shutterless cameras and
users may want to provide their own shutters that are integrated into their systems. In systems that don’t
have a way to periodically perform a FFC, a non-volatile FFC may be stored.
Figure 2: FLIR GUI Setup Tab with SSN Enable/Presets
The SSN algorithm has four presets for each use case of shuttered and non-shuttered cameras. The
presets address many use cases from no scene motion through high scene motion. The algorithm takes
motion into consideration and allows a correction map to be updated
dynamically based on scene content. The first three presets ramp up the
allowed maximum correction of the algorithm while increasing the
threshold for apparent motion. It is important to match the preset with
the actual use case the camera will be experiencing. When motion is
not properly considered then possible image artifacts such as image
burn-in can occur. The fourth preset is a ‘no motion’ preset which is a
conservative setting and provides noise reduction without the fear of
possible burn-in.
3.2 SSN Operational Profile for Shutterless cameras
This section provides examples of operational scenarios for shutterless cameras where SSN can be used
correctly and examples of problems when SSN is used incorrectly. Shutterless cameras require special
considerations when using SSN and the algorithm can be used with any shuttered version of the Tau 2.7
family in similar scenarios.
Suggested operational profile 1:A Pan/Tilt or gimbal mount with a uniform blank scene available at
specific azimuth/elevation position. Set the Camera in External FFC mode. The camera could be
positioned or rotated so the field of view is covered by the blank scene at startup. At power on the
camera will perform an FFC while looking at the blank scene. This startup sequence will provide a good
FFC and a good starting point for the SSN. If the Pan/Tilt is continuously moving the SSN algorithm will
maintain the correction. Expected results: A good image as shown in Figure 5 will be produced.
Depending on how the camera is thermally connected to the mount, thermal non-uniformities might cause
a shaded image over time as shown in Figure 6. To correct a thermal gradient the camera should be
pointed to the blank scene and a new FFC acquired. The risk of SSN artifacts are reduced if the pan tilt is
using both axis of motion, i.e. both pan and tilt. A profile with only tilt motion (vertical scene motion)
may cause unsatisfactory results.
Suggested operational profile 2:A moving vehicle which is used in moderately constant environmental
conditions. An FFC is saved to the camera as an NVFFC as described in section 4. Expected results:
The camera should start and produce a good image as shown in Figure 5. Since the environmental
conditions change gradually, thermal gradients are not expected as shown in Figure 6 so high contrast
scenes may dominate and overwhelm the appearance of artifacts due tothermal gradients. Over a long
period of time an updated NVFFC may be required. If the vehicle path has a constant horizon in the
scene, image artifacts may appear and a fresh SSN map may be necessary.
Suggested operational profile 3: A hand held or helmet imaging unit. An NVFFC would be saved and
in most cases the camera would start and produce a good image. If the image quality degrades the user
would acquire and save a new NVFFC. If any image artifacts result from insufficient movement the user
would reset/restart the SSN algorithm by performing a new FFC or use the no motion default.
Non optimal profiles (not recommended):
1. Any fixed mount application without using the no motion preset.
2. Any moving application where periodic FFC updates are not possible.
3. Any environment with large thermal gradient changes over time.
4. Any slow or near static tracking application without optimized settings.
102-PS241-100-15 Rev110 May 2014 Page 10 of 20
Figure 3:Typical raw image of a LWIR camera without FFC correction applied nor SSN enabled.
Fixed pattern column offset is visible in the image and some columns have large offset. Low
frequency shading (dark area) present as a dark area across the right side of the image. Small
image non-uniformities are also possible. This type of image is very common for LWIR thermal
cameras without a correction.
102-PS241-100-15 Rev110 May 2014 Page 11 of 20
Figure 4: Same scene as Figure 3 but with the camera moving for several seconds (No FFC).
The SSN algorithm has removed most of the fixed column artifacts. The remaining columns are
outside the threshold limits of the SSN correction. The dark area and bright corner are still
present.
102-PS241-100-15 Rev110 May 2014 Page 12 of 20
Figure 5: Same image as Figure 3 with both FFC and SSN applied.
High frequency fixed pattern is removed by SSN and the correction will continuously update. The
large amplitude, high spatial frequency artifacts and low frequency shading have been removed by
FFC. This image can be produced with either current External FFC or a saved NVFFC.
102-PS241-100-15 Rev110 May 2014 Page 13 of 20
Figure 6: An image with a LWIR camera subjected to a thermal gradient.
Even though there is an FFC the thermal non-uniformity is causing a new dark to bright shading
from bottom to top. Image non-uniformities can appear in cases where the FFC is stale and there
are thermal gradients. These image non-uniformities are independent of SSN, and thermal
gradient effects should be avoided.
Image Non-uniformities
102-PS241-100-15 Rev110 May 2014 Page 14 of 20
Figure 7: Scene with FFC and SSN applied, but a high motion preset is used for static scene.
In this case the camera is held by hand. This is with a moderate motion preset and has identifiable
‘burn-in’. The SSN algorithm started to remove the fixed object as if it were image defects. This is
an example of a fixed mounted application where SSN is not appropriate without optimized settings
for the use case.
102-PS241-100-15 Rev110 May 2014 Page 15 of 20
Figure 8: Scene with FFC and SSN applied and no motion preset.
In this case the camera is held by hand again and no apparent ‘burn-in’.
102-PS241-100-15 Rev110 May 2014 Page 16 of 20
4 FFC Operation Mode The camera FFC operation changes depending on whether an NVFFC is saved in the camera. The
recommended FFC mode for SSN operation is External mode with a NVFFC stored to the camera. The
NVFFC is applied when the camera is first powered on. The SSN feature will start correcting high
frequency fixed patter noise with the NVFFC corrected image as a starting point.
No NVFFC: When there is no correction map stored in the NVFFC memory location, the camera will
exhibit the following behavior for the different FFC Modes:
Automatic: Not Recommended with Quark. FFC is performed automatically at start-up and
periodically thereafter as triggered by elapsed time or temperature change. (Both parameters may
be modified by the user.) FFC is also performed upon command. For cameras with an internal
shutter (Tau), the shutter is automatically moved in and out of the FOV when FFC takes place.
Automatic mode is not recommended for shutterless configurations because there is no assurance
that the camera will be imaging a uniform source when it initiates an automatic FFC.
Manual: FFC is performed once automatically at start-up and only then upon command
thereafter. For those configurations that include an internal shutter (Tau), the shutter is
automatically moved in and out of the FOV when FFC is commanded. This mode is
recommended when it is desirable that an FFC event not take place at any arbitrary time (for
example, when tracking a target). Repeated FFC events are necessary to correct for temperature
drift in the camera. The SSN update rate is not controlled by the FFC time/temperature interval
controls in this mode.
External: FFC is performed only upon command. No correction is loaded on startup. The
shutter control signals are not exercised even if the camera includes an internal shutter. That is,
offset terms are generated based upon whatever the camera is imaging at the time FFC is
commanded. (It is recommended to subtend the entire FOV with a uniform scene prior to
commanding external FFC.) For configurations that contain no internal shutter, external FFC
mode is the preferred mode. A SSN update event is triggered upon a user command to do an
External FFC. If the scene movement is slow, the motion limit should be set to a higher value to
avoid image artifacts. Other operational profiles might require a more rapid update rate.
102-PS241-100-15 Rev110 May 2014 Page 17 of 20
With NVFFC: When there is a NVFFC stored in the camera, the behavior will be as follows:
Automatic: Not Recommended with Quark FFC is not automatically performed at start-up.
Instead the stored NVFFC map is applied for the first 5 seconds after power-up, after which time
an automatic FFC event takes place. Normal FFC operation will continue after this base on the
time/temperature FFC interval settings.
Manual mode: FFC is not automatically performed at start-up. Instead the stored NVFFC map is
applied and subsequent FFC events are only applied upon command.
External: The stored NVFFC map is applied at start-up. Subsequent FFC events are only applied
upon command and will use whatever the camera is imaging at the time the command is issued.
The SSN update event is triggered upon a user command to do an External FFC.
5 NVFFC Procedure It is possible to save the currently-applied FFC map to non-volatile memory (NVFFC). Whenever
possible, this should be performed when the camera is integrated into the final operating system. This
replicates the typical thermal conditions that will be experienced during operation. For example, a large
lens or chassis mounting plate might have a significant impact on sensor temperature uniformity and
consequently the gain.
1. Install the FLIR Camera Controller GUI (Use version 111 or later).
2. Open the FLIR Camera Controller and connect to the camera.
3. The Advanced Tab contains many features that can be
considered to be for advanced users. For this reason, a
special keystroke and password is required to display the
Advanced Tab. The keystroke is “Ctrl+Shift+a” and the
password is “www.flir.com”. After the keystroke is
performed, the following dialog will appear. Note that the
text in the following image is added and a password typed
will display as asterisks.
Note: This is only intended for advanced users and it is possible to render the camera inoperable if
the operation is performed incorrectly.
4. Navigate to the Gain Calibration Tab:
Figure 9. FLIR Camera Controller GUI – Gain Calibration Tab
Save FFC Offset: Store the currently-applied map to non-
volatile memory.
memory.
1
2
102-PS241-100-15 Rev110 May 2014 Page 19 of 20
6 Known bugs This section lists known bug or deficiencies with SSN version Main app X.20.15.9, Firmware X.12.14.12.
1. Note of operation for Quark downgrades. The new SW now uses two FLASH blocks instead of
one and as a result it requires a special PCF to downgrade back to Quark 1 release FW/SW.
2. None related to SSN algorithm.
102-PS241-100-15 Rev110 May 2014 Page 20 of 20
© FLIR Commercial Systems, 2014. All rights reserved worldwide. No parts of this manual, in whole or in
part, may be copied, photocopied, translated, or transmitted to any electronic medium or machine readable
form without the prior written permission of FLIR Commercial Systems Names and marks appearing on the p r o d u c t s herein are either registered trademarks or trademarks of
FLIR Commercial Systems and/or its subsidiaries. All other trademarks, trade names, or company names
referenced herein are used for identification only and a re the property of their respective owners. Liberation fonts are copyright 2009 by RedHat and are used under authority of the GNU public license.
Information about these fonts and t h e G N U public license can b e found at:
https://www.redhat.com/promo/fonts/. This product is protected by patents, design patents, patents pending, or design patents pending. If you have q u es t io n s that are not covered in this manual , or need service, contact FLIR Commercial Systems
Customer Support at 805.964.9797 for additional information prior t o returning a camera.
This documentation is subject to change without notice.
This equipment must be disposed of as electronic waste.
Contact your nearest FLIR Commercial Systems, Inc. representative for instructions on how to
return the p r o d u c t to FLIR for proper disposal.
FCC Notice. This device is a subassembly designed for incorporation into other products in order to provide an infrared camera
function. It is not an end-product fit for consumer use. When incorporated into a host device, the end-product will generate, use,
and radiate radio frequency energy that may cause radio interference. As such, the end-product incorporating this subassembly
must be tested and approved under the rules of the Federal Communications Commission (FCC) before the end-product may be
offered for sale or lease, advertised, imported, sold, or leased in the United States. The FCC regulations are designed to provide
reasonable protection against interference to radio communications. See 47 C.F.R. §§ 2.803 and 15.1 et seq.
Industry Canada Notice. This device is a subassembly designed for incorporation into other products in order to provide an
infrared camera function. It is not an end-product fit for consumer use. When incorporated into a host device, the end-product
will generate, use, and radiate radio frequency energy that may cause radio interference. As such, the end-product incorporating
this subassembly must be tested for compliance with the Interference-Causing Equipment Standard, Digital Apparatus, ICES-
003, of Industry Canada before the product incorporating this device may be: manufactured or offered for sale or lease, imported,
distributed, sold, or leased in Canada.
Avis d’Industrie Canada. Cet appareil est un sous-ensemble conçu pour être intégré à un autre produit afin de fournir une
fonction de caméra infrarouge. Ce n’est pas un produit final destiné aux consommateurs. Une fois intégré à un dispositif hôte, le
produit final va générer, utiliser et émettre de l’énergie radiofréquence qui pourrait provoquer de l’interférence radio. En tant que
tel, le produit final intégrant ce sous-ensemble doit être testé pour en vérifier la conformité avec la Norme sur le matériel
brouilleur pour les appareils numériques (NMB-003) d’Industrie Canada avant que le produit intégrant ce dispositif puisse être
fabriqué, mis en vente ou en location, importé, distribué, vendu ou loué au Canada.
EU Notice. This device is a subassembly or component intended only for product evaluation, development or incorporation into
other products in order to provide an infrared camera function. It is not a finished end-product fit for general consumer use.
Persons handling this device must have appropriate electronics training and observe good engineering practice standards. As
such, this product does not fall within the scope of the European Union (EU) directives regarding electromagnetic compatibility
(EMC). Any end-product intended for general consumer use that incorporates this device must be tested in accordance and
comply with all applicable EU EMC and other relevant directives.

Documents

SSN Description and Operation - NetX