A Proposed Method of Camera Modeling and Filter

Selection for Spectral Imaging

Collin Day

Part of ongoing spectral imaging research performed at the MCSL and for fulfillment of requirements for the Master of Science in Imaging Science degree, advised by Dr. Roy Berns, and supported in part by the National Gallery of Art, Washington D.C. and the Museum of Modern Art, NY, NY.

For background or other information with regards to spectral imaging, please see www.art-si.org.

Why spectral imaging?Overcomes limitations to Tri-chromatic Imaging

MetamerismRobust to changes in observer and illuminant

Wide range of applicationsArtworkRemote SensingMedicine

Our main interest lies in Artwork

Painting AnalysisPigment IdentificationClassification

ArchivesAbility to accurately archive imagesAbility to accurately reproduce images

Current GoalsDevelop methods that use tri-chromatic devices for spectral image acquisition and spectral estimation

Camera ModelingFilter Selection

Add the results of this research to a table of choices being created to make suggestions on the best methods of spectral estimation depending on application

The Overall Process

1. Model the camera outputIllumination

Source

P

CharacterizationReflectance

Target

R

CameraSensitivity

S

Build Model

Model

The Overall Process

2. Simulate camera data and build transforms

ModelP,R,S,Filter Set

Fn

SimulatedData

n Filters

PCA / DirectPseudo -Inverse

TransformRelating CameraOutput to Target

Reflectance

T

Decide onnumber ofchannels

The Overall Process

3. Create estimates, compute metrics, evaluate

TEstimate

Reflectance ofCharacterizationTarget - Compute

Quality Metrics

D,SimulatedCameraOutput

Select Filters

Test onSimulatedVerification

Target

Test with theCamera

Write Thesis,Graduate, get job,

leave RIT

Imaging Environment

Gallery / MuseumGenerally use diffuse, tungsten lighting

Pixel Physics TerraPix System

Kodak KAF-16801 SensorBayer Pattern Color Filter Array4096 X 4096 pixelsContax 645 camera body w/ Zeiss T* lenses

Elinchrom Scanlite Digital 1000Uses Chimera Pro Video Light Diffusers

400 450 500 550 600 650 7000

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8Spectral Power Distribution − Elinchrome Scanlite Digital 1000

Wavelength (nm)

Rel

ativ

e P

ower

Reflectance TargetsColorchecker DC – Used to create the camera model for simulation

Reflectance TargetsGamblin – Used for verification – made of a selection of artist pigments

Reflectance TargetsEsser/Blues – Used to create transforms – has a number of reflectance spectra for good statistical basis

Add in blue pigments…

The Basic Camera Model

pixel darkRGB, gray

=400 - 750 gray dark

(D D )kt P R S d *D

(D D )λ λ λλ

λ−

=−∑

Pλ = Source Illumination Power DistributionRλ = Object reflectance spectraSλ = Camera spectral sensitivityk = exposure constant, t = timeD = digital camera signal of the pixel, gray card, or dark exposure

Gamblin Target RGB Model Results

Camera Signals to SpectraMany methods are available

PCADirect Pseudo-InverseWiener filteringNNLS…. And many more

Currently use PCA and Direct Pseudo-InversePrevious research has shown they provide good results

PCA Flowchart

CharacterizationReflectances

RtEigenvectors

E

Scalars

a

Characterization

Digital Counts

Dt

A

T

VerificationDigital Counts

Dv

ReflectanceEstimate

Re

PCA

PINV

Matrix Multiply

Matrix Multiply

PCA: Statistical method to analyze data with orthogonal vectors which account for the most variance of the data set.

Direct Pseudo-Inverse Calculation

CharacterizationTarget

Reflectance

Rt

TSpectral

ReflectanceEstimate

Re

Digital Counts ofCharacterization

Target

Dt

Digital Countsof Verification

Target

Dv

PINV Matrix Multiply

Filter Selection TaskGoal is to decorrelate the individual channel signals as much as possible by filtering. Usually, changes in channel peaks are searched for.

Filter Selection Task

Filter Selection Task

Filter Selection – Filter SetsKodak Wratten

105 uncombined + No filter = 106 filtersSchott Glass Filters

40 uncombined + No filter = 41 filtersCombining in sets of two for a total of 821 filters

Brute Force method – evaluate all possible combinations

Metrics for Estimation EvaluationMany possible metrics

CIEDE2000RMS Spectral ErrorWeighted RMSMetameric Index

Choice depends on applicationMost concerned with curve shapeUse RMS spectral error first, then other metrics to refine evaluation

Camera Model AdditionsCurrent model assumes noise has a mean of zeroAttempt to add noise sources

Shot noiseIrregularities in illuminationLens falloffTarget non-uniformities

Summary of work…Created a Basic Camera ModelFilter SelectionWorking on an improved Camera ModelResearch is an extension of work which has been done at the MCSL. For more info, visit the Art Spectral Imaging website at www.art-si.org