Processing of Polychromatic Interferometric Data for ...interferometer.osupytheas.fr/colloques/OHP2013/presentations/M_T… · – Analysis of (polychromatic) interferometric data

1 Improving optical interferometers — OHP— Sept 23-27, 2013 Michel Tallon & POLCA consortium

Processing of Polychromatic Interferometric Data for Astrophysics : the POLCA project

Michel Tallon & POLCA consortium

CRAL, Lyon, France IPAG, Grenoble, France Lagrange, Nice, France LESIA, Paris, France


Outline

•  Short presentation of POLCA project

•  Polychromatic data –  Analysis of (polychromatic) interferometric data –  Visibilities for image reconstruction

à Ferréol Soulez’s talk

•  3-D (x,y,λ) model fitting

•  3-D (x,y,λ) image reconstruction –  specific approach à Jacques Kluska’s talk –  general approach à Ferréol Soulez’s talk


Summary of POLCA / 1

•  People (26) from 4 French labs –  CRAL (Lyon): Paul Berlioz-Arthaud, Ferréol Soulez, Michel Tallon,

Isabelle Tallon-Bosc, Éric Thiébaut –  IPAG (Grenoble): Gilles Duvert, Jacques Kluska, Sylvain Lafrasse, Bernard Lazareff,

Jean-Baptiste Lebouquin, Fabien Malbet, Guillaume Mella –  Lagrange (Nice): Philippe Bério, Olivier Chesneau, André Ferrari, David Mary,

Florentin Millour, Denis Mourard, Romain Petrov, Antony Schutz, Céline Theys, Martin Vannier

–  LESIA (Paris): Pierre Kervella, Sylvestre Lacour, Thibaut Paumard, Guy Perrin •  Ethnography :

–  Experienced observers in interferometry –  Experts in signal processing –  Experts on instruments (and their data)

•  AMBER, MIDI, VEGA, PIONIER, GRAVITY, MATISSE

•  => Nobody understands everything •  Supported by ANR (French National Research Agency)

–  4 years (2011 — 2015) –  400 k€ –  ~ 6 FTE during the 4 years


Summary of POLCA / 2

•  Facts –  Data now contain a lot of information along the wavelength (dispersed fringes +

spectrum) –  Needs of better tools to make the most of these data

•  Particularly, how to exploit differential visibilities ? –  Need to make experts in interferometry and in signal processing work together –  Need to work on real data

•  Outputs –  New methods and new algorithms for polychromatic data

•  3-D image reconstruction •  model fitting •  image reconstruction + model-fitting

–  Better knowledge of data •  requirements for better data •  => better interferometers and instruments

–  Advance in astrophysics (as much as possible) •  => work with real data


Selection of suitable data

•  Selection of a limited set of data with chromatic features –  in the objects –  in the data –  Simulated data in some cases (GRAVITY)

Field of unresolved stars of various spectra (black bodies)

0. 2. 4. 6.10+7

0.0

0.5

1.0V2 data

B/λ

Vis2


Statistics of polychromatic data

Current processings (image reconstruction, model fitting, …) assume uncorrelated gaussian data…

Minimization of: !2(x) =

Ndata!

i=1

"

di ! mi(x)

"i

#2


Appearance of independence of data

•  simulated data •  model is outside the error bars

(1 sigma) for ~32% of the data

0 10 20 0

5

10

Beware : only one realization here !

0 10 20

!2

0

2

Normalized residuals

1 sigma

•  ~32% of data out of the range [-σ, σ]

ri =

di ! mi

!i


Data with local correlations: 70%

•  correlation coefficient: –  70% with adjacent –  25% with next point –  0% farther away

•  Similar effect as spectral correlations in real data

•  alignments of successive points

•  less dispersion of residuals

0 10 20 0

5

10

0 10 20

!2

0

2

we

igh

ted

re

sid

ua

ls (

da

ta!

mo

de

l)/s

igm

a

Independent 70 % correlation

Beware : only one realization !

0 10 20 0

5

10

0 10 20

!2

0

2

we

igh

ted

re

sid

ua

ls (

da

ta!

mo

de

l)/s

igm

a


Data with global correlations: 70%

•  70% correlation between any points

•  Similar effect as noise on normalization (incoherent flux, calibrator)

•  less dispersion of residuals

•  bias may appear

0 10 20 0

5

10

0 10 20

!2

0

2

we

igh

ted

re

sid

ua

ls (

da

ta!

mo

de

l)/s

igm

a

Independent 70 % correlation

Beware : only one realization !

0 10 20 0

5

10

0 10 20

!2

0

2

we

igh

ted

re

sid

ua

ls (

da

ta!

mo

de

l)/s

igm

a


Examples on real data / 1

GI2T, Vakili et al. 1997

MIDI, Chesneau et al. 2007


Examples on real data / 2

VEGA/CHARA, Bério et al 2011

4.10 4.15 4.20 4.25 4.3010+7

−2

0

2

spatial frequency in 1/rad (max of the 3 frequencies)

wei

ghte

d re

sidu

als

of p

hase

clo

sure

(T3p

hi)

(dat

a−m

odel

)/sig

ma

4.10 4.15 4.20 4.25 4.3010+7

−2

0

2

spatial frequency in 1/rad (max of the 3 frequencies)

phas

e cl

osur

e (T

3phi

) in

rad

AMBER data, T3phi on Sirius Duvert 2013

•  => Need for a study of statistics of real data.

•  accurate error bars is necessary for reliable interferometry


Statistical analysis of interferometric data"

•  On-going work done at Lagrange (Nice) –  Antony Schutz, Martin Vannier, André Ferrari, David Mary, Florentin Millour,

Romain Petrov •  Rationale

–  Analyze assumptions on visibility statistics (distribution, correlation) –  Need to adapt data processing ? –  Need to improve data reduction ? –  Other

•  effective connection between experts from interferometry and signal processing •  understand data reduction necessities and practices

•  Data –  Currently squared (differential) visibilities from AMBER –  Work in progress for data from PIONIER and VEGA


Statistical analysis / dataset

•  AMBER data •  calibrators •  with and without fringe tracker •  UT / AT •  various time exposure •  # frames / # spectral channels

Dataset name UT-MR-NoFT AT-MR-FT AT-LR-FT

Date 2011/01/16 2012/02/22 2012/10/09Nb of files 2 40 37Time Span 0.2 h 5.7 h 3.5 hNb frames per file 970 37 970Nb ! per frame 400 400 13

integration time (s) 0.19 2.0 0.026Spectral Res. 1500 1500 35Spectral Band K K KFringe Tracker OFF ON ONSeeing (arcsec) 0.9, 1.0, 1.0 0.6, 0.9, 1.4 0.5,0.6,0.8,(min., avg., max.)

Unit Telescopes + Medium Resolution + No Fringe Tracker

Auxiliary Telescopes + Medium Resolution + Fringe Tracker

Auxiliary Telescopes + Low Resolution + Fringe Tracker


Statistical analysis / squared visibility

•  Sudden visibility losses •  with (gray) / without (black) piston

correction •  visibility loss and piston introduce

correlations between wavelengths

f01 f02 f03 f04 f05 f06 f070.60.81

V2 B1

!s"

f01 f02 f03 f04 f05 f06 f07

0.20.40.6

V2 B2

f01 f02 f03 f04 f05 f06 f07

0.20.40.6

frame index

V2 B3

!s"

All wavelengths #V

2 pisto

n

V 2100% V 260% V 220%

#V

2 nopist

UT-M

R-N

oFT

#V

2 pisto

n#V

2 nopist

AT-M

R-F

T

#V

2 pisto

n

0 0.5 1

#V

2 nopist

0 0.5 1 0 0.5 1

AT-L

R-F

T

60% selected 40% rejected 100% selected 20% selected

80% rejected

ß no piston correction à

ß piston corrected à

Example of histograms for AT-MR-FT dataset •  Selection based on fringe SNR •  Multi-modal distributions can appear


Statistical analysis / differential visibility

Histogram of ΔV2 for the 3 datasets •  100% selected

•  Unimodal •  Mean close to one

f01 f02 f03 f04 f05 f06 f07 f08 f09 f10 f11 f12

0.6

0.8

1

1.2

!V

2 B1&

V2 B1

!s"!s"

f01 f02 f03 f04 f05 f06 f07 f08 f09 f10 f11 f12

0.2

0.4

0.6

0.8

1

!V

2 B2&

V2 B2

!s"!s"

f01 f02 f03 f04 f05 f06 f07 f08 f09 f10 f11 f12

0.10.20.30.40.50.6

frame index

!V

2 B3&

V2 B3

!s"!s"

AMBER definition :

, with

•  ΔV2 corrected from visibility loss

•  Rescaling: ΔV2 à ΔV2R

–  rescaled to the mean value of visibility with 20% selection

•  Variance increased

ΔV2R(λ)

V2(λ)

ΔV2R(λ)

V2(λ)

#!V

2 pisto

n

UT-MR-NoFT

0.5 1 1.5

#!V

2 nopist

AT-MR-FT

0.5 1 1.5

AT-LR-FT

0.5 1 1.5

No piston correction à

Piston corrected à

V2Ref(!i) =

1

n!!1

n!!

i=1, j!i

V2(! j)!V2(!i) =

V2(!i)

V2Ref

(!i)


Statistical analysis / comparison of standard deviations

•  Example of standard deviation per exposure file for visibility and differential visibility –  results averaged over all files and bases –  divided by the squared root of the number of

frames

2.1 2.2 2.3 2.42

3

4

5

6

7

8 x 10 3

!t(")

Wavelength (µm)

σt for V220%

σt for ΔV2R


Statistical analysis / comparison of correlations

•  Examples of matrices of the temporal correlation coefficients between spectral channels for dataset UT-MT-NoFT, averaged over observation files and over the three baselines.

Di!erence

ofwavelengthindex

Di!erence of wavelength index50 100 150 200 250 300 350 400

50

100

150

200

250

300

350

400

Di!erence of wavelength index

50 100 150 200 250 300 350 400

50

100

150

200

250

300

350

4000.4

0.2

0

0.2

0.4

0.6

0.8

1V2

20%, no piston correction ΔV2R with piston correction


Statistical analysis / probability distribution of data ?

•  Compatibility of empirical distribution of data with standard distributions ? –  χ2 goodness-to-fit test with 5% probability of false alarm

•  Tested standard distributions –  Normal (generally assumed) –  Cauchy (ratio of 2 normal random variable) –  Student (able to fit both Normal and Cauchy and between) –  Log Normal (used in previous work)

Spectral Analysis Temporal analysisAT-MR-FT/piston N t Log N C N t Log N C

V2100 59 29 53 100 34 11 74 12

V220 59 22 49 100 1 0 1 4

!V2R 59 19 53 99 22 1 29 5

AT-MR-FT/no piston N t Log N C N t Log N C

V2100 88 12 51 99 34 11 70 18

V220 83 5 52 100 11 0 12 10

!V2R 88 12 51 99 15 1 16 16

100% selected à 20% selected à

differential Vis. rescaledà

Rejection rate in % for AT-MR-FT dataset


Statistical analysis / summary

•  Student distribution is more suitable than Normal, Cauchy and Log Normal distributions –  True for visibility and differential visibility

•  Visibility versus differential visibility –  Visibility:

•  Visibility loss and piston effect depend on the selection threshold (empirical tuned) •  Can be multi-modal (depends on selection threshold) •  Correlated (depends on selection threshold)

–  Rescaled differential visibility: alternative estimator of visibility ? •  No selection (nothing to tune) •  Larger number of frames (here, x 5), since no selection •  Lower standard deviation, since larger number of frames •  Less correlated •  Best distribution fitting score (Student)

•  Next –  data from VEGA, PIONIER –  differential phase, closure phase


Example of possible effect for model fitting

•  Simulated data for uniform disks (6 different diameters) with real noise. –  real noise is closer to Student law

•  Fit of this data assuming either gaussian or Student statistics –  better accuracy with Student

statistics : relative MSE twice smaller.

0.1 0.5 1 5 10 2010 6

10 5

10 4

10 3

10 2

10 1

100

Diameter (mas)

RelativeM

SE

D iameter Est imat ion Error On Real Noise

N , a, D are unknown

( t) , a, D & ! are unknown

assuming gaussian

assuming Student


Data more suitable to image reconstruction ?

•  Current polychromatic data are not convenient –  closure phase and powerspectrum not linear (lower SNR, loss of information) –  differential phase very helpful : how to use it in a global approach ?

•  Image reconstruction = inverse problem –  Compute (synthetic) data knowing the object (direct model) and priors –  Compare with real data –  Find the model of the object that gives the best match

•  Attempt: allow other unknowns that can help in the inverse problem –  We want to keep linearity => complex visibilities –  We want to accumulate frames => fringe tracking in post-processing (multiple wavelengths) –  We don’t want to spoil the statistics => account for statistics (and keep all frames) –  Reject optimally (i.e. robustly) the information we don’t need. –  Keep remaining unknowns that cannot be rejected => ~“self-calibration”

•  à Ferréol Soulez’s talk


3-D (x,y,λ) model fitting

One example of polychromatic “difficulties”


Need of a model of the spectrograph

•  Data on P Cygni, from VEGA –  Width of differential visibility difficult to explain in Hα line –  Spectrum is affected by detector artifacts (saturation, …)

•  P Cyg spectrum is quite well modeled by radiative transfer codes ⇒ make use of the theoretical spectrum for modeling the object “as seen by VEGA”

differential visibility

spectrum


Need of a model of the spectrograph

•  Spectrograph model includes –  PSF –  wide hallo –  detector saturation, non-uniformity

•  To be included in LITpro at the end of the project

differential visibility

spectrum

654 656 658

2

4

6

8

wavelength (nm)

num

ber

of photo

ns/

fram

e

data

model

theory


Polychromatic image reconstruction

–  Specific approach à Jacques Kluska’s talk –  General approach à Ferréol Soulez’s talk –  Other work done in Nice using sparse representations


Polychromatic image reconstruction : specific approach

•  The image is the addition of components with weights that depend on the wavelength:

–  ik(x) obtained with usual image reconstruction or model fitting

•  à Jacques Kluska’s talk

i(x, !) =

n!

k=1

wk(!) ik(x)

5 mas

B9Ve HD98922

F* = 37% T = 1650K

5.75 AU

0.0 0.5 1.010+8

0.0

0.5

1.0

B/λ

V2


Polychromatic image reconstruction : general approach

•  Make use of the strong relationship between the wavelengths –  Reconstruction of a 3D cube (x,y,λ) –  The object shape depends continuously on the wavelength (with exceptions)

•  Needs –  A 3D cube ! => need to be fast enough (i.e. converge easily) –  Account for the spectrum complexity (lines, dynamics, …)

•  à Ferréol Soulez’s talk


Conclusions

•  POLCA = Processing of Polychromatic Interferometric Data for Astrophysics –  Focus on real data (AMBER, MIDI, VEGA, PIONIER)

•  simulated data if not possible (GRAVITY) –  Improve model of data statistics –  Formalize measurements by current interferometers –  3-D (x,y,λ) image reconstruction –  3-D (x,y,λ) model fitting (and coupling with image reconstruction)

•  Evolutions of the project –  initially starting from reduced data, but now look for possible more appropriate

estimate using wavelength dependency (phase difference, …)

•  Improving the performances of interferometers ? –  Do not forget to improve signal processing

Documents

Processing of Polychromatic Interferometric Data for ...interferometer.osupytheas.fr/colloques/OHP2013/presentations/M_T… · – Analysis of (polychromatic) interferometric data