Michael J. Nicolls and Craig J. Heinselman CEDAR, 21 June 2008isr.sri.com/iono/amisr/meetings/2008/2008-tutorials/Data Processing… · Michael J. Nicolls and Craig J. Heinselman

PFISR Experiment, Data Reduction, and Analysis

Michael J. Nicolls and Craig J. Heinselman

CEDAR, 21 June 2008

SRIInternational

Outline1 Standard Experiments

System InfoBeam PointingF -Region ExperimentsE -Region ExperimentsD-Region Experiments

2 Level-0 ProcessingGeneralPower EstimationACF / Spectra Estimation

3 Level-1 ProcessingNe EstimationACF / Spectral FitsACF / Spectral Fits

4 Level-2 ProcessingVector Velocities / Electric FieldsE-Region WindsCollision Freqs. / Conductivities / Currents / Joule HeatingD-Region Parameters

5 The Future

Standard ExperimentsLevel-0 ProcessingLevel-1 ProcessingLevel-2 Processing

The Future


System Information

128-panel AMISR system (upgraded from 96 in Sep. 07)

Pulse-to-pulse phase capability

∼1.6 MW peak Tx (upgraded from ∼1.3 MW)

∼10% max duty cycle

4 reception channels

Tx band 449-450 MHz

3.5 MHz max Rx bandwidth

4 µs min pulsewidth (freq. allocation limitation)

Fully programmable, remotely operable/ted

Graceful degradation - reliable operations

Michael J. Nicolls and Craig J. Heinselman PFISR Experiment, Data Reduction, and Analysis


The Future


Beam Pointing

Range of pointingpositions within gratinglobe limits

”Normal” experimentsinclude ∼1-10 beams

Main limitation isintegration time /sensitivity


Beam Pointing


The Future


Standard F -region Experiment

0 t1

Time

Range

Tp

h2

h1

t1+8τt2+2τt2

T f

Farley and Hagfors [2005]

At high altitudes, use a single long pulse withmismatched filter (oversampled) to measureall lags of the ACF at once

Sacrifice range resolution

Typically use a 480 µs pulse (F region) or 1ms pulse (topside)


Standard F -region Experiment - Ambiguity Function

Ambiguity function including filter effects.

480 µs long pulse, 30 µs sampling.


The Future


Standard E/F -region Power Measurement

+

ts0

Range

Time

+

+−

−

+

+ ++ −

++

+

+

++

+

+

+

++ +

+

+

− −

−−

−−

++ ++ −


Pulse compression code allow for highsensitivity, high range resolution powermeasurements.

Plasma must remain correlated over pulselength (limits range of use for most systems).

Typical code is 13-baud Barker code, 130 µs.


E/F -region Power Measurement - Ambiguity Function


130 µs (13-baud, 10 µs baud, 5 µs sampling).


The Future


Standard E -region Experiment

Draft DF November 1, 2005

h

0

Time

Range

t0 t1 t2 t3

a0 a1 a2 a3


At lower altitudes, we require better rangeresolution.

For this, we utilize binary coded pulse ACFmeasurements (do not compress pulse oreliminate clutter like BC - eliminatecorrelation of clutter)

Random (CLP) or alternating (cyclic codes)

Standard experiment is 480 µs, 16-baud (4.5km), randomized strong code.

Include an uncoded 30 µs pulse for zero-lagnormalization.


Standard E -region Experiment - Ambiguity Function


480 µs (16-baud, 30 µs baud, 32 pulse).


The Future


Standard D-region Experiments

h

0 ts

Time

Range

τ ts+τ

Long correlation times (narrow spectralwidths) in the D region require pulse-to-pulsetechniques

We employ coded double-pulse techniquesthat give range resolutions up to 600 m andspectral resolutions up to 1 Hz.

Mode Pulse Baud δR τ IPP δf Nyquist δt

0 130 µs 10 µs 1.5 km 5 µs (0.75 km) 2 ms 2 Hz 250 Hz 1 s1 260 µs 10 µs 1.5 km 5 µs (0.75 km) 4 ms 1 Hz 125 Hz 2.5 s2 130 µs 10 µs 1.5 km 5 µs (0.75 km) 2 ms 2 Hz 250 Hz 1.8 s3 280 µs 10 µs 1.5 km 5 µs (0.75 km) 3 ms 1.3 Hz 167 Hz 2.7 s4 112 µs 4 µs 0.6 km 2 µs (0.3 km) 3 ms 1.3 Hz 167 Hz 2.7 s



The Future

GeneralPower EstimationACF / Spectra Estimation

General

A typical experiment consists of:

Data samples

Noise samples

Cal pulse samples



The Future


General


Data samples

Noise samples

Cal pulse samples

Given experiment is complicated by:

Interleaving of pulses (possiblyon different frequencies)

Clutter considerations, Noiseand Cal sample placement

Maximization of duty cycle

Beam pointing, Distribution ofpulses, Integration timeconsiderations



The Future


General


Data samples

Noise samples

Cal pulse samples

Given experiment is complicated by:

Interleaving of pulses (possiblyon different frequencies)

Clutter considerations, Noiseand Cal sample placement

Maximization of duty cycle

Beam pointing, Distribution ofpulses, Integration timeconsiderations

Raw

Samples

Decoded

Samples

Power

Lag Profile

Matrix

Decoding

Process

Noise subtraction

and Calibration

Noise subtraction

and Calibration



The Future


Power Estimation

Received power can be written as

Pr =Ptτp

r2Ksys

Ne

(1 + k2λ2D)(1 + k2λ2

D + Tr )Watts

wherePr - received power (Watts)Pt - transmit power (Watts)τp - pulse length (seconds)r - range (meters)Ne - electron density (m−3)k - Bragg scattering wavenumber (rad/m)λD - Debye length (m)Tr - electron to ion temperature ratio

Ksys - system constant (m5/s)



The Future


Power Estimation

Received signal power needs to be calibrated to absolute units of Watts. To do this, we ingeneral (a) take noise samples and (b) inject a calibration pulse at each AEU, which isthen summed in the same way as the signal. The absolute calibration power in Watts is:

Pcal = kBTcalB Watts

wherekB - Boltzmann constant (J/kg K)Tcal - temperature of calibration source (K)B - receiver bandwidth (Hz)



The Future


Power Estimation

Received signal power needs to be calibrated to absolute units of Watts. To do this, we ingeneral (a) take noise samples and (b) inject a calibration pulse at each AEU, which isthen summed in the same way as the signal. The absolute calibration power in Watts is:

Pcal = kBTcalB Watts

wherekB - Boltzmann constant (J/kg K)Tcal - temperature of calibration source (K)B - receiver bandwidth (Hz)

The measurement of the calibration power (after noise subtraction) can then be used as ayardstick to convert the received power to Watts. This is done as,

Pr = Pcal ∗ (Signal − Noise)/(Cal − Noise) Watts



The Future


ACF / Spectra Estimation - E/F region



The Future


ACF / Spectra Estimation - E/F region



The Future


ACF / Spectra Estimation - D region



The Future

Ne EstimationACF / Spectral FitsACF / Spectral Fits

Electron Density

Recall,

Pr =Ptτp

r2Ksys

Ne

(1 + k2λ2D)(1 + k2λ2

D + Tr )Watts



The Future


Electron Density

Recall,

Pr =Ptτp

r2Ksys

Ne

(1 + k2λ2D)(1 + k2λ2

D + Tr )Watts

Calibrated received power can easily be inverted to determine Ne (if onemakes assumptions about Tr ), but what about Ksys?



The Future


Electron Density

Recall,

Pr =Ptτp

r2Ksys

Ne

(1 + k2λ2D)(1 + k2λ2

D + Tr )Watts

Calibrated received power can easily be inverted to determine Ne (if onemakes assumptions about Tr ), but what about Ksys?

Within Ksys is embedded information on the gain, which for a phased-arrayvaries with the look-angle off boresight, as well as the proximity to thegrating lobe limits.



The Future


Electron Density

f2r ≈ f

2p +

3k2

4π2

kBTe

me

+ f2c sin2 α

wherefr - plasma line frequency (Hz)fp - plasma frequency (Hz)Te - electron temperature (K)me - electron mass (kg)fc - electron cyclotron frequency (Hz)α - magnetic aspect angle



The Future


Electron Density

Ksys = A cosB(θBS) m5/s

θBS - angle off boresightA,B - constants



The Future


Fitting Spectra



The Future


Fitting Spectra

General Complicating Factors:

Range smearing

Lag smearing

Pulse coding effects / ”Self”-clutter

Clutter (geophysical and not - e.g., mountains,irregularities, turbulence, non-Maxwellian)

Signal strength / statistics

Time stationarity



The Future


Fitting Spectra


Range smearing

Lag smearing




Time stationarity

Specific Based on Altitude:

F -region/Topside - Light ion composition

Bottomside - Molecular ion composition

E -region - Collision frequency, Temperature

D-region - Complete ambiguity



The Future


Fitting Spectra


Range smearing

Lag smearing




Time stationarity

Specific Based on Altitude:

F -region/Topside - Light ion composition

Bottomside - Molecular ion composition

E -region - Collision frequency, Temperature

D-region - Complete ambiguity

Approach:

F -region - Te , Ti , vlos , Ne

Bottomside - Assume a composition profile

E -region - <∼ 105km, assume Te = Ti

D-region - Fit a Lorentzian (width, Doppler, Ne )



The Future


Fitting Spectra - Example


Fitting Spectra - Example


The Future

Vector Velocities / Electric FieldsE-Region WindsCollision Freqs. / Conductivities / Currents / Joule HeatingD-Region Parameters

Vector Velocities - Preliminaries

LOS Velocity measurement can be represented as:

v ilos = k i

xvx + k iyvy + k i

zvz



The Future




v ilos = k i

xvx + k iyvy + k i

zvz

where the radar k vector in geographic coordinates is:

k =

ke

kn

kz

=

cosαcosβcos γ

=

x

y

z

R−1



The Future




v ilos = k i

xvx + k iyvy + k i

zvz

where the radar k vector in geographic coordinates is:

k =

ke

kn

kz

=

cosαcosβcos γ

=

x

y

z

R−1

If we can neglect Earth curvature (“high enough” elevation angles),

k =

ke

kn

kz

=

cos θ sinφcos θ cosφ

sin θ

where θ, φ are elevation and azimuth angles, respectively.



The Future



For a local geomagnetic coordinate system we can use the rotation matrix,

Rgeo→gmag =

cos δ − sin δ 0sin I sin δ cos δ sin I cos I

− cos I sin δ − cos I cos δ sin I

where δ (∼ 22) and I (∼ 77.5) are the declination and dip angles, respectively.



The Future



For a local geomagnetic coordinate system we can use the rotation matrix,

Rgeo→gmag =

cos δ − sin δ 0sin I sin δ cos δ sin I cos I

− cos I sin δ − cos I cos δ sin I

where δ (∼ 22) and I (∼ 77.5) are the declination and dip angles, respectively.Then,

k =

kpe

kpn

kap

=

ke cos δ − kn sin δkz cos I + sin I (kn cos δ + ke sin δ)kz sin I − cos I (kn cos δ + ke sin δ)

.



The Future


Vector Velocities - Two Point

Two LOS velocity measurements can be written as,

[

v1los

v2los

]

=

[

k1pe k1

pn k1ap

k2pe k2

pn k2ap

]

vpe

vpn

vap



The Future




[

v1los

v2los

]

=

[

k1pe k1

pn k1ap

k2pe k2

pn k2ap

]

vpe

vpn

vap

Can be solved for vpn and vpe assuming vap ≈ 0,

vpn =v1los −

k1pe

k2pe

v2los − vap

(

k1ap − k2

ap

k1pe

k2pe

)

k1pn

(

1 −k2

pn

k1pn

k1pe

k2pe

) ≈

v1los −

k1pe

k2pe

v2los

k1pn

(

1 −k2

pn

k1pn

k1pe

k2pe

)



The Future




[

v1los

v2los

]

=

[

k1pe k1

pn k1ap

k2pe k2

pn k2ap

]

vpe

vpn

vap

Can be solved for vpn and vpe assuming vap ≈ 0,

vpn =v1los −

k1pe

k2pe

v2los − vap

(

k1ap − k2

ap

k1pe

k2pe

)

k1pn

(

1 −k2

pn

k1pn

k1pe

k2pe

) ≈

v1los −

k1pe

k2pe

v2los

k1pn

(

1 −k2

pn

k1pn

k1pe

k2pe

)

Implies that you need look directions with different k vectors.



The Future


Vector Velocities - Generalization

Multiple measurements can be written as,

v1los

v2los...

vnlos

=

k1pe k1

pn k1ap

k2pe k2

pn k2ap

......

...knpe kn

pn knap

vpe

vpn

vap

+

e1los

e2los...

enlos

orvlos = Avi + elos



The Future




v1los

v2los...

vnlos

=

k1pe k1

pn k1ap

k2pe k2

pn k2ap

......

...knpe kn

pn knap

vpe

vpn

vap

+

e1los

e2los...

enlos

orvlos = Avi + elos

Treat vi as a Gaussian random variable (Bayesian), use linear theory to derive aleast-squares estimator.



The Future




v1los

v2los...

vnlos

=

k1pe k1

pn k1ap

k2pe k2

pn k2ap

......

...knpe kn

pn knap

vpe

vpn

vap

+

e1los

e2los...

enlos

orvlos = Avi + elos

Treat vi as a Gaussian random variable (Bayesian), use linear theory to derive aleast-squares estimator. vi zero mean, Σv (a priori). Measurements zero mean,covariance Σe .



The Future




v1los

v2los...

vnlos

=

k1pe k1

pn k1ap

k2pe k2

pn k2ap

......

...knpe kn

pn knap

vpe

vpn

vap

+

e1los

e2los...

enlos

orvlos = Avi + elos

Treat vi as a Gaussian random variable (Bayesian), use linear theory to derive aleast-squares estimator. vi zero mean, Σv (a priori). Measurements zero mean,covariance Σe . Solution,

vi = ΣvAT (AΣvA

T + Σe)−1vlos

Error covariance,

Σv = Σv − ΣvAT (AΣvA

T + Σe)−1AΣv = (ATΣ−1

e A + Σ−1v )−1



The Future


Electric Fields

While above approach can be used to resolve vectors as a function of altitude(or anything else), we often want to resolve vectors as a function of invariantlatitude.



The Future


Electric Fields


In the F region (above ∼ 150 − 175 km), plasma is E × B drifting.



The Future


Electric Fields


In the F region (above ∼ 150 − 175 km), plasma is E × B drifting.


Electric Fields - Example

Electron Density

LOS Velocities


Resolved Vectors


Comparison to rocket-measured E-fields.

Experiment PlanningThe approach also allows for an efficient means of experiment planning, since the outputcovariance of the measurements is independent of the actual measurements.

Experiment PlanningThe approach also allows for an efficient means of experiment planning, since the outputcovariance of the measurements is independent of the actual measurements.


The Future


E-Region Winds

At lower altitudes, the ions become collisional and transition from E×B drifting athigh altitudes to drifting with the neutral winds at low altitudes.



The Future


E-Region Winds

At lower altitudes, the ions become collisional and transition from E×B drifting athigh altitudes to drifting with the neutral winds at low altitudes.The steady state ion momentum equations relate the vector velocities (as afunction of altitude) to electric fields and neutral winds

0 = e(E + vi × B) − miνin(vi − u)



The Future


E-Region Winds


0 = e(E + vi × B) − miνin(vi − u)

Defining the matrix C as,

C =

(1 + κ2i )

−1−κi(1 + κ2

i )−1 0

κi(1 + κ2i )

−1 (1 + κ2i )

−1 00 0 1

where κi = eB/miνin = Ωi/νin.



The Future


E-Region Winds


0 = e(E + vi × B) − miνin(vi − u)

Defining the matrix C as,

C =

(1 + κ2i )

−1−κi(1 + κ2

i )−1 0

κi(1 + κ2i )

−1 (1 + κ2i )

−1 00 0 1

where κi = eB/miνin = Ωi/νin. The vector velocity can then be solved for

vi = biCE + Cu

where bi = e/miνin = κi/B



The Future


E-Region Winds

vi = biCE + Cu



The Future


E-Region Winds

vi = biCE + Cu

Defining a new matrix asD = [biC C ]



The Future


E-Region Winds

vi = biCE + Cu


we can write the forward model

vlos = (A · D)x + elos .



The Future


E-Region Winds

vi = biCE + Cu




An obvious problem is the ambiguity in terms of E and u. Solution is to invert allmeasurements from all altitudes at once, allowing winds to vary with altitude butthe electric field to map along field lines.



The Future


E-Region Winds

vi = biCE + Cu




An obvious problem is the ambiguity in terms of E and u. Solution is to invert allmeasurements from all altitudes at once, allowing winds to vary with altitude butthe electric field to map along field lines. Forward model becomes,

x = [Epe Epn E|| u1pe u1

pn u1|| u2

pe u2pn u2

|| ... unpe un

pn un||]

T



The Future


E-Region Winds

vi = biCE + Cu




An obvious problem is the ambiguity in terms of E and u. Solution is to invert allmeasurements from all altitudes at once, allowing winds to vary with altitude butthe electric field to map along field lines. Forward model becomes,

x = [Epe Epn E|| u1pe u1

pn u1|| u2

pe u2pn u2

|| ... unpe un

pn un||]

T

This allows for direct constraint of both the vertical wind and the parallel electricfield, both of which we expect to be small.

Σgmagv = Jgeo→gmagΣgeo

v JTgeo→gmag


E-Region Winds - Example



Alti

tude

(km

)

01/23−00 01/23−12 01/24−00 01/24−12 01/25−00 01/25−12 01/26−00 01/26−12 01/27−00

85

90

95

100

105

110

115

120

125

Ver

tical

(m

/s)

−30

−20

−10

0

10

20

30

Alti

tude

(km

)

01/23−00 01/23−12 01/24−00 01/24−12 01/25−00 01/25−12 01/26−00 01/26−12 01/27−00

85

90

95

100

105

110

115

120

125

Mer

idio

nal (

m/s

)

−200

−100

0

100

200

Alti

tude

(km

)

01/23−00 01/23−12 01/24−00 01/24−12 01/25−00 01/25−12 01/26−00 01/26−12 01/27−00

85

90

95

100

105

110

115

120

125

Zon

al (

m/s

)

−200

−100

0

100

200

01/23−00 01/23−12 01/24−00 01/24−12 01/25−00 01/25−12 01/26−00 01/26−12 01/27−00−0.05

0

0.05

Time UT

E−

field

(V

/m)

10

20

30

40

50

60ZonalMeridional


The Future


Collision Frequency

Two approaches (that I know of) for assessing collision frequency:



The Future


Collision Frequency


1 Direct fits at lower altitudes (spectral width ∼∝ Tn/νin)



The Future


Collision Frequency


1 Direct fits at lower altitudes (spectral width ∼∝ Tn/νin)

2 Examination of variation of LOS velocity with altitude



The Future


Collision Frequency - Method 1

Semi-diurnal variation overseveral days.

4

4.2

4.4

4.6

4.8

5Vertical Beam

log

10

ν in (

s−

1) 87.8 km

3.8

4

4.2

4.4

4.6

log

10

ν in (

s−

1) 92.3 km

3.5

4

4.5

log

10

ν in (

s−

1) 96.8 km

12/16−12 12/17−00 12/17−12 12/18−00 12/18−12 12/19−00 12/19−12 12/20−00

3.4

3.6

3.8

4

4.2

log

10

ν in (

s−

1) 101.3 km

Up B Beam

85.7 km

90.1 km

94.5 km

12/16−12 12/17−00 12/17−12 12/18−00 12/18−12 12/19−00 12/19−12 12/20−00

98.9 km

Altitude profile andextrapolation.

10−2

10−1

100

101

102

80

85

90

95

100

105

110

115

120

125

130

Alti

tud

e (

km)

Ωi/ ν

in

Beam 1

H=5.9 km

10−2

10−1

100

101

102

80

85

90

95

100

105

110

115

120

125

130

Ωi/ ν

in

Beam 2

H=6.1 km

10−2

10−1

100

101

102

80

85

90

95

100

105

110

115

120

125

130

Ωi/ ν

in

Beam 3

H=4.5 km

10−2

10−1

100

101

102

80

85

90

95

100

105

110

115

120

125

130

Ωi/ ν

in

Beam 4

H=6.4 km

10−2

10−1

100

101

102

80

85

90

95

100

105

110

115

120

125

130A

ltitu

de

(km

)

Ωi/ ν

in

Beam 5

H=6.2 km

10−2

10−1

100

101

102

80

85

90

95

100

105

110

115

120

125

130

Ωi/ ν

in

Beam 6

H=5.6 km

10−2

10−1

100

101

102

80

85

90

95

100

105

110

115

120

125

130

Ωi/ ν

in

Beam 7

H=6.3 km



The Future


Collision Frequency - Method 2 - Example

The rotation of the LOS velocity with altitude is a good indicator of collisionfrequency effects.



The Future



The rotation of the LOS velocity with altitude is a good indicator of collisionfrequency effects.E.g., take the vertical beam,

vz = v⊥n cos I + v|| sin I



The Future





Perp-north and parallel components given by,

v⊥n = κi (1 + κ2i )

−1 (biE⊥e + u⊥e) + (1 + κ2i )

−1 (biE⊥n + u⊥n)

v|| = u|| + biE||



The Future






v⊥n = κi (1 + κ2i )

−1 (biE⊥e + u⊥e) + (1 + κ2i )

−1 (biE⊥n + u⊥n)

v|| = u|| + biE||

Define a new variable,v ′z = vz − v|| sin I



The Future






v⊥n = κi (1 + κ2i )

−1 (biE⊥e + u⊥e) + (1 + κ2i )

−1 (biE⊥n + u⊥n)

v|| = u|| + biE||

Define a new variable,v ′z = vz − v|| sin I

Under strong convection (electric field) conditions, neglect winds

v ′z ∼ bi (1 + κ2

i )−1 [κiE⊥e + E⊥n] cos I



The Future



v ′z ∼ bi (1 + κ2




The Future



v ′z ∼ bi (1 + κ2


If κi(z) = κ0e(z−z0)/H , vertical ion velocity will maximize at

zmax v ′

z= z0 + H lnκ−1

0 + H ln

[

cosα ± 1

sin α

]



The Future



v ′z ∼ bi (1 + κ2


If κi(z) = κ0e(z−z0)/H , vertical ion velocity will maximize at

zmax v ′

z= z0 + H lnκ−1

0 + H ln

[

cosα ± 1

sin α

]

−150 −100 −50 0 50 100 150100

110

120

130

140

150

160

α (degrees)

Altitude of Max Vz ′

NESE NWSW

−1 −0.5 0 0.5 1100

110

120

130

140

150

160

NS EW NESW SENW

Alti

tud

e (

km)

Normalized Vz ′

Vz′(z)



The Future





The Future



Profiles of v ′

z during high convection conditions.Dashed - with MSIS; Solid - scaled by a factor of 2.

−50 0 50 100100

110

120

130

140

150

16017/7.00−8.00 UT

Vz

′ (m/s)

Alti

tud

e (

km)

|E|=30 mV/m

α=89 °

−100 −50 0

17/12.15−12.50 UT

Vz

′ (m/s)

|E|=64 mV/m

α=−87 °

0 50 100

18/5.00−5.50 UT

Vz

′ (m/s)

|E|=35 mV/m

α=79 °

−100 −50 0 50

18/5.00−5.50 UT

Vz

′ (m/s)

|E|=35 mV/m

α=−95 °



The Future


Conductivities / Currents / Joule Heating Rates


Conductivities / Currents / Joule Heating Rates

D-Region Parameters - Raw Power and Spectra

D-Region Parameters - Raw Power and Spectra

D-Region Parameters - Ne and Spectral Widths

D-Region Parameters - Ne and Spectral Widths

D-Region Parameters - Velocities and Winds

D-Region Parameters - Velocities and Winds


The Future

Future

1 Move towards full profile techniques

2 Take advantage of space and time information

3 Standardize approaches

4 Molecular ion composition, height-resolved plasma lines,topside parameters, etc.

5 Make these products available to interested users

6 Extend our arsenal of products (e.g., D-regionmomentum fluxes, higher altitude winds, etc.)


Documents

Michael J. Nicolls and Craig J. Heinselman CEDAR, 21 June 2008isr.sri.com/iono/amisr/meetings/2008/2008-tutorials/Data Processing… · Michael J. Nicolls and Craig J. Heinselman