Embed Size (px)
Citation preview
Assimilation procedures for updating ionospheric profilesbelow the F2 peak
Leo F. McNamara,1 Matthew J. Angling,2 Sean Elvidge,2,3 Sergey V. Fridman,4
Mark A. Hausman,4 L. J. Nickisch,4 and Lee-Anne McKinnell5
Received 13 November 2012; accepted 29 January 2013; published 24 April 2013.
[1] This paper describes and compares two real-time assimilative ionospheric models, withan emphasis on their ability to provide accurate profiles of the electron density below the peakof the F2 layer at a midlatitude location, given automatically processed vertical incidenceionograms at a single location. The two models are specifically oriented toward severalimportant practical applications of high-frequency (HF) radio propagation: HFcommunications, single station location of HF transmitters, and coordinate registration forOTHR. Both models start with the International Reference Ionosphere (IRI) as a backgroundionosphere and assimilate digisonde observations (either the ionogram or the profile) andavailable GPS total electron content observations. The digisonde data from one site in theRepublic of South Africa (RSA) provide the ionosonde assimilation data, while the otherthree digisondes in the RSA provide the ground-truth observations of foF2, hmF2, and theplasma frequency profile. Since the four RSA digisondes receive each other’s transmissions,maximum observed frequencies have also been used as ground truth. The models tested haveboth been found to provide significant improvements over the IRI and to have similaraccuracies for the study interval (September 2011). The errors of the models are very close tothe minimum achievable errors for all the validation parameters, which seem to be set by theubiquitous traveling ionospheric disturbances being different at the different locations. For theoptimum ground-truth location ~685 km from the digisonde providing the assimilation data,the RMS errors in foF2 were found to be 0.2MHz (night) and 0.5MHz (day).
Citation: McNamara, L. F., M. J. Angling, S. Elvidge, S. V. Fridman, M. A. Hausman, L. J. Nickisch, and L.-A.McKinnell (2013), Assimilation procedures for updating ionospheric profiles below the F2 peak, Radio Sci., 48, 143–157,doi:10.1002/rds.20020.
1. Introduction
[2] The assimilative models considered in this study areGPSII (GPS Ionospheric Inversion) and EDAM (ElectronDensity Assimilation Model), which are discussed in moredetail in the following section. Both models can be usedfor global specifications, but the nature of the assimilationdata, especially in the current study, often limits the effectivecoverage to more restricted regions.[3] The analysis interval for the present study is September
2011, and the study concentrates on observations from the
four digisondes in the Republic of South Africa (RSA).September 2011 was chosen for analysis mainly because itwas the last full month before the study was initiated. As itturned out, September 2011 showed a large amount of solarand geomagnetic variability. For example, (1) the dailysunspot number ranged from 47 to 167, (2) the global Apindex exceeded the nominal storm threshold (25) on 6 days(253, 254, 255, 260, 279, and 270), and (3) the Dst index wentdown to �103 near midnight on 26/27 September.[4] The models are initially compared in terms of the accu-
racy of their values of three peak parameters: foF2, hmF2, andh0.8 (the altitude at which the profile has a plasma frequencyequal to 0.8 foF2). foF2 and hmF2 are standard metricsadopted in other analyses. The models are then compared interms of their ability to specify the maximum observedfrequency (1F MOF) for one-hop high-frequency (HF) propa-gation on three circuits in the RSA (i.e., the highest frequencythat would be supported for oblique propagation on a point-to-point circuit). The MOF comparisons are directly relevant tothe major HF applications, which all involve oblique HF radiopropagation. Finally, the models are compared in terms of theaccuracy of their plasma frequency profiles.[5] The RSA digisondes were also used as assimilation and
ground-truth data in an earlier study by McNamara et al.
1Air Force Research Laboratory, AFRL/RVBXI, Kirtland AFB,Albuquerque, New Mexico, USA.
2Centre for RF Operational Environments, QinetiQ, Malvern, UK.3Poynting Research Institute, University of Birmingham, Birmingham,
UK.4North West Research Associates, Monterey, California, USA.5Space Science, South African National Space Agency, Hermanus,
South Africa.
Corresponding author: L. F. McNamara, Air Force Research Labora-tory, AFRL/RVBXI, Kirtland AFB, Albuquerque, NM 87117, USA.([email protected])
©2013. American Geophysical Union. All Rights Reserved.0048-6604/13/10.1002/rds.20020
143
RADIO SCIENCE, VOL. 48, 143–157, doi:10.1002/rds.20020, 2013
[2011]. Pezzopane et al. [2011] describe how the joint use ofautoscaled data and two climatologic ionospheric modelscan provide a useful tool for obtaining a real-time three-dimensional electron density mapping of the ionosphere inthe central Mediterranean region. Settimi et al. [2013] madelimited comparisons of calculated and observed obliqueionograms for a 1235 km Rome-Crete circuit, using theionospheric specification developed by Pezzopane et al.[2011]. They found that this ionospheric specification gavemore accurate oblique ionograms than the IRI.[6] Section 2 of this paper reviews the GPSII and EDAM
models. Section 3 describes the assimilation scenarios andintroduces the observed values of the MOF and variousionogram processing issues. Section 4 briefly discussesthe observed traveling ionospheric disturbances (TIDs).Because the TIDs are different at the different digisondesites, they have a strong influence on the achievable accura-cies, as discussed in section 5. The assimilated digisondedata were processed automatically (autoscaled), necessitat-ing some filtering of that data, as described in section 6. Someobviously spurious model results are also filtered out. Sections7, and 9 present the model results for foF2, hmF2, and h(0.8foF2). The model values of the 1F MOF are discussed insection 10. The accuracy of the GPSII, EDAM, and IRIplasma frequency profiles are discussed in section 11. Becausethere are ~2800 profiles for each digisonde, the comparisonsare heavily summarized into profiles of the average andstandard deviation of the errors in plasma frequency. Theconclusions of the study are presented in section 12.
2. The Assimilation Models
[7] GPSII and EDAM both use the International Refer-ence Ionosphere IRI-2007 (http://iri.gsfc.nasa.gov/) astheir background model. The latest version is IRI-2012,which is equivalent to the IRI-2007 that was used for thepresent study.
2.1. IRI-2007
[8] The IRI is a monthly median model that draws favorbecause of its ease and speed of use and generally accept-able level of validity. It uses either the URSI or CCIRworld maps of foF2 and M(3000)F2 to define the F2peak parameters foF2 and hmF2, and thence the plasmafrequency profile. The world maps relate the monthlymedian values of foF2 and M(3000)F2 to two indices,namely the 13month running means IG12 and RZ12. Thevalues of these indices are made available in a tableIG_RZ.dat that is regularly updated. Final values of theseindices for the study interval, September 2011, were notavailable when GPSII and EDAM were run, so the teamsagreed to use IG12 = 67.4 and RZ12 = 47.3. These valueswere passed to the IRI as the variables oarr(39) and oarr(33), respectively, and took precedence over the values inthe table IG_RZ.dat.[9] In fact, neither GPSII nor EDAMshowed any sensitivity
to the assumed values of IG12 and RZ12, as long as they werereasonable choices. The main reason for care in selecting thevalues was to establish representative IRI baseline values ofthe F2 peak parameters for comparison with the GPSII andEDAM results.
2.2. EDAM
[10] EDAM has been developed to assimilate ionosphericmeasurements into a background ionospheric model [Anglingand Khattatov, 2006; Angling et al., 2009]. The backgroundmodel is currently provided by IRI-2007 [Bilitza and Reinisch,2008]. The EDAM assimilation is based on a weighted,damped least mean squares estimation. This is a form ofminimum variance optimal estimation (or best linear unbiasedestimation) which provides an expression for an updatedestimation of the state (the analysis) that is dependent on aninitial estimate of the state (provided by the backgroundmodel) and the differences between the background modeland the observations [Menke, 1989; Twomey, 1977].[11] A magnetic coordinate system is used in this assimila-
tion, along with a time step of 15min. The electron densitydifferences between the voxels of the analysis and the back-ground model are propagated from one time step to the nextby assuming persistence combined with an exponentialdecay. The time constant for this decay is set at 4 h. Thus,if the data feed is interrupted, the analysis will decay backto the background model.[12] EDAM assimilates GPS slant total electron content
(TEC) observations as well as ionosonde information in theform of the ionogram virtual height traces [Angling andJackson-Booth, 2011]. It can also assimilate radio occultation(RO) observations of slant TEC [Angling and Cannon, 2004;Angling, 2008] and in situ electron densities, but these capabil-ities were not exercised for the present study. During the studyperiod, very few RO observations were available over theRSA, which covers a relatively small part of the globe.[13] This study is the first extensive analysis of EDAM
results when ionogram traces were assimilated and indepen-dent ionosondes were used to provide truth data. An earlierstudy addressed only a single day of data and was limitedto an examination of the residual errors [Angling andJackson-Booth, 2011].
2.3. GPSII
[14] GPSII (pronounced “gypsy”) is a recursive data assim-ilation algorithm [Fridman et al., 2006, 2009] that provides anelectron density distribution model for a fixed geographicalarea. At each time step of the solution, the resulting electrondensity model matches all ionospheric data accumulatedduring the model update time interval to within the datameasurement error. The updatedmodel is obtained bymodify-ing the solution extrapolated from the preceding step. Thisdata-driven modification of the model is determined with thehelp of the Tikhonov method [Tikhonov and Arsenin, 1977].The GPSII processing ensures that the resulting model isspatially and temporally smooth and is yet in agreement withall measurements.[15] Ionospheric measurements that GPSII is able to assim-
ilate include raw absolute and relative TEC data from ground-and space-based GPS receivers, relative TEC data obtainedwith LEO satellite beacons, in situ electron density measure-ments, and data from vertical incidence ionosondes. GPSIIis also able to assimilate various HF measurements such asOTHR backscatter ionograms [Fridman et al., 2012], butthese capabilities were not exploited for the present effort.GPSII assimilated only data from ground-based GPS receiversand bottomside profiles of electron density from selected
MCNAMARA ET AL.: UPDATING IONOSPHERIC PROFILES
144
ionosondes (Grahamstown). IRI-2007 [Bilitza and Reinisch,2008] was used as the background model. The solution updateinterval was 15min.
2.4. Biases in the Validations
[16] Assimilation of the ionogram virtual height trace (byEDAM), rather than the true height plasma frequency profile(as done by GPSII), avoids having to use an independentprocedure to perform the ionogram inversion to derive theplasma frequency profile. The best known methods forinverting an ionogram to get the profile are NHPC [Reinischand Huang, 1983] and POLAN [Titheridge, 1988]. In bothcases the inversion procedure must make assumptions aboutthe profile below the first frequency at which ionogramechoes are obtained as well as in the E-F valley.[17] The different assimilation approaches used by GPSII
and EDAM lead to potential biases in favor of GPSII whenvalidating the model values of hmF2 and the profile shapes.In particular, GPSII assimilates the ARTIST/NHPC profiles(for Grahamstown), while the ARTIST/NHPC profiles atthe other three digisonde sites are used as ground truth. Onthe other hand, EDAM assimilates the virtual height traceat Grahamstown.[18] However, the model validations against the observed
1F MOF on the Hermanus and Louisvale to Grahamstowncircuits are independent of the ARTIST/NHPC profiles andtherefore favor neither model.
3. The Assimilation Scenario
[19] As with the study by McNamara et al. [2011], thefour digisondes (actually the Digisonde DPS-4D) [Reinischet al., 2009] in the RSA provided the ionograms and derivedplasma frequency profiles. Figure 1 shows the locations ofthe digisondes.[20] For the present study, the Grahamstown (GR13L)
ionograms were used to provide assimilation data, while theHermanus (HE13N), Louisvale (LV12P), and Madimbo(MU12K) ionograms were used to provide the ground-truth
observations. The ionogram observations can be downloadedfrom the University of Massachusetts Lowell “DIDBase”http://umlcar.uml.edu/DIDBase/.
3.1. RSA Digisonde Ionograms
[21] Figure 2 shows as an example the Grahamstownionograms for 20110916 (day 259 in 2011), 1200 UT.[22] Because the four digisondes are GPS-synchronized,
they can receive each other’s transmissions as well as theirown, thus providing both vertical incidence and obliqueincidence ground truths. Figure 2 shows the usual verticalincidence traces out to ~9MHz (red: ordinary trace; green:extraordinary trace), with double-hop traces at virtual heightsof ~600 to 650 km.[23] The two sets of multicolored traces that go out to
~11.5MHz correspond to one-hop oblique propagation fromthe digisondes at Hermanus and Louisvale. There are ordinaryand extraordinary traces for each circuit. The nose of eachtrace is the junction frequency or MOF for that polarizationand circuit. There is a partial two-hop oblique Madimbo-Grahamstown trace near 8.5MHz, 800 km. The four MOFsin Figure 2 are as follows: LV12P (O-ray), 11.4MHz;LV12P (X-ray), 11.7MHz; HE13N (O-ray), 11.7MHz; andHE13N (X-ray), 11.9MHz.
3.2. RSA Oblique Propagation
[24] Figures 3 and 4 show the diurnal variation of theobserved 1F MOFs (i.e., for a 1F mode) for the threeGrahamstown circuits for days 253 (10 September) and270 (27 September) of 2011.[25] Days 252 and 253 were disturbed days (Ap = 36, 33).
Day 253 was the only day for which the Madimbo-Grahamstown 1F MOF was lower than the digisondes’frequency limits at both locations during daylight hours. Day270 shows a local increase in the 1F MOF near 21 UT, whichshould be reproduced by the two models. Both figures showthe presence of TIDs, which also appear in the other obser-vations such as foF2 and hmF2. We discuss the TIDs brieflyin section 5.[26] Because the circuits from Hermanus (685 km) and
Louisvale (735 km) to Grahamstown are very similar inlength, the MOFs are also very similar. The changes to thepropagation conditions due to ubiquitous TIDs often precludethe use of the relative sizes of the MOFs to identify the twosets of traces. However, we have followed the argumentsgiven by Davies [1990, p. 165] to identify the traces withthe smaller O-X separation as corresponding to the west-east Hermanus-Grahamstown circuit. The ordinary mode1F MOFs for the Hermanus and Louisvale circuits have beenscaled manually using this criterion to identify the two circuits.The MOFs for the Madimbo-Grahamstown circuit are signifi-cantly greater than those for the other circuits, and there areno identification issues. Unfortunately, the MOFs for theMadimbo circuit exceeded the adjustable upper frequency ofthe Grahamstown digisondes during most of the day (theionosonde upper frequency limits are appropriate to verticalpropagation, not oblique).
3.3. Observations Available to Modelers
[27] The modeling teams were given free rein as to whatdata other than ionosonde data they assimilated, but bothteams restricted themselves to ground-based GPS TEC
Figure 1. Locations of the four Republic of South Africaionosondes (Digisonde DPS-4D). Louisvale and Hermanusare about 700 km from Grahamstown. Madimbo is about1300 km from Grahamstown.
MCNAMARA ET AL.: UPDATING IONOSPHERIC PROFILES
145
observations available from http://gsac.ucsd.edu, in particular,the GPS sites ABPO, ADIS, CAGZ, HARB, HNUS, MAD2,MAL2, MAS1, MAT1, MAUA, NKLG, NOT1,NURK,RBAY, REUN, SEY1, TDOU, WIND, YRCM, and ZAMB.Details of the available GPS sites are available at http://igscb.jpl.nasa.gov/network/list.html. AFRL validation studies,as well as studies using EDAM, showed that RO observationsof TEC tend to be too sparse to contribute usefully when GPSTEC data are also available (the RO data are more importantover the oceans).[28] The RSA ionograms and autoscaled data were
downloaded from the University of Massachusetts LowellDIDBase by each team member using SAO Explorer. The
ground-truth ionogram traces and vertical incidence obser-vations were thus freely available. However, the obliqueionograms were scaled manually by LFM and provided anindependent set of ground-truth observations. The verticalionograms were automatically processed (autoscaled) byARTIST5 [Galkin et al., 2007].
4. Properties of the Observed TIDs
[29] TIDs are ubiquitous in the RSA observations. Wehave already seen them in the observations of the 1F MOF(Figures 3 and 4). Midlatitude TIDs have a wide range ofperiods, ranging from 15–20min to several hours. Since the
800
750
700
650
600
550
500
450
400
350
300
250
200
150
802.0 2.5 3.0 3.5 4.0 5.04.5 5.5 6.0 6.5 7.0 7.5 8.0 8.5 9.5 10.0 10.5 11.0 11.5 12.0
SAOExplorer, v 3.4.19
9.0
Grahamstown, GR13L 2011.09.16 (259) 12:00:00.000
Figure 2. Grahamstown ionograms for 20110916 (day 259), 1200 UT. The four ionogram traces extendingto ~11.5MHz correspond to oblique propagation from Hermanus and Louisvale to Grahamstown.
0 4 8 12 16 20 24
Universal Time
2
3
4
5
6
7
8
9
10
11
12
13
14
Max
imum
Obs
erve
d F
requ
ency
[MH
z]
Hermanus-GrahamstownLouisvale-GrahamstownMadimbo-Grahamstown
Figure 3. Maximum observed frequency for the threeGrahamstown circuits, day 253, 2011.
0 4 8 12 16 20 24
Universal Time
2
3
4
5
6
7
8
9
10
11
12
13
14M
axim
um O
bser
ved
Fre
quen
cy [M
Hz]
Hermanus-GrahamstownLouisvale-GrahamstownMadimbo-Grahamstown
Figure 4. Maximum observed frequency for the threeGrahamstown circuits, day 270, 2011.
MCNAMARA ET AL.: UPDATING IONOSPHERIC PROFILES
146
observations are made only every 15min, short-period TIDscannot be studied with the present data set. There is also aCatch-22 situation in that the TIDs perturb the ionogram tracesand make it difficult to measure the key ionospheric character-istic (foF2) that we analyze to determine their properties.[30] Preliminary analysis of the observations of foF2 using
the Lomb-Scargle periodogram approach has isolated a periodof ~2 h in some situations, which is consistent with the 1FMOF observations. Hunsucker [1982] classifies TIDs withperiods of 30min to 3 h as “Large Scale TIDs”, which havehorizontal velocities of 400 to 1000m/s and wavelengthsgreater than 1000 km. These large-scale traveling ionosphericdisturbances (LSTIDs) are generated in the polar regionsduring geomagnetic storms and propagate equatorward.[31] Figure 5 shows by way of illustration the deviations
of the Hermanus foF2 from a smoothed curve for days259–261 (chosen because the TIDs are well defined). Apwas 32 on day 260, with very quiet days before and after.There is clearly a disturbance with a period of ~2 h. Theperturbations at night are smaller than those during the dayand close to the measurement error. Ignoring the extremevalues (some of which could be suspect), the TID variabilityis about �4% of the smoothed values of foF2.[32] The Lomb-Scargle approach found an ~2 h periodicity
(1.5 to 2.5 h) in the Hermanus foF2, but only for disturbeddays. For example, day 260 (Ap= 32) showed a clear peakat 1.90 h, while day 269 (Ap= 67) showed peaks at 1.696,1.667, and 1.508 h. The difference between periodicities fordifferent disturbed days would not be unexpected if theLSTIDs are generated in the distant polar regions.
5. The Minimum Achievable Error
[33] Neither GPSII nor EDAM takes specific account ofthe perturbations in the profiles and peak parameters causedby TIDs. Subjective comparisons of the fine-scale variationsof the observed daytime values of foF2 show that they
are not similar in detail for different digisondes, evenallowing for possible phase shifts. The same situation holdsfor the 1F MOF observations illustrated in Figures 3 and 4.We therefore propose that the differences between the TIDeffects at the driver station (Grahamstown) and theground-truth stations (Hermanus, Louisvale, and Madimbo)will set a minimum error that can be achieved by GPSIIor EDAM.[34] As an estimate of this minimum achievable error, we
take the standard deviation of the differences between corre-sponding observed values of the driver (Grahamstown) andground-truth (such as Hermanus) values of foF2. Averagedifferences are ignored on the basis that the models wouldhave no systematic errors when they are working perfectly.The actual RMS values of the GPSII/EDAM errors can thenbe compared with the standard deviations of the differencesin the observed values of foF2.[35] Figure 6 indicates that the minimum achievable error in
foF2 at Hermanus will have a standard deviation of ~0.4MHzduring the middle of the day, when foF2 values are highest,dropping down to ~0.2MHz at night. The minimum achiev-able error is ~50% higher for Grahamstown-Louisvale, andhigher still for Grahamstown-Madimbo. Since the observedmidday values of foF2 are ~9MHz (as shown in the laterFigure 9), the minimum achievable error for the HermanusfoF2 at 11 UT (noon) is ~4%.[36] Figure 7 shows the estimated minimum achievable
error in the peak height hmF2 at the three ground-truth sites.The standard deviation of the differences has a local maxi-mum in the middle of the day, local minima near 06 and16 UT, and high values at night. This diurnal variationfollows that of hmF2 (see the later Figure 12 for the Hermanusvalues of hmF2). In the middle of the day, the RMS errors inhmF2 correspond to ~6% of hmF2.[37] It can be expected that the minimum achievable errors
would be affected by ionogram autoscaling inaccuracies as
0 12 24 36 48 60 72
Elapsed Time (hours)
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
Dep
artu
re o
f foF
2 fr
om S
moo
thed
Var
iatio
n [M
Hz]
Figure 5. Hermanus deviations of foF2 from a smoothedcurve for days 259–261.
0 4 8 12 16 20 24
Universal Time
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1.1
1.2
Sig
ma
of D
iffer
ence
in fo
F2
[MH
z]
HE13N-GR13LLV12P-GR13LMU12K-GR13L
Figure 6. Diurnal variation of the standard deviation of thedifferences between the observed Grahamstown and ground-truth values of foF2 at the other two digisonde sites.
MCNAMARA ET AL.: UPDATING IONOSPHERIC PROFILES
147
well as by the different TID effects, but there seems to be noway to disentangle the two effects.
6. Filtering the Observations and Model Results
[38] The assimilated Grahamstown digisonde observationswere all automatically processed (autoscaled) by ARTIST5,as were the observations for the three ground-truth digisondes.While the RSA digisondes are well maintained, the RSAionograms are relatively simple, and ARTIST5 is a significantadvance on earlier versions of ARTIST, there are still someautoscaling “blunders.” These blunders can lead to apparentlylarge errors in the model results, so procedures have beenadopted to prevent them from distorting the results of thepresent study.[39] The first step is to filter the assimilation data. Since this
process is automated, both GPSII and EDAMwill produce theoccasional large error which is due to undetected autoscalingissues. The GPSII/EDAM/IRI values of the validation param-eters are therefore filtered before they are presented here.
6.1. Filtering the Autoscaled Assimilation Data
[40] The final comparison and analysis of the model out-puts, including filtering of errant results, were performedby LFM. The first step in the filtering process was to runQualScan [McNamara, 2006] for all ionograms for all foursites. QualScan performs multiple checks on the likelyvalidity of the scaled ionogram trace, calculates the POLAN[Titheridge, 1988] plasma frequency profile, and comparesthe POLAN and ARTIST (NHPC) profiles to establish theplasma frequency and uncertainty at a standard grid of altitudes.Profiles such as these are provided to the Air Force WeatherAgency global real-time model of the ionosphere.[41] For the Grahamstown ionograms, there were nomi-
nally 30� 96 = 2880 ionograms. QualScan immediatelyrejected ~10% of these because ARTIST5 had determinedpost facto that its results were unreliable. Another ~7% were
subsequently rejected mainly because POLAN would proba-bly have failed to derive a profile from the scaled trace, evenafter some judicious patching of the ionogram trace byQualScan, or the derived POLAN profile was nonphysical insome way. As well as “failing” an ionogram because noPOLAN profile was derived, QualScan provides a qualityfigure for each ionogram. In this case, ~10% of the Grahams-town quality figures were 0 (could not process ionogram) or 1(high risk/low quality), leaving ~90% with acceptable or goodquality. The autoscaling process can be affected detrimentallyby the presence of spread F echoes, but extensive perusal ofthe ionograms suggested that this was not an issue for thecurrent study. This conclusion is consistent with the statisticalcharacterization of spread F over South Africa made byAmabayo et al. [2011].[42] The essential elements of QualScan’s processing are
saved in a file called peak_vs_time.dat, which lists ~30 character-istics of the ARTIST and POLAN profiles. These peak_vs_time.dat files for the four digisonde sites play a key role in filteringthe model results. An ionogram (and a GPSII/EDAM/IRIspecification) is included in the error analysis if it appears inthe peak_vs_time.dat files for both Grahamstown and thedigisonde location being processed. Recall that Grahamstownprovides the assimilation data, while the other three sites areground-truth sites. An ionogram at either site is also rejectedif it fails some other simple sanity checks on the ARTISTvalues of the peak height hmF2 and the obliquity factorM(3000)F2. The test on hmF2 mainly rejects some invalidARTIST results for Louisvale daytime ionograms.
6.2. Filtering the Model Results
[43] The ARTIST5 autoscaled values of foF2 are generallyvery reliable for the present set of ionograms, and both GPSIIand EDAM yield very small RMS errors in foF2, i.e., onlyabout a third the size of the IRI errors (as discussed in section8). However, the situation is not so simple for the models’nighttime values of hmF2 (and to some extent the height h(0.8foF2), the height of the subpeak F2 layer at 0.8 foF2) forwhich a few percentages of outlier values have sufficientlylarge errors to severely distort the calculated RMS errors.[44] In principle, the large errors can arise in two ways:
undetected bad data can be assimilated, or good data canbe assimilated badly. The first case includes the effects ofundetected autoscaling blunders. These can cause modelerrors directly as the bad data are assimilated and for sometime afterward, since the models have to “unlearn” the effectsof bad assimilation data. Deciding whether a new observationthat differs significantly from the preceding ones is valid ispart of the “innovation” issue in data assimilation. BothEDAM and GPSII are affected by these errors.[45] However, for EDAM, a small number of cases appear
to be affected in the second way, i.e., reasonable virtual heighttraces are assimilated but result in grossly nonphysical verticalelectron density structures within the EDAM grid. As with thebad data case, the resulting poor results can take a significanttime to decay out of the grids and lead to clumping of theoutliers. Figure 8 shows the corresponding EDAM andARTIST values of hmF2 for Hermanus, illustrating the outlierEDAM values.[46] It is not yet clear why such nonphysical results occa-
sionally arise in EDAM from the assimilation of good data.It should be relatively straightforward to detect and exclude
0 4 8 12 16 20 24
Universal Time
5
10
15
20
25
30
35
Sig
ma
of D
iffer
ence
in h
mF
2 [k
m]
HE13N-GR13LLV12P-GR13LMU12K-GR13L
Figure 7. Diurnal variation of the standard deviation of thedifferences between the observed Grahamstown and ground-truth values of hmF2 at the other two digisonde sites.
MCNAMARA ET AL.: UPDATING IONOSPHERIC PROFILES
148
these poor assimilations, but the necessary procedures havenot yet been implemented. Consequently, for the purposeof the current testing, the hmF2 results have been filtered.After several attempts, we chose to filter the model valuesof hmF2 by excluding values that lay outside the 2s errorsin the IRI values. Before any filtering was applied, the IRIerrors in hmF2 at Hermanus had an average error of6.63 km and a standard deviation of 20.41 km. We havetherefore rejected all ionograms for which the EDAM errorin hmF2 laid outside the range of �34.2 and 47.7 km, butonly for the calculation of the RMS errors in hmF2 (thisfilter was thus also applied to the GPSII and IRI results;the same Hermanus-based filter was applied to the Louisvaleand Madimbo results).
7. Model and Observed Values of foF2
[47] foF2 is one of the most useful characteristics of aplasma frequency (or electron density) profile. It is the highestO-ray frequency that would be reflected at vertical incidence.Ionograms were recorded with a 15min cadence.
7.1. Observed Values of foF2
[48] Figure 9 shows the observed (autoscaled) values of foF2for Grahamstown, starting at 00 UT on day 244 (1 September).[49] The decreases in the daytime values of foF2 on days
253 and 270 are associated with increases in Ap on theprevious day(s). The red C indicates missing ionograms.There is a general increase of foF2 throughout the month,in line with the increase of the 10.7 cm radio flux from112 to 190 on 24 September (day 267), followed by adecrease back to 138.
7.2. RMS Errors in Model Values of foF2
[50] Figure 10 shows the diurnal variation of the RMSmodel errors in foF2 for GPSII, EDAM, and IRI. The IRIwas run for the full month with a fixed value of IG12 = 67.4
to provide a reference level for figures such as this. The IRIerrors were positive for the first half of the month, afterwhich they became negative. The GPSII and EDAM speci-fications of the profiles and peak parameters do not retainany memory of the value of IG12 used to provide the back-ground ionosphere used for the assimilation process.[51] Figure 10 shows that the GPSII and EDAM errors are
generally less than 0.5MHz. They are larger during the daywhen the values of foF2 are higher, and significantly lessthan the IRI error except around dawn. Comparison of theGPSII and EDAM errors with the Hermanus minimum
200 250 300 350 400 450
ARTIST hmF2 [km]
200
250
300
350
400
450M
odel
hm
F2
[km
]
Figure 8. Corresponding EDAM and ARTIST values ofhmF2, Hermanus September 2011. The connecting linesindicate that the large EDAM errors occur in groups.
0 200 400 600 800
Hours since 00UT, 01 September [Day 244]
0
2
4
6
8
10
12
foF
2 [M
Hz]
Day 253 [Sep 10] Day 270 [Sep 27]
253
270
261
C
C
Figure 9. Observed (autoscaled) values of foF2 forGrahamstown, September 2011. The red C indicates missingionograms. foF2 was depressed on the three storm days 253,261, and 270.
0 4 8 12 16 20 24
Universal Time
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1.1
1.2
RM
S E
rror
in fo
F2
[MH
z]
GPSIIEDAMIRI
Figure 10. RMS errors in GPSII, EDAM, and IRI valuesof foF2 for Hermanus, September 2011.
MCNAMARA ET AL.: UPDATING IONOSPHERIC PROFILES
149
achievable errors given in Figure 6 shows that the model andestimated errors are very similar.[52] One of the objectives of the present study is to deter-
mine if either GPSII or EDAM is clearly better than the other.Table 1 lists the statistics of the GPSII, EDAM, and IRIaverage error, standard deviation, and RMS error in foF2 forHermanus for the three models. All valid 15min ionogramcases have been considered. The sample size was 2188. TheGPSII and EDAM errors in foF2 are very similar and are lessthan half that of the IRI.[53] It is of some interest to see how far GPSII and EDAM
can extend the effects of the Grahamstown observations.Table 2 lists the RMS errors in foF2 for the three modelsand four digisonde sites (again unfiltered). The “Separation”column gives the distance from Grahamstown (the assimila-tion station). The Grahamstown results show how closelyGPSII and EDAM fit the assimilation data, but this is actuallya matter of modeling preference. The ARTIST values of foF2(and hmF2) have finite errors.[54] It can be seen from Table 2 that EDAM has the
smallest RMS errors in foF2 at Hermanus and Louisvaleand (as expected) that the IRI errors are the largest. The GPSIIerrors are significantly smaller than the EDAM errors forMadimbo, which is 1287 km from the driver site, Grahams-town. Hermanus and Louisvale are only ~700 km fromGrahamstown, and GPSII and EDAM have similar accuracyfor these two sites. The increase of the errors with distancefrom Grahamstown is in keeping with the decrease in thecross-correlation coefficient found between the deviations offoF2 from the median values for all times: Hermanus, 0.83;Louisvale, 0.78; Madimbo, 0.69.[55] The reason for the Hermanus errors being smaller
than the Louisvale errors probably lies in the geometry ofthe digisonde network. As shown in Figure 1, Hermanus isbasically west of Grahamstown, while Louisvale is to thenortheast. East-west correlation coefficients for foF2 aregenerally higher than north-south correlation coefficients[McNamara, 2009; McNamara and Wilkinson, 2009].Madimbo is 1287 km NNW of Grahamstown, so high corre-lation coefficients would not be expected.
7.3. Assimilating Only One Data Type
[56] Both GPSII and EDAM have also been run usingeither the GPS TEC or the ionogram observations (profile
or trace) separately to see if the relative utilities of the twotypes of assimilation data can be determined. Overall, itappears that the Grahamstown ionograms are more usefulthan the GPS TEC observations.[57] When used alone, the GPS TEC observations are used
to provide the model values of foF2 indirectly by assumingsome F2 slab thickness model as discussed by McNamaraet al. [2011]. Since the TEC observations cannot contributeusefully to specify hmF2 (except for very dense GPSnetworks), both GPSII and EDAM must rely on the IRIvalues of hmF2. In general, the relative utilities of the GPSTEC and ionogram observations will depend on the numbersand qualities of each data type. In this particular case, theGrahamstown ionogram data are abundant and reliable,whereas the GPS TEC data are rather sparse.
8. Model and Observed Values of hmF2
[58] The observed values of hmF2 show much morevariability than the values of foF2. This larger variability isat least partly due to inaccuracies in the ionogram autoscalingand in the derivation of the plasma frequency profile from thescaled ionogram trace. However, periodic nighttime variationsof hmF2 with amplitudes up to ~30 km and a period of ~3 hwere observed at all four digisonde sites on days 267 and268, so some of the variability is real.
8.1. Observed Values of hmF2
[59] There are several ways to illustrate the variation of theobserved values of hmF2. Figure 11 shows the diurnalvariation of the individual observed values of hmF2 atHermanus, September 2011. It illustrates the variability ofthe observed (ARTIST) values of hmF2 at Hermanus. Thehigh values of hmF2 for the night of 269/270 were associatedwith a magnetic stormwith an Ap of 67 andDst of�103. BothGPSII and EDAM follow this storm-time variation of hmF2
Table 1. Hermanus Average, Standard Deviation, and RMS Errorsin foF2 for GPSII, EDAM, and IRI
Model Average Sigma RMS
GPSII 0.00 0.36 0.36EDAM �0.14 0.28 0.31IRI �0.05 0.81 0.82
Table 2. RMS Errors in foF2 for GPSII, EDAM, and IRI, for theFour Digisonde Sites
Digisonde Separation GPSII EDAM IRI
Grahamstown 0.0 0.06 0.18 0.83Hermanus 685 0.36 0.31 0.82Louisvale 735 0.48 0.46 0.90Madimbo 1287 0.58 0.92 1.05
0 4 8 12 16 20 24
Universal Time
200
250
300
350
400
450
500
hmF
2 [k
m]
Day 269Ap = 67
Day 270
00UT not plotted
Figure 11. Diurnal variation of the observed values ofhmF2 at Hermanus, September 2011. The values of hmF2were atypically high on the disturbed night of 269/270.
MCNAMARA ET AL.: UPDATING IONOSPHERIC PROFILES
150
very well. The values of hmF2 that lie below ~220 km areprobably due to autoscaling blunders. As mentioned earlier,such blunders are most common at Louisvale during themiddle of the day.
8.2. RMS Errors in Model Values of hmF2
[60] The EDAM values of hmF2 were compared with theobserved values in Figure 8.[61] Figure 12 shows the diurnal variation of the RMS
model errors in hmF2 for GPSII, EDAM, and IRI. The errorshave been filtered using the IRI statistics to overcome thesometimes dramatic effects of autoscaling blunders andlarge outliers for EDAM. If an ionogram case had an outlierEDAM error based on this filtering, the GPSII and IRI caseswere also filtered out along with the EDAM case. The IRIabsolute errors were generally positive throughout themonth, following the increase of the 10.7 cm solar flux.[62] Figure 12 shows that the GPSII errors during the day
are less than the EDAM and IRI values, which are verysimilar. The estimated minimum achievable errors given inFigure 7 reach ~20 km in the middle of the day. The GPSIIand EDAM RMS errors have local minima of ~10 km near06 and 16 UT, which are consistent with the estimated achiev-able errors. The errors in the model values of hmF2 at night aresignificantly larger than the minimum achievable errors.[63] The GPSII errors tend to be the smallest, with the
EDAM errors being closer to the IRI errors. However, acomparison of the GPSII and EDAM errors in hmF2 isbiased toward GPSII because GPSII assimilates the ARTISTprofiles, and we are using ARTIST profiles as the groundtruth. GPSII adjusts the model to achieve a reasonable agree-ment with the ARTIST profile (including its hmF2 value).On the other hand, EDAM assimilates the ionogram traceand basically assumes that the IRI value of hmF2 is correct,which is why the EDAM and IRI values are so similar.
[64] As with foF2, it is of some interest to see how far themodels can extend the effects of the Grahamstown observa-tions of hmF2. Table 3 lists the RMS errors in hmF2 for thefour digisonde sites. In view of the larger errors at night,only the daytime data between 08 and 15 UT are compared(the hmF2 values have been filtered, but the effects of thefiltering are marginal during the day). As with the foF2 errors,the GPSII errors in hmF2 increase with increasing distancefrom Grahamstown. The GPSII errors are systematically lessthan the IRI errors, but the EDAM errors do not show anyimprovement over the IRI. The systematic increase of theIRI errors seems to be fortuitous.
9. The Profile Altitude Corresponding to 0.8 foF2
[65] The autoscaled values of hmF2 are subject to thevagaries of TIDs as well as the uncertainties of ionogramautoscaling and the conversion from the ionogram to theplasma frequency profile. Thus, it is not usually possible toquantify the level of uncertainty that exists in the ARTISTvalues of hmF2. We have therefore included the profilealtitude at a plasma frequency of 0.8 foF2 (0.64 NmF2) asanother validation parameter. The more common figure of0.5 NmF2 was avoided so as to restrain the altitude to theF2 layer. For simplicity, we call this altitude h0.8.[66] The altitude h0.8 is expected a priori to suffer from
less natural and processing noise than hmF2. It also hasthe advantage that it is a point on the F2 profile that is morerelevant than hmF2 to HF radio propagation on obliquecircuits. However, as with hmF2 the EDAM results includeoutliers that distort the RMS errors. Overall, both GPSIIand EDAM provide accurate values of h0.8, as illustratedin Figures 13 and 14, and Table 4.[67] Comparison of Figures 13 and 14 shows that GPSII
has fewer outliers than EDAM and that both sets of modelvalues are biased a little high.[68] Table 4 lists the average and standard deviation of the
GPSII, EDAM, and IRI errors in the values of foF2, hmF2(filtered), and h(0.8 foF2) for Hermanus. The table showsthat the GPSII and EDAM h(0.8foF2) errors are smaller thanthose for hmF2. The models offer significant improvementover the IRI for foF2 and h(0.8foF2), but not for hmF2.The calculations confirm the slight bias in the model valuesof h(0.8 foF2) that appears in the figures. The EDAM errorsin h(0.8 foF2) are somewhat smaller than the GPSII errors,in spite of the large outliers seen in Figure 14.
10. Model and Observed Values of 1F MOF
[69] The values of 1F MOF for the Hermanus, Louisvale,and Madimbo to Grahamstown circuits were manuallyscaled for days 251–273 inclusive, except that there wereno ionograms for Grahamstown on day 262 (19 September)
0 4 8 12 16 20 24
Universal Time
5
10
15
20
25
30
35
RM
S E
rror
[km
]
GPSIIEDAMIRI
Figure 12. Filtered RMS errors in model values of hmF2for Hermanus, September 2011. The filtering was imposedto remove some large EDAM outlier errors that remain tobe explained.
Table 3. RMS Errors in hmF2 for GPSII, EDAM, and IRI, for theFour Digisonde Sites, 08-15 UT Only (Daytime)
Digisonde Separation GPSII EDAM IRI
Grahamstown 0.0 2.1 15.3 17.5Hermanus 685 13.5 17.5 17.7Louisvale 735 17.9 24.9 22.6Madimbo 1287 20.1 23.9 23.2
MCNAMARA ET AL.: UPDATING IONOSPHERIC PROFILES
151
and parts of other days. These days covered all of themagnetically disturbed intervals, with the global Ap havingvalues of 30 or higher on days 9, 10, 17, 26, and 27. Therewere 14 days with Ap less than 10. An example of obliqueionograms was given in Figure 2.[70] Examples of the diurnal variation of the observed
values of the 1F MOF, along with the TID effects, weregiven earlier in Figures 3 and 4. The GPSII and EDAMvalues of the 1F MOF were calculated by the respectivemodelers using three-dimensional ray tracing and homingwith the Earth’s magnetic field included. Since neithermodel allows for TIDs, the model values of the MOF willhave a minimum uncertainty.[71] Figure 15 shows the corresponding observed values
of the Hermanus and Louisvale to Grahamstown 1F MOF,for both quiet and disturbed days. Figure 15 shows firstlythat the Louisvale MOF is greater than the HermanusMOF. This is consistent with the Louisvale circuit beingthe longer of the two (735 km versus 684 km) and the reflec-tion point being further north than that for the Hermanuscircuit (Figure 1). The scatter of the points can be attrib-uted to the TIDs being different at the two reflection“points.” The scatter is larger during the day, when theMOFs are higher.[72] Figure 16 is the GPSII equivalent of Figure 15. Com-
paring Figure 16 with the observations in Figure 15 showsthat the model values of the Hermanus and Louisvale MOFsare almost identical on average, whereas the Louisvaleobservations were slightly higher than the Hermanus obser-vations. The GPSII results also show less scatter, which isbecause GPSII is not accounting for the TIDs. Some of thescatter would be due to the TID effects on the assimilated
100 150 200 250 300 350 400
ARTIST-Observed h(0.8foF2) [km]
100
150
200
250
300
350
400
Mod
el h
(0.8
foF
2) [k
m]
Figure 13. GPSII versus ARTIST values of h0.8 forHermanus, September 2011. There are fewer outlier GPSIIpoints than with EDAM (Figure 14).
100 150 200 250 300 350 400
ARTIST-Observed h(0.8foF2) [km]
100
150
200
250
300
350
400
Mod
el h
(0.8
foF
2) [k
m]
Figure 14. EDAM versus ARTIST values of h0.8 forHermanus, September 2011. It is the occasional outliers thatlead to large RMS errors in the EDAM value of h0.8.
Table 4. Average and Standard Deviation of the GPSII, EDAM,and IRI Errors in foF2, hmF2, and h(0.8) for Hermanus
GPSII EDAM IRI
Average Sigma Average Sigma Average Sigma
foF2 0.01 0.36 �0.14 0.28 �0.02 0.82hmF2a 6.62 16.59 8.79 17.37 6.60 19.87h08 5.21 11.15 2.47 10.35 �0.49 16.80
aThe values of hmF2 have not been filtered; this was done only for theRMS calculations.
2 4 6 8 10 12 14
Hermanus-Grahamstown MOF [MHz]
2
4
6
8
10
12
14
Loui
sval
e-G
raha
mst
own
MO
F [M
Hz]
Figure 15. Observed values of the Hermanus andLouisvale to Grahamstown 1F MOF, September 2011. Sincethe MOFs were hand-scaled, the scatter is attributed to theTIDs being different at the reflection “points” for the two cir-cuits. The Louisvale MOFs are generally higher than theHermanus MOFs because they correspond to a longercircuit.
MCNAMARA ET AL.: UPDATING IONOSPHERIC PROFILES
152
Grahamstown profiles/traces. The same situation holds forthe EDAM results, except for about 20 to 30 outlier points.[73] Figures 17 and 18 compare the errors in the GPSII
and EDAM calculated values of the 1F MOF for theHermanus to Grahamstown circuit. They show that theGPSII errors in the calculated 1F MOF are usually smallerthan the EDAM errors, with a smaller diurnal scatter. Thestandard deviations are largest between about 07 and 13UT (about 08–12 LT). The largest EDAM errors are
associated with ~20 outlier points. Taking the daytimevalues of the 1F MOF to be 11MHz (Figure 4) gives adaytime percentage error of about 0.5/11 or ~5% for theGPSII results, and a little higher for the EDAM results.The bulk of this error is probably due to the TIDs.[74] Table 5 lists the error statistics for the model values of
the 1FMOF for the Hermanus and Louisvale to Grahamstowncircuits, accumulated over all days and times, summarizing theresults plotted in Figures 17 and 18. The sample size for theMadimbo circuit was too small to warrant detailed analysis.The GPSII results are consistently more accurate than theEDAM results.[75] Comparing the Hermanus and Louisvale entries shows
that the errors are consistently higher for the Louisvale circuit.This agrees with the comparisons of the foF2 and hmF2 errorsgiven in and Tables 2 and 3.[76] Splitting the days up into quiet (Ap< 30) and disturbed
days shows that the errors in the model values of the 1F MOFare smaller for the quiet days. This is as expected since stormeffects would not usually be coherent over the area covered bythe digisondes.[77] The 1F MOFs corresponding to the IRI plasma
frequency specifications are not of great interest because theIRI is a very simplified monthly median model, whereas wehave seen that there are significant changes in the observed
2 4 6 8 10 12 14
Hermanus-Grahamstown MOF [MHz]
2
4
6
8
10
12
14
Loui
sval
e-G
raha
mst
own
MO
F [M
Hz]
Figure 16. GPSII values of the Hermanus and Louisvale toGrahamstown 1F MOF, September 2011. The MOFs for thetwo circuits are closer to each other than found in practice(Figure 15). The spread about the red y= x line probablyoriginates from TIDs at the driver station (Grahamstown).
0 4 8 12 16 20 24
Universal Time
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
Ave
rage
Err
or in
Mod
el 1
F M
OF
[MH
z]
GPSIIEDAM
Figure 17. Diurnal variation of the average errors inthe GPSII and EDAM calculated values of the 1F MOF,Hermanus-Grahamstown, September 2011.
0 4 8 12 16 20 24
Universal Time
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1.1
Sta
ndar
d D
evia
tion
of E
rror
in M
odel
1F
MO
F [M
Hz] GPSII
EDAM
Figure 18. Diurnal variation of the standard deviation ofthe errors in the GPSII and EDAM calculated values of the1F MOF, Hermanus-Grahamstown, September 2011. Thelarge errors for EDAM at ~10 UT (near midday) are due toa few large outlier errors that have yet to be explained.
Table 5. Error Statistics for the Model Values of the 1F MOF,All Data
Case Count Average Sigma RMS Correlation
Hermanus/GPSII 1650 �0.112 0.417 0.432 0.992Hermanus/EDAM 1650 �0.319 0.568 0.651 0.985Louisvale/GPSII 1659 �0.313 0.506 0.595 0.988Louisvale/EDAM 1659 �0.398 0.636 0.750 0.981
MCNAMARA ET AL.: UPDATING IONOSPHERIC PROFILES
153
MOFs due to TIDs. Consequently, we have calculated the IRIMOFs only for 1 day, 16 September (day 259). These MOFsconfirm the inability of the IRI to track the 1 to 2MHz effectson the MOFs of TIDs. In contrast, the GPSII and EDAMMOFs do follow the largest TID fluctuations, although notthe smaller ones. The day-to-day diurnal variability of theIRI is also more limited than that of the observations, but wehave not considered the effects of this variability on the IRIMOFs. Settimi et al. [2013] found that their ionospheric spec-ification gave more accurate oblique ionograms than the IRI.
11. Errors in the Plasma Frequency Profiles
[78] Before discussing the errors in the plasma frequencyprofiles, we illustrate some typical profiles. Figure 19 showsthe observed Hermanus plasma frequency profiles for 1045UT (about noon). Incidentally, the ray tracing resultsshowed that at 1045 UT the ray apogee at the MOF rangedfrom 230 to 280 km over the month, so this is the part ofthe profiles that is tested in the 1F MOF validations. Therays would reach hmF2 only at vertical incidence.[79] The model plasma frequency profiles can differ from
the ground-truth profiles because of differences in foF2,hmF2, or the basic shape of the profile. Any comparison ofsets of profiles will average over the effects of each of thesedifferences. In this section, we first illustrate the errorprofiles for all times of the day as a possible way of comparingthe accuracies of the models. We then consider the averageerrors for two fixed times of the day in order to illustrate thevariability of the profile errors.
11.1. Profile Errors for All Times
[80] Figure 20 shows the average plasma frequency errorsin the GPSII, EDAM, and IRI profiles for all days and timesat Hermanus. Because the profiles are averaged over alltimes, the diurnal variation of the errors has been lost, but
the figures provide a simple indicator of the relative meritsof the three models.[81] The IRI profiles rarely extended above 300 km, so the
IRI curve suffers from low counts above that altitude.Most of the observed values of hmF2 lay below ~330 km(Figure 11), although they exceeded 375 km for the disturbednight of 269/270. It can be seen that the GPSII errors have asmall negative bias at most altitudes, while the EDAM errorschange sign at ~220 km. All three errors are very similar at~250 km, which may or may not be fortuitous.[82] The average errors do not tell the whole story, since
there is wide variability of the profiles at F2 layer altitudes.Figure 21 shows by way of example the average and averagestandard deviation of the plasma frequency errors in theGPSII profiles for all days and times at Hermanus. TheGPSII and EDAM errors at Hermanus are systematicallysmaller than the IRI errors. The 2s width (the separation ofthe outside curves in Figure 21) of the profile errors inMHz for Hermanus at some representative altitudes aregiven in Table 6. The GPSII and EDAM errors for the Fregion are thus about 0+/�0.5MHz, with only a small meanerror. The errors are larger for all models at Louisvale andMadimbo, with the GPSII and EDAM average errors againbeing systematically smaller than the IRI errors, and theGPSII errors being less than the EDAM errors. The 2s widthprofile errors in MHz for Louisvale are given in Table 7.[83] Tables 6 and 7 show that the IRI profile errors are
very similar for Hermanus and Louisvale, as expected. Forthe 2s errors at 250, 275, and 300 km, the GPSII errorsincreased for Louisvale by a factor of ~18% (average overthe three altitudes), while the EDAM errors increased by afactor of ~66%. Thus, GPSII seems to have been able toextend the influence of the Grahamstown ionograms to
1 2 3 4 5 6 7 8 9 10 11
Plasma Frequency [MHz]
75
100
125
150
175
200
225
250
275
300
325
Alti
tude
[km
]
Figure 19. Observed plasma frequency profiles for 1045UT, Hermanus, September 2011. The profiles show littleday-to-day variability in the E and F1 regions.
-1 -0.75 -0.5 -0.25 0 0.25 0.5 0.75 1
Error in MHz
50
100
150
200
250
300
350
Alti
tude
[km
]
IRIGPSIIEDAM
Figure 20. Average plasma frequency errors in the GPSII,EDAM, and IRI profiles for all days and times, Hermanus,September 2011. The IRI profiles did not usually extendabove 300 km, so the IRI curve is unreliable. The IRI errorsat hmaxE (~110 km) are small because ARTIST uses a similarempirical model of foE in its fit to the E layer virtual trace.
MCNAMARA ET AL.: UPDATING IONOSPHERIC PROFILES
154
Louisvale better than EDAM did (this difference should be in-vestigated further; it may be due simply to the way the twomodels use the GPS TEC observations).[84] For Madimbo, the GPSII and EDAM errors were sim-
ilar to the IRI errors. Thus, the Grahamstown ionograms hadminimal utility for updating the Madimbo profiles 1287 kmaway. The increase in the errors from Hermanus to Louisvaleto Madimbo is consistent with the increase in the errors infoF2 that were listed in Table 2.
11.2. Profile Errors at Fixed Universal Times
[85] Accumulating the profile errors over all days andtimes tends to hide large outlier profiles. We have thereforeconsidered the profiles specifically for two fixed UTs.Figures 22 and 23 show the altitude dependence of the aver-age errors in the Hermanus plasma frequency for the GPSII,EDAM, and IRI profiles for noon (1045 UT) and midnight(2245 UT), accumulated over all days for September 2011.Figure 22 shows unexpectedly large errors for the IRIbetween ~150 and ~250 km that are significantly greater thanthe errors accumulated over all days and times (Figure 20).The GPSII and EDAM errors are much lower than the IRIerrors. The GPSII and EDAM errors at midnight (Figure 23)are about twice those shown in Figure 20.
12. Conclusions
[86] This section summarizes the important results givenin previous sections.
12.1. Overview
[87] The main thrusts of this paper were to:
• Determine the extent to which digisonde observations canbe used to update the plasma frequency profiles at otherlocations in the same region
• Compare the relative merits of two real-time assimilationmodels, GPSII and EDAM,which take different approachesto assimilate the digisonde observations.
[88] The region considered was the midlatitude Republic ofSouth Africa (RSA), since it has four digisondes, at Grahams-town, Hermanus, Louisvale, and Madimbo. The analysis wasfor a month of convenience, September 2011, which turnedout to have a wide range of solar variability. The digisondeobservations at Grahamstown were assimilated by the two
-1 -0.5 0 0.5 1
Error in MHz
50
100
150
200
250
300
350A
ltitu
de [k
m]
AverageAverage - SigmaAverage + Sigma
Figure 21. Plasma frequency errors (average, and averagestandard deviation) in the GPSII profiles for all days andtimes, Hermanus, September 2011. These curves representaverages over ~2000 profiles.
Table 6. The 2s Widths at Hermanus for the IRI, GPSII, andEDAM Models
Altitude GPSII EDAM IRI
150 0.38 0.62 1.05200 0.81 1.22 1.84225 1.04 1.13 1.38250 1.02 0.90 1.44275 0.82 0.79 1.33300 0.65 0.82350 0.57 1.08
Table 7. The 2s Widths at Louisvale for the IRI, GPSII, andEDAM Models
Altitude GPSII EDAM IRI
150 0.39 0.81 0.93200 0.89 1.86 1.91225 1.23 1.81 1.46250 1.24 1.55 1.43275 0.98 1.28 1.36300 0.72 1.34350 0.66 1.46
-2 -1.5 -1 -0.5 0 0.5 1
Plasma Frequency Error [MHz]
50
100
150
200
250
300
350
Alti
tude
[km
]
IRIGPSIIEDAM
Figure 22. Average plasma frequency errors in the GPSII,EDAM, and IRI profiles for noon (1045 UT), Hermanus,September 2011.
MCNAMARA ET AL.: UPDATING IONOSPHERIC PROFILES
155
models, with the other three digisondes providing the groundtruth. GPSII assimilated the digisonde plasma frequencyprofiles, while EDAM assimilated the ionogram trace.[89] The autoscaled data were found to be sufficiently
accurate for the study, although some ionograms with themore egregious errors (blunders) were excluded from theanalysis. Cases for which there was no Grahamstownionogram (or the autoscaling was classified as unreliable) werealso excluded. Thus, we have not tested in detail the ability ofthe models to cope with missing assimilation data.
12.2. Accuracy of Model Values of foF2
[90] Overall, EDAM was slightly better than GPSII atspecifying the values of foF2 at Hermanus and Louisvale.However, GPSII did better at the most remote site, Madimbo.The RMS errors for the models at Hermanus (our mainground-truth site, 735 km east of Grahamstown) lay between0.2MHz (night) and 0.5MHz (day). For typical daytimefoF2 values of ~9MHz, a 0.5MHz error corresponds to~6%. These errors are consistent with the minimum expectederrors, which are apparently set by the TIDs being different atGrahamstown and the ground-truth sites. The errors increasedwith increasing distance from Grahamstown.
12.3. Accuracy of Model Values of hmF2
[91] EDAM encountered difficulties with some cases thatled to large errors in hmF2 even when good data had beenassimilated. This resulted in nonphysical electron densityprofiles, mostly at night. The cause of these assimilationerrors has not yet been determined, and the outlier valueshave simply been excluded for the present analysis. In thefuture, it should be relatively straightforward to detect andexclude these poor assimilations, but the required procedureshave not yet been implemented.
[92] The GPSII errors in hmF2 during the day were lessthan the EDAM and IRI values, which are very similar.The estimated minimum achievable errors reached ~20 kmin the middle of the day. The GPSII and EDAM RMS errorshave local minima of ~10 km near 06 and 16 UT, which areconsistent with the estimated achievable errors. As withthe foF2 errors, the GPSII errors in hmF2 increase withincreasing distance from Grahamstown. The GPSII errorsare systematically less than the IRI errors, but the EDAMerrors do not provide any general improvement.
12.4. Accuracy of Model Values of h(0.8 foF2)
[93] Because hmF2 is a noisy parameter that seems to bevery sensitive to autoscaling and profile derivation issues,as well as to TIDs, we have included the profile altitude ata plasma frequency of 0.8 foF2 (0.64 NmF2) as anothervalidation parameter. It was found that GPSII and EDAMoffer significant improvement over the IRI for h(0.8 foF2)(as well as for foF2, but not for hmF2). The EDAM errorsin h(0.8 foF2) are somewhat smaller than the GPSII errors,in spite of some large outliers.
12.5. Accuracy of Model Values of 1F MOF
[94] We chose to use the 1F MOF as a validation parameterbecause it is so closely related to the main subpeak applica-tions of the ionosphere. The MOF for the oblique ionogramsis not currently scaled automatically by ARTIST, so theoblique ionograms were displayed using SAO Explorer andscaled manually for the Hermanus, Louisvale, and Madimboto Grahamstown circuits.[95] The GPSII errors in the calculated 1F MOF were
usually found to be smaller than the EDAM errors. Thestandard deviations of the errors were largest between about07 and 13 UT (about 08–12 LT). Taking the daytime valuesof the 1F MOF to be 11MHz gives a daytime percentageerror of about 0.5/11 or ~5% for the GPSII results, and alittle higher for the EDAM results. The bulk of this error isprobably due to the TIDs, for which neither GPSII norEDAM accounts successfully.
12.6. Accuracy of the Model Plasma Frequency Profiles
[96] Given that there are nominally 30� 96� 4 profilesfor each of the three ground-truth digisondes, we havereduced the validations to accumulations of the errors inthe profiles (1) over all acceptable ionograms and (2) overall 30 (nominally) ionograms at two fixed times of the day(essentially noon and midnight).[97] For the accumulation over all profiles, it was found
that the GPSII and EDAM plasma frequency errors for theF region at Hermanus were about 0� 0.5MHz, with only asmall mean error. The IRI errors were consistently larger.[98] Profiles at noon and midnight were used to provide a
reminder that the errors accumulated over all days and timescan mask some quite large individual errors. In fact, it isquite difficult to gain a coherent impression of the accuracyof the profiles—it is much easier to characterize the errors inoperationally important parameters such as the 1F MOF thatare based on the profiles.
[99] Acknowledgments. EDAM was developed as part of the UKMinistry of Defence ISTAR and Sensors Domain Research Programme.The development of GPSII is currently funded in part by AFRL undercontract FA9453-11-C-0157.
-2 -1.5 -1 -0.5 0 0.5 1
Plasma Frequency Error [MHz]
50
100
150
200
250
300
350A
ltitu
de [k
m]
IRIGPSIIEDAM
Figure 23. Average plasma frequency errors in the GPSII,EDAM, and IRI profiles for midnight (2245 UT), Hermanus,September 2011. The GPSII and EDAM errors are abouttwice those obtained by accumulating over all UTs.
MCNAMARA ET AL.: UPDATING IONOSPHERIC PROFILES
156
[100] The following web sites were accessed:
Solar Data: http://www.swpc.noaa.gov/ftpdir/indices/old_indices/2011Q3_DSD.txt
Geomagnetic Data: http://www.swpc.noaa.gov/ftpdir/indices/old_indices/2011Q3_DGD.txt
Dst Indices: http://wdc.kugi.kyoto-u.ac.jp/dst_realtime/201109/index.html
SAO Explorer: http://ulcar.uml.edu/SAO-X/SAO-X.html. SAO Explorer is used todownload the digisonde ionogramfiles (GRM) and files of scaled data(SAO) from DIDBase, as well as todisplay the ionograms for scaling the1F MOF.
Online Ionograms: http://umlcar.uml.edu/DIDBase/
ReferencesAmabayo, E. B., L.-A. McKinnell, and P. J. Cilliers (2011), Statistical charac-terisation of spread F over South Africa, Adv. Space Res. 48, 2043-2052.
Angling, M. J. (2008), First assimilations of COSMIC radio occultation datainto the Electron Density Assimilative Model (EDAM), Ann. Geophys.,26(2), 353-359.
Angling, M. J., and P. S. Cannon (2004), Assimilation of radio occultationmeasurements into background ionospheric models, Radio Sci., 39,RS1S08, doi:10.1029/2002RS002819.
Angling, M. J., and N. K. Jackson-Booth (2011), A short note on theassimilation of collocated and concurrent GPS and ionosonde data intothe Electron Density Assimilative Model, Radio Sci., 46, RS0D13,doi:10.1029/2010RS004566.
Angling, M. J., and B. Khattatov (2006), Comparative study of two assimilativemodels of the ionosphere, Radio Sci., 41, RS5S20, doi:10.1029/2005RS003372.
Angling, M. J., J. Shaw, A. K. Shukla, and P. S. Cannon (2009), Develop-ment of an HF frequency selection tool based on the EDAM near real timeionosphere, Radio Sci., 44, RS0A13, doi:10.1029/2008RS004022.
Bilitza, D., and B. W. Reinisch (2008), International Reference Ionosphere2007: Improvements and new parameters, Adv. Space Res., 42, 4, 599-609doi:10.1016/ j.asr.2007.07.048.
Davies, K. (1990), Ionospheric Radio, Peter Peregrinus, London.Fridman, S. V., L. J. Nickisch, M. Aiello, and M. Hausman (2006), Real-timereconstruction of the three-dimensional ionosphere using data from a networkof GPS receivers, Radio Sci., 41, RS5S12, doi:10.1029/2005RS003341.
Fridman, S. V., L. J. Nickisch, and M. Hausman (2009), Personal-computer-based system for real-time reconstruction of the three-dimensionalionosphere using data from diverse sources, Radio Sci., 44, RS3008,doi:10.1029/2008RS004040.
Fridman, S. V., L. J. Nickisch, and M. Hausman (2012), Inversion ofbackscatter ionograms and TEC data for over-the-horizon radar, RadioSci., 47, RS0L10, doi:10.1029/2011RS004932.
Galkin, I. A., G. M. Khmyrov, A. V. Kozlov, B. W. Reinisch, X. Huang,and V. Paznukhov (2007), The ARTIST-5, in Radio Sounding andPlasma Physics, AIP Conference Proceedings, edited by P. Song, Amer-ican Institute of Physics, Melville, NY.
Hunsucker, R. (1982), Atmospheric gravity waves generated in the high-latitude ionosphere: A review, Rev. Geophys., 20, 293-315.
McNamara, L. F. (2006), Quality figures and error bars for autoscaledDigisonde vertical incidence ionograms, Radio Sci., 41, RS4011,doi:10.1029/2005RS003440.
McNamara, L. F. (2009), Spatial correlations of foF2 deviations and theirimplications for global ionospheric models: 2. Digisondes in the UnitedStates, Europe, and South Africa, Radio Sci., 44, RS2017, doi:10.1029/2008RS003956.
McNamara, L. F., and P. J. Wilkinson (2009), Spatial correlations of foF2deviations and their implications for global ionospheric models: 1.Ionosondes in Australia and Papua New Guinea, Radio Sci., 44, RS2016,doi:10.1029/2008RS003955.
McNamara, L. F., G. J. Bishop, and J. A. Welsh (2011), Assimilation ofionosonde profiles into a global ionospheric model, Radio Sci., 46,RS2006, doi:10.1029/ 2010RS004457.
Menke, W. (1989), Geophysical Data Analysis: Discrete Inverse Theory,Academic Press, San Diego.
Pezzopane, M., M. Pietrella, A. Pignatelli, B. Zolesi, and L. R. Cander(2011), Assimilation of autoscaled data and regional and local iono-spheric models as input sources for real-time 3D InternationalReference Ionosphere modeling, Radio Sci., 46, RS5009, doi:10.1029/2011RS004697.
Reinisch, B. W., and X. Huang (1983), Automatic calculation of electrondensity profiles from digital ionograms: 3. Processing of bottomsideionograms, Radio Sci., 18(4), 477–492.
Reinisch, B. W., et al. (2009), The new Digisonde for research and monitor-ing applications, Radio Sci., 44, RS0A24, doi:10.1029/2008RS004115.
Settimi, A., M. Pezzopane, B. Zolesi, M. Pietrella, C. Bianchi, C. Scotto,E. Zuccheretti, and J. Makris (2013), Testing the IONORT-ISP system:A comparison between synthesized and measured oblique ionograms,Radio Sci., submitted.
Tikhonov, A. N., and V. Y. Arsenin (1977), Solution of Ill-Posed Problems,Halsted, New York.
Titheridge, J. (1988), The real height analysis of ionograms: A generalizedformulation, Radio Sci., 23(5), 831-849.
Twomey, S. (1977), Introduction to the Mathematics of Inversion in RemoteSensing and Indirect Measurements, Dover Publications, New York.
MCNAMARA ET AL.: UPDATING IONOSPHERIC PROFILES
157
