7
IEEE TRANSACTIONS ON BROADCASTING, VOL. BC-31, NO. 4, DECEMBER 1985 NON-INTRUSIVE TESTING USING PROGRAMME STATISTICS Stefan Mare : Research Department : SABC Broadcasting Centre. P.O. Box 4559 Johannesburg 2000 South Africa ABSTRACT Statistical distribution curves are introduced as a means for monitoring the performance of sound broad- cast circuits in situations where removal of cir- cuits from service requires an interruption in transmission. Gradual drift of circuit parameters can be monitored in addition to performance tests on circuits or equipment under operational condi- tions. The Spectral Amplitude Distribution (SAD) can be used to characterise non-linear equipment like limiters or compressors or any other equip- ment in the transmission circuit. The Time Ampli- tude Distribution (TAD) is not a sensitive indica- tor of changes in signal parameters. Quality of the broadcast programme is reflected in certain para- meters of the distribution curves and these may be developed to give an objective measure of quality. The results of measurements on 'live' broadcasts are presented. INTRODUCTION The operation of a broadcasting network can be main- tained within the limits set by-the operating agency through periodic tests of the network using deter- ministic test signal suited to the application of any circuit. Such a network is not time invariant: the equipment drifts and operator accessible para- meters such as microphone gain are frequently va- ried. It is then necessary to establish suitable intervals for conducting tests on the network. While test signals can establish the operation under con- trolled conditions the real signals representing the flow of information across a circuit, may be very different and can only be described statistically. Attempts have been made to model these real signals. For broadcasting this has been done by Ehara 1,2,3 A program-model signal is generated which closely approximates the statistical distribution of the programme material actually broadcast. Many broadcasting organisations operate 24 hours a day and cannot test the alignment of equipment during programme time. Some equipment operation is depen- dent on programme content, e.g. limiters and complex test signals like the programme-model signal, are required to establish whether such equipment is functioning correctly. These problems can be over- come by in-service monitoring or non-intrusive tes- ting using the programme signal as a test signal. This paper introduces the use of statistical distri- bution curves as a method for in-service monitoring of broadcast equipment functioning. The use of similar techniques for the alignment of non-linear equipment is being investigated at present. Non-intrusive testing as applied here cannot quan- tify parameters such as harmonic distortion but the frequency response,dynamic range and amount of com- pression introduced can be compared to a subjective- ly acceptable standard and deviations from this are easily observed. THE BASIS OF NON-INTRUSIVE TESTING The basis of non-intrusive testing is the use of the programme signal as the stimulus exciting the pro- gramme circuit. Due to the time varying nature of the audio signal in a broadcasting system it is not possible to infer much about the signal quality by displaying its waveform on an oscilloscope or its spectrum on a spectrum analyser. Non-linear distortion measurements duirng programme time are meaningless even though experienced observers can detect very low levels of distortion and changes in dynamic range by listening to the programme. In a statistical sense the audio signal is quasi- stationary [4]. The amplitude distribution measured at time T will be similar to the distribution mea- sured at time T+t provided that the time taken to collect data for calculation of the distribution is long enough. The signal long term statistics can be used as a semi-invariant characteristic of the signal and any changes in the measured distri- butions attributed to the equipment through which the signal is transmitted. These changes can be related to circuit parameters such as frequency response and dynamic range. The choice of sampling duration depends on the dis- tribution being determined and on the variation in programme content. The sample statistics must be a good approximation of the long term programme material statistics. The amplitude distribution of an audio signal can be determined in a relatively short period of time but it does not contain much information. Variations in dynamic range are re- flected as changes in the distribution but the sen- sitivity to changes is low and the results are not readily interpreted. The amplitude distribution contains no spectral information. The spectral ampl itude distribution (SAD) contains statistical information in both the time and fre- quency domain. Ehara [1] measured the SAD for re- corded music using 1/6th octave filters. From his results it is clear that a single symphony is not a good approximation to the statistics of normal programme material. Even between different sym- phonies the variation in SAD is so large that the SAD of one symphony is not representative of the SAD of all symphonies. More generally the same can be said for any programme excerpt when compared to another excerpt containing different voices, music etc. (Excerpt is used in the sense of a short ex- tract from a programme). A representative sample of the programme content must be obtained to get a distribution from which deductions about the opera- tion of the system can be made. The variation in programme content during a day is large enough for the complete frequency and ampli- tude range of normal programme material to be in- cluded. The quasi-stationary nature of a day long sample can be seen in Figure 1 where the SAD for a particular programme as measured on 10 consecutive days is plotted on one graph. The variation between days is less than +/- ldB when the curves are smoothed (geometric smoothing with smoothing con- stant of 0.25 is used [5]). APPLICATION OF NON-INTRUSIVE TESTING IN PRACTICE When a broadcaster receives complaints from the lis- tening public regarding the quality of the broad- casts, impairments like poor signal to noise ratio can be readily identified but dissatisfaction with the dynamic range is not that easily measured. Subjective assessment has to be used and it takes time and effort to establish that a problem does in fact exist. Using non-intrusive testing the sig- nal can be monitored at the origin,as it is finally 0018-9136/85/1200-0088$01.00O1986 IEEE 88

Non-Intrusive Testing Using Programme Statistics

  • Upload
    s

  • View
    216

  • Download
    3

Embed Size (px)

Citation preview

IEEE TRANSACTIONS ON BROADCASTING, VOL. BC-31, NO. 4, DECEMBER 1985

NON-INTRUSIVE TESTING USING PROGRAMME STATISTICSStefan Mare : Research Department : SABC

Broadcasting Centre. P.O. Box 4559Johannesburg 2000 South Africa

ABSTRACTStatistical distribution curves are introduced as ameans for monitoring the performance of sound broad-cast circuits in situations where removal of cir-cuits from service requires an interruption intransmission. Gradual drift of circuit parameterscan be monitored in addition to performance testson circuits or equipment under operational condi-tions. The Spectral Amplitude Distribution (SAD)can be used to characterise non-linear equipmentlike limiters or compressors or any other equip-ment in the transmission circuit. The Time Ampli-tude Distribution (TAD) is not a sensitive indica-tor of changes in signal parameters. Quality of thebroadcast programme is reflected in certain para-meters of the distribution curves and these may bedeveloped to give an objective measure of quality.The results of measurements on 'live' broadcastsare presented.

INTRODUCTIONThe operation of a broadcasting network can be main-tained within the limits set by-the operating agencythrough periodic tests of the network using deter-ministic test signal suited to the application ofany circuit. Such a network is not time invariant:the equipment drifts and operator accessible para-meters such as microphone gain are frequently va-ried. It is then necessary to establish suitableintervals for conducting tests on the network. Whiletest signals can establish the operation under con-trolled conditions the real signals representing theflow of information across a circuit, may be verydifferent and can only be described statistically.Attempts have been made to model these real signals.For broadcasting this has been done by Ehara 1,2,3A program-model signal is generated which closelyapproximates the statistical distribution of theprogramme material actually broadcast.Many broadcasting organisations operate 24 hours aday and cannot test the alignment of equipment duringprogramme time. Some equipment operation is depen-dent on programme content, e.g. limiters and complextest signals like the programme-model signal, arerequired to establish whether such equipment isfunctioning correctly. These problems can be over-come by in-service monitoring or non-intrusive tes-ting using the programme signal as a test signal.This paper introduces the use of statistical distri-bution curves as a method for in-service monitoringof broadcast equipment functioning. The use ofsimilar techniques for the alignment of non-linearequipment is being investigated at present.Non-intrusive testing as applied here cannot quan-tify parameters such as harmonic distortion but thefrequency response,dynamic range and amount of com-pression introduced can be compared to a subjective-ly acceptable standard and deviations from this areeasily observed.

THE BASIS OF NON-INTRUSIVE TESTINGThe basis of non-intrusive testing is the use of theprogramme signal as the stimulus exciting the pro-gramme circuit. Due to the time varying nature ofthe audio signal in a broadcasting system it is notpossible to infer much about the signal quality bydisplaying its waveform on an oscilloscope or itsspectrum on a spectrum analyser. Non-linear

distortion measurements duirng programme time aremeaningless even though experienced observers candetect very low levels of distortion and changesin dynamic range by listening to the programme.

In a statistical sense the audio signal is quasi-stationary [4]. The amplitude distribution measuredat time T will be similar to the distribution mea-sured at time T+t provided that the time taken tocollect data for calculation of the distributionis long enough. The signal long term statisticscan be used as a semi-invariant characteristic ofthe signal and any changes in the measured distri-butions attributed to the equipment through whichthe signal is transmitted. These changes can berelated to circuit parameters such as frequencyresponse and dynamic range.The choice of sampling duration depends on the dis-tribution being determined and on the variation inprogramme content. The sample statistics must bea good approximation of the long term programmematerial statistics. The amplitude distribution ofan audio signal can be determined in a relativelyshort period of time but it does not contain muchinformation. Variations in dynamic range are re-flected as changes in the distribution but the sen-sitivity to changes is low and the results are notreadily interpreted. The amplitude distributioncontains no spectral information.The spectral ampl itude distribution (SAD) containsstatistical information in both the time and fre-quency domain. Ehara [1] measured the SAD for re-corded music using 1/6th octave filters. From hisresults it is clear that a single symphony is nota good approximation to the statistics of normalprogramme material. Even between different sym-phonies the variation in SAD is so large that theSAD of one symphony is not representative of theSAD of all symphonies. More generally the same canbe said for any programme excerpt when compared toanother excerpt containing different voices, musicetc. (Excerpt is used in the sense of a short ex-tract from a programme). A representative sampleof the programme content must be obtained to get adistribution from which deductions about the opera-tion of the system can be made.The variation in programme content during a day islarge enough for the complete frequency and ampli-tude range of normal programme material to be in-cluded. The quasi-stationary nature of a day longsample can be seen in Figure 1 where the SAD for aparticular programme as measured on 10 consecutivedays is plotted on one graph. The variation betweendays is less than +/- ldB when the curves aresmoothed (geometric smoothing with smoothing con-stant of 0.25 is used [5]).

APPLICATION OF NON-INTRUSIVE TESTING IN PRACTICE

When a broadcaster receives complaints from the lis-tening public regarding the quality of the broad-casts, impairments like poor signal to noise ratiocan be readily identified but dissatisfaction withthe dynamic range is not that easily measured.Subjective assessment has to be used and it takestime and effort to establish that a problem doesin fact exist. Using non-intrusive testing the sig-nal can be monitored at the origin,as it is finally

0018-9136/85/1200-0088$01.00O1986 IEEE

88

TIME EXCEEDED..T . .E .E , C.E E .-1% 1 % 5% 25-% 50% 75--< 9 -0

FIGURE: 1REFERENCE PROGRAMME

broadcast or at any intermediate point. Simple com-parison of the distribution curves shows up anychanges in the signal and a subjective evaluationis no longer necessary. A mask defining performancewould eliminate the need for measurements to be doneat both origin and at final broadcast point but di-rect comparison of measurements made at two pointsdoes have the benefit of quantifying the degradationin performance between input and output over anysection of the transmission.RESULTS OF TESTS ON'LIVE' BROADCASTS

Time Sequential Measurements- Figure 2 showsthe result caused by a compressor/limiter. The sig-nal at the input of the compressor was monitored for

E

m1U

LdJ

I

I

a:

z

U)

one day followed by a one day sample of the signalat the output of the compressor. The two uppercurves represent the levels exceede.d by 1% of thetime and the two lower curves define the levels ex-ceeded 90% of the time. The inner curves are thosemeasured at the output. The dynamic range (1-90%)at every frequency is the distance in dB between the1% and 90% curves. For this compressor there is verylittle difference between input and output indica-ting that it has been set up in the way the userintended and that the input signal is at the correctlevel.The peak in response at about 15kHz is due to asub-audible carrier used for data transmission.

FREQUENCY kHz

FIGURE: 2

COMPRESSOR CORRECT

E

m

Lii

-Jz

(-9

(I)

89

A further comparison of the input/output curves ismade in Figure 3. Here the 1% lines are plottedwith input as independent and output as the depen-dent variable. For exact correspondence a line at45 deg. should result. The close correspondence tothis is shown by the +/- ldB lines drawn on thegraph. The lack of similarity at low levels is dueto the effect of the bandlimiting filter in thecompressor output. These measurements do depend onthe input and output levels being the same: leveldifferences result in a shift in the curve.Listener complaints resulted in the measurementsmade on another compressor shown in Figure 4. As

m 0

-25

-2s

'30

-35:3 q-

in Figure 2 the lines represent 1% and 90% limitswith the input being the outer lines for most ofthe time. The change in dynamic range at the higherfrequencies is clearly visible and amounts to about6dB at lIkHz. Figure 5, input vs. outputalso showsthe difference which can be interpreted as a de-crease in output signal level since the curve liesbelow the -ldB line for a significant portion ofits length. Differences in the distribution curvesother than signal levels can be seen as deviationsfrom the 45 deg. line, here seen at the lower sig-nal levels as a slope steeper than 45 deg. The de-viation from 45 deg. could be due either to a fre-quency response error or a change in dynamic range.

FIGURE: 3COMPRESSOR CORRIEICT

FIGURE: 4COMPRESSOR INCORRECT

90

INUTJ SIGNAL dEm

E

.'a

wLi,-J

a:z

U)

FIGURE: 5COMPRESSOR INCORRECT

EC o

-5

SIGNAL: DRY L cBnv

FFIGURE: 6

REFERENCE PROGRFIMME

91

-J

z

cn

D0~

RI

cE.

z

U,

E

-55 -S -45 -40 -35 -30 -59 -20 -S -10 5 0

- SIGNAL: DRY I Bm

FIGURE: 7REFERENCE PROGRAMIMME

To establish which of these causes the problem wehave to refer to the SAD curves. There is a changein dynamic range as shown in Figure 4 but this doesnot exclude frequency response problems or frequen-cy dependent compression. The measurements shouldbe repeated at a level where no compression isintro,duced. This will leave frequency response asthe only variable.The reference distribution curves of Figure 1 areevaluated for day to day correspondence in Figure6, (day 1 and day 2) and Figure 7 (day 1 to day 6).The day to day variation in SAD is small and thedeviation from the 45 deg. line is minimal, almost

W 35

z

I: 25

cc2z

>- 20

the whole curve lies within the +/- ldB limits.This justifies the use of one day as a representa-tive sample.Direct comparison of the dynamic range removes theuncertainty due to different signal levels and fre-quency response errors. The badly adjusted compres-sor of Figure 4 is re-evaluated in Figure 8 wherethe dynamic range derived from the SAD curve forthe input and output are superimposed. The uppercurve is the dynamic range at the input and theamount of compression introduced between input andoutput is reflected in the lower curve which showsthe range at the output.

FRLOUENCY k Hz

FIGURE: 8DYNAMIC RRNGE COMPRRISON

92

CD

cc:

z

U)

Em 0

FREQUENCY kHz

FIGURE: 9PIANO SONATR: COMPARISON

Figure 9 is a comparison of two recordings of apiano sonata. The upper curve was derived fromSAD measurements at the studio output while thelower curve is for the same signal received 'offair'. A loss in dynamic range is seen but thehigh frequency effects seen in Figure 6 are obscu-red by the lack of spectral content.

Simultaneous Measurements. To characterisethe pertormance of items--ofequipment in a pro-gramme circuit, simultaneous measurements at inputand output eliminates the time constraints onsingle point measurements. It is no longer neces-sary to sample the signal for a period of timelong enough for the statistics to be regarded as

representative of programme content in general.The distribution curves measured simultaneouslyat two points must be the same and any differencescan be attributed to the imperfections in thechannel between the two points.A suitable programme excerpt should be used as thetest signal, for example a signature tune and pro-gramme announcement, and the distribution curvesfor this excerpt stored for reference. Care mustbe taken that the SAD of the 'test signal' hasfrequency and amplitude range sufficient to showup defects in the circuit being tested.

FIGURE; 10SYSTEM BLOCK DIRGRRM

Em

LLIza:

r:

a:Z:

93

LINE

MEASUREMENT TECHNIQUEThe measuring set up is shown in block diagramform in Figure 10. It consists of an analogue todigital converter, FFT processor and micro com-puter. The signal is sampled and the spectrum ina 20kHz band evaluated with 50Hz resolution. Themeasured spectrum is used to update an array con-taining the cumulative spectral amplitude distri-bution (SAD). The array has 200 frequency pointsand 100 amplitude points spanning the range 100Hzto 20kHz and -9OdBm to OdBm. Each entry in thearray is incremented by one if the measured spec-trum contains a component at that frequency andamplitude. The only information stored is theSAD array- Mass storage requirements are keptlow. The SAD array is updated once every 5 mi-nutes if data is collected over a day. The datacollection can be done at shorter intervals if aparticular programme excerpt is to be characte-rised or if simultaneous measurements are madeat input and output of the equipment.The results presented in this paper were obtainedusing a Hewlett Packard type HP 3561A DynamicSignal Analyser. This instrument is expensive andcannot be used as an operational tool where manypoints in a broadcast transmission system need tobe measured. With the signal processing IC s nowavailable a dedicated instrument could be construc-ted at a cost which will make non-intrusive testingaccessible to most broadcasters. Such an instru-ment is being developed at the SABC.

CONCLUSIONSThe day to day variation in broadcast signal sta-tistics are small enough for these statistics tobe used in evaluating the performance of the sys-tem during programme time. Where the functioningof equipment is dependent on the input signal, asin a compressor, correct operation can be esta-blished'during programme time since the programmesignal is being used as a test signal.The spectral amplitude distribution contains fre-quency domain information and also shows the dy-namic amplitude characteristics of the signal atthe measuring point. This can be interpreted toevaluate the system frequency response and dynamicrange variations.Non-intrusive testing using distribution curvesdoes not provide a high resolution measurement ofthe characteristics of a transmission system andrequires a fairly long time period of gatheringdata before the programme signal has passed throughall amplitude and frequency points of interest.Data gathering time can be shortened by using adedicated programme excerpt which contains all therelevant amplitude and frequency excursions.

REFERENCES[1] Shiro Ehara. Spectral Amplitude and Power

Distributions of Sound Programme Signals:NHK Laboratories. Note No. 269. Jan. 1982

[2] Shiro Ehara. Some Applications of Program-Model Signals: NHK Laboratories. Note' No.207. Jan. 1977

[3] CCIR. Characteristics of Signals Sent OverSound-Programme Circuits: CCIR Report 491-2

[4] P.Z. Peebles. Probability Random Variables andand Random Signal Principies: McGraw-Hill.1980. pp. 126-130

[5] E.A. Robinson. M.T. Silvia. Digital Foundationsof Time Series Analysis: San Fransisco. HoldenDay. 1979. pp. 23-28.

94