A knowledge-based expert system for flood frequency analysis

A knowledge-based expert system for flood frequency analysis -

K. C. ANDER CHOW' AND W. E. WATT Department of Civil Engineering, Queen's University, Kingston, Ont., Canada K Z 3N6

Received April 28, 1989

Revised manuscript accepted January 15, 1990

Single-station flood frequency analysis is an important element in hydrotechnical planning and design. In Canada, no single statistical distribution has been specified for floods; hence, the conventional approach is to select a distribution based on its fit to the observed sample. This selection is not straightforward owing to typically short record lengths and attendant sampling error, magnified influence of apparent outliers, and limited evidence of two populations. Nevertheless, experienced analysts confidently select a distribution for a station based only on a few heuristics. A knowledge-based expert system has been developed to emulate these expert heuristics. It can perform data analyses, suggest an appropriate distribution, detect outliers, and provide means to justify a design flood on physical grounds. If the sample is too small to give reliable quantile estimates, the system performs a Bayesian analysis to combine regional information with station-specific data. The system was calibrated and tested for 52 stations across Canada. Its performance was evaluated by comparing the distributions selected by experts with those given by the developed system. The results indicated that the system can perform at an expert level in the task of selecting distributions.

Key words: flood frequency, expert system, single-station, fuzzy logic, inductive reasoning, production system.

L'analyse de la frCquence des crues 2 station unique est un ClCment important de la planification et de la conception hydrotechnique. Au Canada, aucune repartition statistique n'a CtC spCcifiCe pour les crues; par consequent, la mCthode tradi- tionnelle consiste i choisir une rtpartition de manibre ti ce qu'elle corresponde aux donnCes de crue recueillies 2 une station prbs de l'emplacement choisi. Cette selection n'est pas precise en raison de la courte durCe des enregistrements des dCbits des cours d'eau et des erreurs d'tchantillonnage, de I'influence accrue des avant-buttes apparentes, ainsi que de la presence limitte de deux populations. NCanmoins, les analystes experts choisissent une repartition pour une station basCe sur quelques heuristiques seulement. Un systbme expert bask sur la connaissance a Ctt ClaborC pour &tre 1'Cmule de ces heuristiques experts. I1 peut effectuer des analyses de donnCes, proposer une repartition approprike, dCtecter les avant-buttes et justifier une crue de calcul en fonction de critbres physiques. Si 1'Cchantillon est trop petit pour fournir des prCvisions fiables, le systbme effectue une analyse bayCsienne afin de combiner des informations regionales et des donnCes spCcifiques au site. Le systbme a Ctt CtalonnC et soumis 2 des essais dans 52 stations dans le Canada. Sa performance a CtC CvaluCe en comparant les repartitions choisies par des experts et celles fournies par le systbme ClaborC. Les rtsultats indiquent que le systbme peut offrir une performance de niveau expert en ce qui concerne la stlection des rkpartitions.

Mots c l b : frCquence des crues, systbme expert, station unique, logique floue, raisonnement inductif, systbme de production.

[Traduit par la revue]

Can. J. Civ. Eng. 17, 597-609 (1990)

Introduction

Design flood criteria for hydrotechnical projects often require that the flood magnitude corresponding to a specified return period or exceedance probability be determined. The conventional approach is to apply a statistical distribution to fit the flood data collected at a station near the study site; the specified quantile is then estimated from the fitted distribution. Many distributions have been proposed, but no single distribution has been found to be capable of describing all flood data. In the United States, the log Pearson type 3 distribution has been adopted for use in flood analysis (USWRC 1976), whereas in the United Kingdom, the general extreme value distribution has been recommended (NERC 1975). However, in Canada, no distribution has been specified for general application to floods. Instead, an appropriate distribution is selected based on its ability to fit the observed record, which is assumed to be a random sample from the underlying population of floods at the station of interest.

The choice of a distribution for a station is not straightfor-

ward because of the short record length of streamflow data in Canada (typically 20-50 years). With such small samples, it is impossible to state with confidence that a particular distribution represents the true population. There are no deductive rules for deciding on the form of the distribution and hence the appropriateness of each candidate must be examined individu- ally. Secondly, there is no powerful goodness-of-fit index to discriminate between fitted distributions; more than one distribution may be suitable for a given sample. Thirdly, a great deal of uncertainty is associated with the parameter and quantile estimates. Nevertheless, experienced analysts are reason- ably confident in selecting a distribution for a station; these "experts" are believed to rely-on a limited number of heuristics which help them reduce the'complex task to simpler judg- mental operations. For example, an expert may be able to suggest a few candidate distributions by looking at the sample skewness and then choose the most appropriate one based on the corresponding probability plots. In general, these heuristics are quite useful in the decision-making process, but sometimes they may lead to incorrect results if used inamropriately. - - .. -

NOTE: Written discussion of this paper is welcomed and will be a rare event in a can produce an unrealis-

received by the ~ d i ~ ~ ~ until ~ ~ ~ ~ ~ t , ~ ~ 31, 1990 (address inside front tic estimate of the skewness coefficient; if the heuristic that a cover). distribution is chosen based on sample skewness is followed

'Present address: Atria Engineering Hydraulics Inc., 8 Stavebank closely, an inappropriate distribution will be selected. Road North, Mississauga, Ont., Canada L5G 2T4. Numerous computer programs are available for conventional Prinad in Canada / IrnprirnC au Canada

Can

. J. C

iv. E

ng. D

ownl

oade

d fr

om w

ww

.nrc

rese

arch

pres

s.co

m b

y M

erce

d (U

CM

) on

05/

06/1

4Fo

r pe

rson

al u

se o

nly.

598 CAN. J. CIV. EN( 2. VOL. 17, 1990

flood frequency analysis. These programs produce results for a few fitted distributions, but generally, no guidance is provided to determine which distribution is the most suitable. An inexperienced analyst may have difficulty deciding which distribution should be used.

Using advanced artificial intelligence technology, a knowledge-based expert system has been developed to emulate expert heuristics for selecting appropriate distributions. The system has been tested for 52 stations across Canada. Its performance was evaluated by comparing the distributions selected by the expert system with those given by four analysts: two "experts," a statistician, and a consulting engineer. The results indicated that the system can perform at an expert level in selecting an appropriate distribution for a station.

Distribution: selection statistics and heuristics Statistical distributions

Statistical distributions that are used for flood frequency analysis include the normal (N), lognormal (LN), extreme value type 1 or Gumbel (EV 1) , general extreme value (GEV) , Pearson type 3 (P3), log Pearson type 3 (LP3), and three-parameter lognormal (3LN). Detailed descriptions of most of these distributions are available in the Flood Studies Report (NERC 1975). Since the LP3 distribution has been widely used in North America and has sufficient flexibility to fit logarithmically transformed samples, the 3LN distribution is not considered in this project. Similarly, because the GEV distribution has been used exten- sively in the United Kingdom and has sufficient flexibility to fit untransformed samples, the P3 distribution is not considered. The remaining five distributions (N, EV1, GEV, LN, and LP3) are deemed sufficient to cover the range of samples encoun- tered in practice.

Heuristics for distribution selection When selecting an appropriate distribution to describe flood

data, analysts (both expert and novice) concentrate on two measures: (i) comparison of sample skewness with the theoretical distribution value and (ii) comparison of fitted distribution with observed data series. There are two drawbacks to using the skewness alone for selection. First, the dimensionless fourth moment (kurtosis) should also be considered. Second, the accu- racy of the skewness estimate is very dependent on sample size. For example, Fisher (1930) has shown that for the normal (or lognormal) distribution, the standard errors of estimate are 0.464, 0.337, and 0.277 for sample sizes of 25, 50, and 75, respectively. Therefore, for the record lengths of flood peaks typically available in Canada, the sample skewness should be used as a weak indicator to suggest a set of possible distributions. Other evidence, physical or statistical, should be collected to select the most appropriate one.

For large samples, visual examination of the goodness of fit of the candidate distributions to the observed data can be an effective procedure for selecting a distribution. The observed data are assigned plotting positions; the compromise formula proposed by Cunnane (1978) has found general acceptance and is used for the plots presented in this paper. A linear trend of the observed data series on conventional probability paper (i.e., for two-parameter distributions such as normal, lognormal, or Gumbel) indicates that a sample may be appropriately described by the distribution to which the probability paper applies. If the trend is not linear, this may indicate that the two-parameter distributions (which have fixed skewness) are not adequate and that a three-parameter distribution should be considered. For

small samples, however, the nonlinearity in the observed data series may result from sampling error or from inaccurate plotting position of the extremes. For example, Fig. 1 shows the probability plots of five distributions fitted to the annual flood peaks of the LaHave River at West Northfield, Nova Scotia (WSC (Water Survey of Canada) No. 01EF001). The maximum observed flow deviates from the fitted line significantly, but most of the data points follow the fitted EV1 and LN distributions. Hence, visual evaluation of the goodness-of-fit should be assessed based on the fit of all data points rather than solely on the extreme points. As a substitute for visual evaluation, linearity of probability plots for two-parameter distributions can be measured using the probability plot correlation coefficient (PPCC) proposed by Filliben (1975) for the normal distribution and extended to the EVl by Vogel (1986). If the plot is linear, the PPCC will be close to 1.

Another means of evaluating goodness of fit is to compare the shape of the fitted probability density function with the relative frequency histogram of the sample. Figure 2 shows these plots for the LaHave River data referred to above. Sometimes, gaps in the relative frequency histogram occur when the group- ing interval is too small. Undue attention should not be paid to these gaps; rather, emphasis should be placed on whether the form of the probability density function has the same shape as the relative histogram of the sample. In this figure, the EVl , GEV, LN, and LP3 distributions seem to fit the sample reason- ably well.

Occasionally, objective goodness-of-fit tests such as Chi- square, Kolmogorov-Smirnov, and Cramer-Von Mises are used to help select the most appropriate distribution. However, as reported in the Flood Studies Report (NERC 1975), these tests are of limited value for flood data largely because of the typically small sample size. An additional limitation is that the Kolmogorov-Smirnov and Cramer-Von Mises test statistics do not apply when distribution parameters have been estimated from the data. Special tables have been prepared to account for this, but they are only for some of the distributions used for floods.

If goodness of fit is the sole criterion, the distribution with more parameters is usually selected, since credit is given only to the models that fit the data well and no "penalty" is imposed for model -complexity (i.e., increasing number of parameters). The information criterion developed by Akaike (1974) is employed in this expert system to discriminate among fitted distributions. It is defined as

[I] AIC = -2 x LL(x; 0 ) + 2m

where LL(x; 0 ) is the log-likelihood function for a distribution with m parameters, 0 . If more than one model can describe the observed data, the model that has the minimum AIC is pre- ferred. From [I], it can be seen that the AIC can be decreased by reducing LL(x; 0 ) (i .e. ,Etter fit), and increased by adding model parameters. Hence, the use of a three-parameter distribution as opposed to a two-parameter distribution is justified only if the improved fit compensates for the penalty due to the additional parameter.

Outliers can complicate selection of a distribution. The prac- tical problems are first identification and second treatment. A test for identification (Grubbs and Beck 1972) has been proposed for use in the United States, but there are no general guidelines in Canada for either identification or treatment.

In the case of three-parameter distributions such as GEV and LP3, comparison of upper bounds to the fitted distributions

Can

. J. C

iv. E

ng. D

ownl

oade

d fr

om w

ww

.nrc

rese

arch

pres

s.co

m b

y M

erce

d (U

CM

) on

05/

06/1

4Fo

r pe

rson

al u

se o

nly.

CHOW AND WATT

RECURRENCE INTERVAL (YEARS) RECURRENCE INTERVAL (YEARS)

RECURRENCE INTERVAL (YEARS)

GEV . . . . . . . . . . . . . . . . : ............ : ............................................... . . . . . .

:. . . . . . . . . . . . . . . . . , . . , . . . . . . . . . .

. . . . . :. , . .

1.003 2 5 10 20 50 100200 500


FIG. 1. Probability plots for the LaHave

with maximum observed flows is an important step in selecting a distribution. In general, if maximum observed flows exceed the fitted upper bound, the fitted distribution should be rejected because an upper bound implies that there is no chance that a flood will exceed this limit. The maximum observed flows could exceed the estimated upper bound (but not the true value); however, it is difficult to prove on a physical basis that the maximum observed floods coincide with the maximum flood that can occur in the drainage basin. Figure 3 shows the observed annual maximum series and the fitted LP3 distribution for the Richelieu River at Rapides Fryers, Quebec (WSC No. 020J007). It can be seen that the largest observed floods are very close to the upper bound of the fitted LP3 distribution. Hence, this LP3 distribution should not be used.

As long as a distribution is selected based on its fit to the sample, the problem of the choice of distribution will remain unsolved. Owing to the small samples normally available, a particular distribution cannot be proven to be far superior to

LP3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . lo3

.... 5 . . . . . . . . . . . . . . . . . . . . . . . ...................... 1 .......... 1.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.................. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . : . .

.............. ............................................................ . . .:. . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . ...... . . 1 ................. ; : ...'............. 2.. 103 ...........................................................................

1.003 1.05 1 2 5 20 I00 500


River at West Northfield, N.S. (01EF001).

others and to represent the true population. Overfitting is a concern when the sample size is small, say less than 20, because the parameter estimates are subject to large sampling errors. Hence, the use of complex distributions to fit the samples is not justified.

- Development of the knowledge-based expert system

Selection of a distribution for a data sample from a station is based primarily on someone's judgement. There is no correct or standard answer and different experts may select different distributions. Accordingly, the reasoning system developed herein will not select the "correct" distribution, but rather provide an effective tool to help an inexperienced analyst select an appropriate one from among the many proposed. Because there are no deductive rules and experts' heuristics cannot be used systematically, it is difficult to write a conventional program to emulate the reasoning process used by such

Can

. J. C

iv. E

ng. D

ownl

oade

d fr

om w

ww

.nrc

rese

arch

pres

s.co

m b

y M

erce

d (U

CM

) on

05/

06/1

4Fo

r pe

rson

al u

se o

nly.

600 CAN. J. CIV. ENG. VOL. 17, 1990

DISCHARGE ( m3ls ) DISCHARGE ( m3ls )

DISCHARGE ( m3ls ) DISCHARGE ( m 3 h )

FIG. 2. Comparison plots of the probability density function of the fitted distribution and relative frequency histogram of the sample for the LaHave River at West Northfield, N.S. (01EF001).

experts. Therefore, a reasoning system was developed using expert system techniques. In this section, the heuristics and reasoning process employed in this reasoning system are discussed and the methodologies presented. Then, application of this developed system to a long-term station is illustrated.

Heuristics It should be noted that these heuristics reflect only one

approach to selecting distributions in single-station flood frequency analysis. They could be modified and revised to reflect other approaches.

1. The sample skewness is used to suggest candidate distributions.

2. Probability plots are used to test the hypothesis that a two-parameter distribution provides an adequate fit to the sample data.

3. A parsimonious approach is adopted to help prevent overfitting. Two-parameter distributions are selected first,

even though a three-parameter distribution may also be suitable, unless strong evidence is available to suggest that the GEV or LP3 distribution is more appropriate.

4. When an outlier is detected, it is considered to be a rare event and is treated as historical data. This means that it is not discarded but that its plotti= position computed in the normal way is considered to be ifiappropriate. Hence, these rare events should not be considered when visually evaluating the goodness of fit of the probability plots.

Fuzzy logic and fuzzy numbers With the heuristics specified above, it might appear easy for

an expert to choose an appropriate distribution. However, it is not easy to encode these heuristics in a computer program and to use them systematically. First, the heuristics require much numerical information which is subject to large sampling errors. Questions then arise. For example, what range of sample skewness should be considered "close to O"? Or, how

Can

. J. C

iv. E

ng. D

ownl

oade

d fr

om w

ww

.nrc

rese

arch

pres

s.co

m b

y M

erce

d (U

CM

) on

05/

06/1

4Fo

r pe

rson

al u

se o

nly.

CHOW AND WATT


FIG. 3. The LP3 plot for the Richelieu River at Rapides Fryers, Quebec (0205007).

SKEWNESS COEFFICIENT

FIG. 4. Membership function for the fuzzy statement that skewness is close to zero.

good a probability plot should be considered a "good fit"? The sample skewness could be considered to be 0 if it is within one or two standard errors of 0. The probability plots could be considered to be good fits if the goodness-of-fit index is within the 90-95 % significance levels of the index statistics. Hence, quantitative measures are required to express this numerical information in qualitative (linguistic) statements. Secondly, the heuristics themselves are fuzzy and can be

0

A-D A A+D

BASE VARIABLE - FIG. 5. Membership function for a fuzzy number in FLOPS.

changed when new information is available. For example, suppose that an outlier is detected in a sample of size 60. When this event is removed from the sample, the sample skewness is reduced from 0.4 to 0.01. In this case, an expert may suggest a normal distribution, even though the sample skewness of the original data is not "close to 0," that is, greater than two standard errors of the skewness estimate for the normal distribution.

Can

. J. C

iv. E

ng. D

ownl

oade

d fr

om w

ww

.nrc

rese

arch

pres

s.co

m b

y M

erce

d (U

CM

) on

05/

06/1

4Fo

r pe

rson

al u

se o

nly.

602 CAN. J. C N . ENG. VOL. 17, 1990

............................................. FLEX REASONING SYSTEM CHOW 03/88

............................................. loading the program . . . This program may take seve ra l minutes t o r u n . . . read i n the da ta . . .

(1) I n t e r p r e t da ta . . . . . - Skewness is pos i t ive - Skewness of log da ta is pos i t ive - k f o r GEV i s not zero with confidence of 917.

(2 ) Se lec t d i s t r i b u t i o n by A I C . . . . . (3 ) Se lec t d i s t r i b u t i o n based on sample skewness . . . . (4 ) Se lec t d i s t r i b u t i o n based on p robab i l i t y p l o t s , GEV k , e t c . . .

- GEV i s se l ec t ed because k is not zero with confidence of 917. (5) Find reason t o r e j e c t GEV. . . . ( 6 ) Find reason t o r e j e c t LP3 . . . . (7) Post ana lys i s examination t o d e t e c t o u t l i e r s . . . .

- Since t h e skewness can be reduced s i g n i f i c a n t l y by removing t h e l a r g e s t f lood , a HIGH o u t l i e r i s suspected with confidence of 1000.

- Since t h e l a r g e s t observed f lood is g r e a t e r than the maximum value given by Grubbs and Beck s t a t i s t i c , a H I G H o u t l i e r i s suspected.

( 8 ) Check whether the o u t l i e r s a f f e c t the choice of N , L N and EV1.. - Since the 2 parameter d i s t r i b u t i o n s may be r e j e c t e d due

t o the high o u t l i e r , t he 2 parameter d i s t r i b u t i o n w i l l be reconsidered by ignoring the l a r g e s t f lood.

(9) Reconsider 2 parameter d i s t r i b u t i o n s . . . - Skewness of log da ta is about zero with confidence of 737. - PPCC ind ica te s t h a t LN is l i n e a r with confidence of 888. - PPCC ind ica te s t h a t E V 1 i s l i n e a r with confidence of 970. - LN is se l ec t ed because skew i s zero with confidence of 737. - LN i s se l ec t ed because l i n e a r PPCC with confidence of 888. - E V 1 i s s e l e c t e d because l i n e a r PPCC with confidence of 970.

. L - S -? 0 x 6 Out l i e r s *** Type of Ou t l i e r de tec t ed : , "HIGH"

.I-.+ Conclusions *** Normal, 501 Lognormal, 888 Gumbel, 970 General extreme value,917 Log Pearson type 111, 504

****** system e x i t i n g ****** FIG. 6. Output for the LaHave River at West Northfield, N.S. (01EF001)

Numerical information is essential to the reasoning system, but techniques to interpret and use this information are the most critical part of the system. The reasoning system described herein consists of two components: (i) a computer program, written in FORTRAN, which extracts relevant information from a given sample and produces necessary numerical information; and (ii) a second program, written in FLOPS (Siler and Tucker 1986), which interprets the information and uses it to select distributions.

Since the numerical information is subject to a great deal of uncertainty, the corresponding linguistic statements should have associated with them a degree of confidence. Fuzzy logic and fuzzy numbers are used to convert this uncertain numerical information into linguistic statements.

If a statement is completely true, the degree of confidence

associated with it is unity; and if a statement is completely false, the degree of confidence is zero. But when statements are made based on judgeme~t and evaluation, the truth of these statements cannot be preciseh defined and hence the degree of confidence associated with these fuzzy statements lies between 0 and 1. Fuzzy logic is used to describe the truth of a fuzzy statement with a degree of confidence.

Zadeh (1965) extended the concept of fuzzy logic to set theory and developed fuzzy set theory. In conventional set theory, objects either belong or do not belong to a set. However, in the real world, an object can belong partially to one set and partially to another. Hence, the degree of belonging is expressed as a grade of membership ranging from 0 (the object absolutely does not belong to the set) to 1 (the object absolutely belongs to the set).

Can

. J. C

iv. E

ng. D

ownl

oade

d fr

om w

ww

.nrc

rese

arch

pres

s.co

m b

y M

erce

d (U

CM

) on

05/

06/1

4Fo

r pe

rson

al u

se o

nly.

CHOW AND WATT

TABLE 1. Study stations

wsc No. Name

Province or Area z o w territory (krn2) condition*

Stations for calibration 09AB001 Yukon River at Whitehorse 08EC001 Babine River at Babine 08EE004 Bulkley River at Quick 08LE031 South Thompson River at Chase 08ME001 Bridge River near Shalalth 08MG005 Lillooet River near Pemberton 08NH006 Moyie River at Eastport 08NH032 Boundary Creek near Porthill 08NN012 Kettle River near Laurier O8NN013 Kettle River near Ferry 05AA022 Castle River near Beaver Mines 05AD003 Waterton River near Waterton Park 05BB001 Bow River at Banff 05BJ005 Elbow River above Glenmore Dam 05CC002 Red Deer River at Red Deer 05HD036 Swift Current Creek below Rock Creek 05PA006 Namakan River at outlet of Lac la Croix 05PB014 Turtle River near Mine Centre 05QA001 English River near Sioux Lookout 05QA002 English River at Umfreville 04LJ001 Missinaibi River at Mattice 02AA001 Pigeon River at Middle Falls 02BD003 Magpie River near Michipicoten 02EA005 North Magnetawan River near Burk's Falls 02EC002 Black River near Washago 02FC001 Saugeen River near Port Elgin 02FC002 Saugeen River near Walkerton 02GA010 Nith River near Canning 02HB001 Credit River near Cataract 020J007 Richelieu (Riviere) aux Rapides Fryers 02ZK001 Rocky River near Colinet 01AD002 Saint John River at Fort Kent OlAKOOl Shogomoc Stream near Trans-Canada Highway 01AQ002 Magaguadavic River at Elmcroft 01AR003 St. Croix River near Baileyville OlBEOOl Upsalquitch River at Upsalquitch 01DG003 Beaver River near Kinsac OlECOOl Roseway River at Lower Ohio OlEFOOl LaHave River at West Northfield OlEOOOl St. Mary's River at Stillwater OlFBOOl Northeast Margaree River at Margaree Valley 01FB003 Southwest Margaree River near Upper Margare

Stations for testing 02ED003 Nottawasga River near Baxter 02FF002 Ausable River near Springbank 02GG002 Sydenham River near Alvinston 02LB008 Bear Brook near Bourget 02YC001 Torrent River near Bristol's pool 02YL001 Upper Humber River near Reidville 05FE001 Battle River near Unwin 05GC007 Opuntia Lake West Inflow 05LB002 Etomami River near Bertwell 05MC001 Assiniboine River at Sturgis

Y.T. 19 400 N B.C. 6 480 N B.C. 7 360 N B.C. 16 200 N B.C. 3 650 N B.C. 2 160 N B.C. 1 480 N B.C. 25 1 N B.C. 9 840 N B.C. 5 750 N Alta. 826 N Alta. 614 N Alta. 2 210 N Alta. 1 220 N Alta. 11 600 R Sask. 1 430 N Ont . 13 400 N Ont . 4 870 N Ont . 13 900 N Ont. 6 230 N Ont . 8 940 N Ont. 1 550 N Ont. 1 930 N Ont. 32 1 N Ont. 1 520 N Ont. 3 960 N Ont . 2 150 N Ont . 1 030 N Ont. 205 N Que. 22 000 N Nfld. 285 N N.B. 14 700 N N.B. 234 N N.B. 1 420 R N.B. 3 420 N N.B. 2 270 N N.S. 96.9 N N.S. 495 N N.S. 1 250 N N.S. - 1 350 N N.S. 368 N

:e N.S. 357 R

Ont . Ont. Ont. Ont. Nfld. Nfld. Sask. Sask. Sask. Sask.

*N, natural; R , regulated.

Now, consider the fuzzy statement "the sample skewness is close to 0. Hence, the fuzzy statement, which may be defined close to 0." The precise value of sample skewness that can be with a membership function as shown in Fig. 4, can be considered to be "close to 0" cannot be specified. Intuitively, described by a set of real numbers.It can be seen that a sample if the skewness estimate is 0.001, the skewness can be said to skewness of 0 gives a confidence level of 1, whereas a sample be close to 0 with great confidence. When the skewness esti- skewness greater than 4 or less than -4 gives a confidence mate is much greater than 0, say 10, the confidence level is level of 0. In this sense, the sample skewness could be consid-

Can

. J. C

iv. E

ng. D

ownl

oade

d fr

om w

ww

.nrc

rese

arch

pres

s.co

m b

y M

erce

d (U

CM

) on

05/

06/1

4Fo

r pe

rson

al u

se o

nly.

CAN. J . C N . ENG. VOL. 17. 1990

TABLE 2. Sample information of study stations

WSC Area Record length Maximum flow* Minimum Tow No. (km2) (years) (m3/s) (m3/s)

09AB001 08EC001 08EE004 08LE03 1 08ME001 08MG005 08NH006 08NH032 08NN012 08NN013 05AA022 05AD003 05BB001 05BJ005 05CC002 05HD036 05PA006 05PB014 05QAOOl 05QA002 04LJOO1 02AA001 02BD003 02EA005 02EC002 02FC001 02FC002 02GA0 10 02HB001 020J007 02ZK00 1 01AD002 0 1 AKOO 1 0 1 AQ002 0 1 AR003 OlBEOOl 01DG003 OlECOOl OlEFOOl OlEOOOl OlFBOOl 01FB003 02ED003 02FF002 02GG002 02LB008 02YC001 02YL001 05FE001 05GC007 05LB002 05MC001

*Annual maximum series of mean daily flows.

ered as a fuzzy number, that is, a set of numbers for describing bership function with a maximum confidence level of 1000 at a fuzzy (linguistic) statement. A and a confidence level of 500 at A + D, where D =

J ( B ~ + (CA)2), as shown in Fig. 5. In this application, the Fuzzy number in FLOPS fuzzy number is taken to be the skewness coefficient, the prob-

In FLOPS, a fuzzy number is defined by a central value, ability plot correlation coefficient, a variable related to the A, and either an absolute uncertainty, B, or a relative uncer- shape factor in the GEV distribution, and a variable related to tainty, C, and is characterized by a normally distributed mem- the change in skewness coefficient (Chow 1988). The central

Can

. J. C

iv. E

ng. D

ownl

oade

d fr

om w

ww

.nrc

rese

arch

pres

s.co

m b

y M

erce

d (U

CM

) on

05/

06/1

4Fo

r pe

rson

al u

se o

nly.

CHOW AND WATT

TABLE 3. Selected distributions for study stations -

Expert system AIC Expert 1 Expert 2 Statistician Engineer

Choice Choice Choice Choice Choice Choice WSC No. 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd 1st 2nd

09AB001 08EC001 OSEE004 08LE03 1 08ME001 08MG005 08NH006 08NH032 08NN012 08NN013 05AA022 05AD003 05BB001 05BJ005 05CC002 05HD036 05PA006 05PB014 05QAOOl 05QA002 04LJ001 02AA00 1 02BD003 02EA005 02EC002 02FCOOl 02FC002 02GA010 02HB001 0205007 02ZK001 0 1 AD002 0 1 AKOO 1 01AQ002 0 1 ARM3 OlBEOOl 01DG003 OlECOOl OlEFOOl OlEOOOl 0 1 FBOO 1 01FB003 02ED003 02FF002 02GG002 02LB008 02YC001 02YL001 05FE001 05GC007 05LB002 05MC001

N LN N LN EV1 LN LN EV1 LN LN N LN EV 1 EV 1 EV 1 GEV N EV1 N EV1 LN LN N N EV1 LN LN LN GEV GEV GEV LP3 LP3 LN EV1 LN LN EV1 LN LN EVl LN EV1 LN LN EV 1 EV 1 EVl LN LN EV1 LN LN LN EVl LN EV1 LN LN EV1 LN EV1 EV1 LN EV1 EV1 LN LN N EV1 N N EV1 N EV1 LN EV1 N EV1 N EV1 LN EV1 N GEV LN EV1 LP3 N EV1 N EV I EV 1 LN EV1 GEV EV1 LN LP3 EV1 LN EV1 EV1 LN LN EVI LN LN EV1 LN GEV EV1 LN EV1 LN EVl EV1 EV1 LN LN LN EV1 LN EV1 LN EV1 EV1 LN EV1 EV1 LN EV1 EV1 LN EV1 EV1 LN EV1 LN LN LN LN EV1 N EV1 EV 1 EV 1

GEV EV 1 N N GEV EV 1 GEV GEV GEV EV 1 LN LN EV 1 LP3 LP3 EV 1 GEV EV 1 EV 1 EV 1 GEV LN LN EV 1 GEV GEV LN EV 1 GEV N GEV GEV LN LP3 LN LN EV 1 EV 1 LP3 LN GEV EV 1 EV 1 LN LN LN LN LN LP3 LP3 GEV LN

N GEV LN N LN LN N GEV EV1 GEV N LN N LN LN GEV GEV LN

LN LN LN EV 1 N EV1 LN LN LN LN N EV 1 EV1 LN EV 1 N GEV N EV 1 N LN N GEV N EV 1 GEV EV 1 LN EV1 EVl LN EV 1 EV 1 EV1 LN LN GEV EV1 LN LN EV1 EV1 LN EV1 LN LN EV1 LN EV1 EV1 LN LN LN EV 1 EV 1

LP3 LP3 LP3 LP3 LP3 LP3 LP3 EV 1 GEV EV 1 LP3 LP3 LP3 LN LN GEV LN EV 1 LP3 LN LN EV 1 LN EV 1 LN LP3 LP3 LP3 LN GEV LP3 LP3 LP3 LP3 LN LP3 LN LN LP3 LP3 LP3 LN LP3 LP3 LN LN LN LN LN LP3 LP3 EV 1

N LN LN LN LN LN LN LP3 LP3 LP3 LP3 LP3 EV 1 LP3 EV 1 LN LP3 LN LP3 LN LN N LP3 LP3 GEV N LN LN EV 1 LN LP3 EV 1 LN LN LN LP3 LN N EV 1 LP3 LP3 EV 1 LP3 LN N LP3

N LP3 LN LP3 LP3 LP3 N GEV LP3 LN LN

LN LN LN GEV LN LP3 LN LN LN EV 1 LN EV 1 LN GEV LP3 LP3 GEV GEV GEV GEV LP3 LP3 LP3 GEV GEV LP3 - LP3 LP3 LP3 LP3 LP3 GEV EV 1 LP3 LN EV 1 LP3 LP3 GEV LP3

LN LN LP3 LN GEV GEV LP3 LP3 GEV LP3 LP3

LP3 LP3 LP3 LP3 LP3 LN LP3 LP3 LP3 GEV LP3 GEV LP3 LP3 GEV

LP3 EV 1 LN LN LN GEV LN LN LP3 GEV LN LP3 GEV LN - LN LP3 EV 1

N LP3 LP3 LP3 LP3 LP3 N GEV N LP3 LP3 LP3 LN LP3 LN EV 1 LP3 LP3 LN LN LN EV1 EV 1 EV 1 LP3 LP3 LP3 LP3 LP3 LP3 LP3 LP3 EV 1 LP3 LP3 LP3 LN EV 1 EV 1 LP3 LP3 LN EV 1 LP3 EV 1 LN LN

- LN = LN

LN EV 1 EV 1

LN LN GEV LN GEV GEV

GEV

GEV LN

GEV GEV GEV EV 1

LN LN GEV

GEV

LN GEV GEV

value, A, was set to the expected value (e.g., 0 and 1.14 is the of the 95 % significance levels. For example, Filliben (1975) case of the skewness coefficient for N and EVI distributions). gives 95% significance levels of the probability plot correla- The relative uncertainty, C, was generally set to 0. The values tion coefficient (PPCC) for the normal distribution. For a for absolute uncertainty, B, were assigned on an intuitive sample size of 30, the value is 0.964. Therefore, A - D equals basis, but generally A + D were thought to be representation 0.964 and hence B is equal to 0.036 (A is unity and C is zero).

Can

. J. C

iv. E

ng. D

ownl

oade

d fr

om w

ww

.nrc

rese

arch

pres

s.co

m b

y M

erce

d (U

CM

) on

05/

06/1

4Fo

r pe

rson

al u

se o

nly.

606 CAN. J. CIV. ENG. VOL. 17, 1990

The acceptable performance of the expert system, which is TABLE 4. Summary of distributions selected described below, indicated that these values for B are appropriate. umber of stations

Reasoning procedures Because there are no deductive rules, a distribution is

chosen by executing a few sets of rules sequentially (i.e., inductive reasoning mode) whereby each set of rules represents a particular heuristic. At the end of execution of each set of rules, a set of distributions is proposed based on the final confidence level. The inductive process can be summarized in the following steps.

1. To ensure selection of a valid distribution by the reasoning system, a default distribution is selected based on the AIC value. At the beginning, the system assigns the distribution with the smallest AIC value a confidence level of 505; the distribution with the second smallest AIC value a confidence level of 504, and so on. The fact that approximately the same confidence levels are assigned to the proposed distributions means that all are assumed to be essentially equally suitable.

2. More information, such as sample skewness and the PPCC, is collected to increase the confidence level of the proposed distributions. It should be noted that the confidence level can only increase.

3. An outlier which significantly changes the sample skewness and distorts the probability plots will provide fallacious evidence, indicating that the two-parameter distributions are inappropriate. To minimize the chance of overfitting, this outlier is first identified using the Grubbs and Beck (1972) test. Then if the two-parameter distributions are not selected owing to its presence, the outlier is ignored temporarily and the two- parameter distributions are reconsidered.

An example The system was applied to a long-term station, the LaHave

River at West Northfield, Nova Scotia (WSC No. OlEFOOl), which has a record length of 71 years. The maximum and minimum flows are 1080 and 94.3 m3/s. Sample skewness values for the original and log-transformed data are 4.2 and 0.63 respectively. Using the sample skewness alone, a three- parameter distribution seems necessary to fit the data owing to the high skewness. However, the probability plots shown in Fig. 1 indicate that a high outlier is suspected. This explains why the sample skewness is so high. When examining these plots closely, the EV1 and LN distributions seem to provide a good fit to all of the data, except for the high outlier, and hence these distributions should be considered.

When the reasoning system was applied to this station, the FORTRAN program computed all the required information and the FLOPS program read in this information and printed out all distributions with their accompanying confidence levels, as shown in Fig. 6. It can be seen that a high outlier was detected in step 7. The system removed the outlier temporarily and reconsidered the two-parameter distributions at steps 8 and 9. At this stage, the EV1 distribution is proposed on this basis of PPCC confidence level of 970 and the LN distribution is considered as a second choice (PPCC confidence level of 888).

Performance evaluation Streamflow data

Streamflow data from 52 stations across Canada were selected to calibrate and test the reasoning system. These stations were selected according to two criteria: (i) the record length must

Analyst

Engineer Statistician Expert 1 Expert 2 AIC Expert system

Agreement with expert system

Distribution

N LN EV1 LP3 GEV

3 12 11 25 1 2 13 4 21 11 9 17 18 0 7 0 17 6 26 3 7 20 17 3 5 8 14 29 0 1

be at least 25 years and (ii) there must be no significant regula- tion or diversion upstream of the station. With these criteria, the stations were chosen from the Pacific to the Atlantic drainage systems and therefore could be considered geographically representative of Canada. Condie (1977) used 32 of these stations in analyzing the maximum likelihood estimators for the LP3 distribution. Each station with its Water Survey of Canada identification number, name, province, drainage area, and flow condition is given in Table 1. Table 2 lists the record length and the maximum and minimum flows. The average record length is 55 years, with a maximum of 78 years and a minimum of 27 years. The untransformed samples have an average skewness of 1.1 with a maximum of 4.2 and a minimum of -0.6.

Results To evaluate the performance of the reasoning system, inter-

views were conducted with four analysts: a consulting engineer, a statistician, and two experts. These persons could be considered as experts with different levels of training and experience in this field. In the interviews, the analysts were asked to select the best distribution based on sample statistics and probability plots. If they felt that another distribution was equally suitable, the distribution was treated as the second choice. The analysts' selections as well as the distributions selected by the reasoning system and by using the AIC are summarized in Table 3. Using the distributions selected by the reasoning system as a-standard, the number of identical distributions selected by the analysts is summarized in Table 4. The reasoning system agreed with the engineer, the statistician, the two experts, and the AIC for 58, 60, 75, 90, and 94% of the total 52 stations. Both first and second choices were considered, because the analysts were asked to identify a second choice if two distributions were considered equally appropriate.

Discussion Selecting a distribution i u o t a determininstic problem for

which there is clearly a right or wrong choice. Very often it is impossible to demonstrate in a straightforward manner that an analyst is correct, and in what sense the expert system is considered wrong. In light of this, the expert system is said to perform at an expert level if it agrees with the experts, even if both the expert system and the experts turn out to be wrong.

To demonstrate this point, probability plots for two stations are presented. The first, the English River of Umfreyville (05QA002), is one case where the selection procedure is straightforward and the lognormal distribution was selected by all parties (see Table 3 and Fig. 7). The second, the Rocky

Can

. J. C

iv. E

ng. D

ownl

oade

d fr

om w

ww

.nrc

rese

arch

pres

s.co

m b

y M

erce

d (U

CM

) on

05/

06/1

4Fo

r pe

rson

al u

se o

nly.

CHOW AND WATT



GEV


FIG. 7. Probability plots for the English River at Umfreyville, Ont. (05QA002).

River near Colinet (02ZK001), is a case where all of the analysts do not agree with the expert system (see Table 3 and Fig. 8). In general, as shown in Tables 3 and 4, the analysts sometimes agree and sometimes disagree with the expert system, to a variable extent. The reasons for this are discussed below.

Analysts may be classified according to their area of expertise, level of training, years of experience in a field, number of publications, and so on. With the different attributes of the analysts involved in the interview, it was not surprising to find that the reasoning system did not completely agree with them.

The engineer is familiar with the LP3 distribution in practice, but has little knowledge of the GEV distribution. He favours the use of the LP3 distribution because of its flexible form which fits the data. As indicated in Table 4, the engineer selected the LP3 distribution for 25 stations (about half the sample). The statistician has no bias as to the choice of distribution, but puts more emphasis on the tail behavior of the fit-

ted distributions. Hence, the LP3 and GEV distributions were chosen for 21 and 11 stations respectively.

Expert 1 prefers the approach adopted in the development of the expert system, that is, a three-parameter distribution will not be used unless it is clearly necessary. Comparing the distributions given by expert 1 and the reasoning system, the selected distributions are almost identical except for the stations 08MG005,05AD003, O ~ P A ~ , 02ZK001, and OlAQ002. For three of these stations (08MG005, 02ZK001, and 01AQ002), all analysts disagreed with the expert system. Expert 1 chose the GEV distribution to minimize the deviation of the largest floods from the fitted line, whereas the reasoning system ignored the largest data point in the probability plots and found that the LN or EVl distribution was adequate to fit the data. For station 05AD003, when a "hockey stick" or strong indi- cation of two populations was observed in the probability plots, neither expert 1 nor the statistician would choose any distribution. For station 05PA006, it was quite difficult to

Can

. J. C

iv. E

ng. D

ownl

oade

d fr

om w

ww

.nrc

rese

arch

pres

s.co

m b

y M

erce

d (U

CM

) on

05/

06/1

4Fo

r pe

rson

al u

se o

nly.

CAN. J . CIV. ENG. VOL. 17, 1990



- GEV



FIG. 8. Probability plots for the Rocky River near Colinet, Nfld. (02ZK001)

prove whether the EV1 or normal distribution was superior to the other.

Expert 2 uses different heuristics for selecting distributions. He was reluctant to use the GEV and N distributions unless they were clearly better. In case of tied distributions, he selected the one that was more conservative near the top end of the fitted distribution. Hence, the LP3 distribution was chosen as first choice for 26 stations, even though the two- parameter distributions (N, LN, or EV1) were considered appropriate for these stations.

Comparing these two experts' approaches, it appears that they use different methods in treating the largest floods, par- ticularly the high outliers. Expert 1 puts more emphasis on the fit of all data points and believes that there is a great uncertainty in the plotting position of the largest floods owing to small sample size. Hence, he will not choose a distribution in order to fit these extreme data points, even though they deviate

from the fitted line significantly. Expert 2 prefers a better fit to these largest data points. Therefore, expert 2 selected the three-parameter distributions more frequently than expert 1.

The AIC results given in Tables 3 and 4 are worthy of note for three reasons. First, this criterion has not been used in flood frequency analysis as an aid to distribution selection, although it has been widelzused in time series applications. Second, the distributions seltcted on the basis of the AIC are in most cases (49 of 52) the same distributions selected by the expert system. The reason for this agreement is that the measures implicit in the two approaches are consistent. That is, two-parameter distributions are compared in terms of goodness of fit and three-parameter distributions will be selected only if the additional parameter can be justified. Finally, a limitation of the AIC is illustrated by the common factor for the three stations for which the AIC and expert system choices were different (02ZK001, OlAQ002, and 01EF001). The AIC

Can

. J. C

iv. E

ng. D

ownl

oade

d fr

om w

ww

.nrc

rese

arch

pres

s.co

m b

y M

erce

d (U

CM

) on

05/

06/1

4Fo

r pe

rson

al u

se o

nly.

CHOW AND WATT 609

led to three-parameter distributions (GEV or LP3), whereas the expert system selected two-parameter distributions. The improvement in fit was largely due to the highest point (one of these stations, 0 1 EF00 1, has been discussed above).

Based on the samples used in this project, the reasoning system seems to perform at an expert level and is able to select appropriate distributions automatically. Shortcomings of the reasoning system were also found. For instance, it does not know how many high outliers are in a sample. If there are more than two high outliers in a sample, the reasoning system does not know whether they come from the same population or not. If a "hockey stick" is observed in the probability plots, the reasoning system cannot detect this phenomenon.

Summary An expert system has been developed to conduct single-

station flood frequency analysis. Using expert system techniques to emulate expert heuristics, it is capable of selecting an appropriate distribution. The performance of the system was evaluated by comparing its selections with those of analysts with varied experience and expertise. For the samples used in the study, the system performs at an expert level in the task of selecting distributions. The results also showed that different experts employ different heuristics.

Acknowledgement Financial assistance for this research was provided by the

Natural Sciences and Engineering Research Council of Canada (NSERC).

AKAIKE, H. 1974. A new look at the statistical model identification. IEEE Transactions on Automatic Control, AC-19(6): 7 16 - 721.

CHOW, K. C. A. 1988. Towards a knowleage-based expert system for flood frequency analysis. Ph.D. thesis, Queen's University, King- ston, Ont.

CONDIE, R. 1977. The log Pearson type 3 distribution: the t-year event and its asymptotic standard error by maximum likelihood theory. Water Resources Research, 13(6): 987 -99 1.

CUNNANE, C. 1978. Unbiased plotting positions: a review. Journal of Hydrology, 37: 205 -222.

FILLIBEN, J. J. 1975. The probability plot correlation coefficient test for normality. Technometrics, 17(1): 1 1 1 - 1 17.

FISHER, R. A. 1930. The moments of the distribution for normal samples of measures of departure from normality. Proceedings Royal Society, London (A), 130: 16-28.

GRUBBS, F. E., and BECK, G. 1972. Extension of sample sizes and percentage points for significance tests of outlying observations. Technometrics, 14(4): 847-854.

NERC. 1975. Flood studies report. Natural Environment Research Council, London, United Kingdom.

SILER, W., and TUCKER, D. 1986. FLOPS, a fuzzy logic production system. User's Manual, Kemp-Carraway Heart Institute, Birming- ham, AL.

USWRC. 1976. Guidelines for determining flood flow frequency. Bulletin 17, Hydrology Committee, U.S. Water Resources Coun- cil, Washington, DC.

VOGEL, R. M. 1986. The probability plot correlation coefficient test for the normal, lognormal and Gumbel distributional hypotheses. Water Resources Research, 22(4): 587-590.

ZADEH, L. A. 1965. Fuzzy sets. Information and control, 8: 338-353.

Can

. J. C

iv. E

ng. D

ownl

oade

d fr

om w

ww

.nrc

rese

arch

pres

s.co

m b

y M

erce

d (U

CM

) on

05/

06/1

4Fo

r pe

rson

al u

se o

nly.

Documents

A knowledge-based expert system for flood frequency analysis