copyright 0 WAC Artificial Intelligence in Real Time Control, Valencia, Spain, 1994

A COMPUTATIONAL INTELLIGENCE PERSPECTIVE ON PROCESS MONITORING AND OPTIMIZATION

Y. H. PA0

Case Western Reserve University and AI WARE Inc., Cleveland, OH 44106, USA

Abstract. In the design of autonomous computer-based systems, we often face the embarrassing situation of having to specify, to the system, how it should carry out certain tasks, which involve computations known to be intractable or are suspected of being so. To circumvent such impasses, we resort to complexity-reducing strategies and tactics which trade some loss of accuracy for significant reductions in complexity. The term computational intelligence refers to such complexity reduction methods and to the research aimed at identifying such methods. ln this paper we describe briefly some of our own work in this area and then develop a computational intelligence view of the task of process monitoring and optimization, as performed by autonomous systems. Some important current fields of discovery in computational intelligence include neural-net computing, evolutionary programming, fuzzy sets, associative memory and so on.

Key Words. Neural net computing; Computational intelligence, Process monitoring, Process optimization; Electric power heat rate optimization

1. INTRODUCTION

Those who are engaged in the design and implementation of computer-based autonomous systems are all too often in the embarrassing position of having to specify, in detail, computational procedures which grow superpolynomially with the size of the problem. For all practical purposes, such computations are intractable, if approached naively and directly. To circumvent such impasses, we resort to complexity- reducing strategies and tactics which trade some loss of accuracy for, hopefully, very large reductions in complexity.

We suggest that the term computational intelligence might be used to refer to such complexity-reducing methods and to the research aimed at identifying and understanding these methods.

Some important current fields of endeavor and discovery in computational intelligence include those of neural-net computing, genetic algorithms, evolutionary programming, fuzzy sets, associative memories, artificial life and so on. Robert Marks II (1993), in a delightful editorial, ascribes the first usage of that phrase to James C. Bezdek (1992) who emphasized that computational intelligence is not Artificial intelligence

previous work in dynamic programming, Expert Systems and so on. It is merely that as further explorations are carried out in research, a deeper understanding of the power and the limitations of the various methods is attained and it becomes possible to transcend some of the limitations of the earlier works.

Also it is helpful to mention that computufional

intelligence is not incompatible with some aspects of traditional Artijcial Intelligence; rule-based methods can also be viewed with a fuzzy-set perspective, to greater advantage and with better understanding.

In principle, computational intelligence is directly concerned with the research for powerful paradigms for decreasing complexity and increasing efficiency. These paradigms are for use in computer-based autonomous systems, i.e. machines, and there is no constraint that these machines should tinction internally in the same manner as humans do, nor are the non-deterministic steps, in the algorithms, necessarily supplied by human experts. Instead such knowledge may be (and perhaps should be) determined by observations, by data and by computations. On the other, hand if such knowledge has already been documented and compiled by humans, there is no reason why such documented, adaptable knowledge should not be used in computational intelligence systems.

I am happy with both of those views but hasten to say that these paradigms are not incompatible with

25

In compu~tionaI ~ntelIigen~ research, it is important that the new algorithms discovered by experimentation be validated, justified, or rationdlized with traditional mathematical procedures, when possible. Of cause, that process can also be reversed and mathematics might provide guidance in the experimentation.

In this paper our intent is to descrih an ~~te~~igent procedure in terms of three basic ~n~tionalities of &ompu~~onal intelligence. The procedure is that of process monitoring and optimization, and the three basic functionalities are

-- function approximation, or learning a computational model of the monitored process,

- optimization, and - associative memorization and recall.

We describe some of our works aimed at realization of these functionalities, in sections 2, 3, 4, Application of such matters to the task of process monitoring and optimization is discussed in section 5, and two examples of process monitoring and optimization are described briefly in section 6.

hidden laya nodes

input pattern

(8)theGDRnet

(Ntst I) prameten for epcfi output node

cnhmced @tern

(b) the Rpndam-Ve&r FL net

Fig.1 Comparison of Backpropagation and Random- Vector Functional-Link Procedures

2. THE FUNCTION&LINK NET: EFFIC~N~Y IN LEAFS A MODEL OF A

PROCESS

Nonlinear regression is a difflculc task, In other words, it is difficult to formulate an accurate analytical expression for a nonlinear function of many variables, given a finite number of examples

of the ~nctional relations~p. (Bates 1988, Ratkowsky 1983, Kolmogorov 1957)

In terms of learning a model, the input variables describe the state (or set of states) of the monitored system and the function or the output of the model describes the outcome of the process, either the value of a characteristic or perhaps the next connation.

One of the contribution of (artificial) neural-net computing was to venture that the nonlinear function could be regarded as an expansion in terms of a number of basis function each of which is already a nonlinear function of all the input variables. However it was only recentIy that it was in fact proved that such a procedure could indeed yield universal appr~~mation under certain circ~nces, the generalized Perceptron of neural- net computing being one of those circumstances. Such a computational description of a function is depicted in Figure l(a) in the form of a feedforward net when the function to be learned, f(xJ, is

expressed as f(x) = ~fijg(~~~+b,). In the

Backpropagation algorithm for realizing such a net all the parameters Ej,gjand b-j need to learnedd.

This makes the algorithm to be of exponential complexity in (dN) where d is the dimentionality of the input space and N is the number of hidden layer nodes.

We found experimentally (Pao, Park and Sobajic 1994) that the learning of the external parameters /?. could be separated from the learning of the --I

internal parameters aiand kj and that in fact the

latter two sets did not have to be learned at all, but could be specified randomly in a particular manner, hence the name the Random-Vector Functional- Link (RVFL) net, for which the learning of the ,$

then becomes a linear learning task. Recently we have proved that such a procedure can also yield support univer~l approximation (Igelnik and Fao 1994). In addition our work now suggests that there is no particular virtue in jumping to a particular choice in the selection of the multivariate basis functions and expansion in sums of products of one- variable tinctions would work just as well. Such results lead to a better understanding of the practice of ad-hoc expansion in terms of various arbitrarily chosen basis functions.

In the Random-Vector Fictions-ail Net (RVFL), the hidden layer nodes are in fact replaced by the randomly generated enhancement nodes as depicted in Figure 1 (b). The learning of the external expression parameters is a linear tasks, and in principle, addition of enhancement nodes can

26

also be handled easily, if necessary, when large numbers of new examples are encountered. Overlearning is readily avoided by removing all the enhancement nodes which do not contribute to the expression.

The Random-Vector Functional-Link Net can be implemented with different activation functions (or basis functions), and Gaussian functions constitute one of the admissible classes. This reconciles the RVFL approach with the practices of expansion in terms of arbitrary nonlinear basis functions but the RVFL provides a rationale for the procedure and suggests methods for error reduction.

Appropriate expansion in terms of properly chosen basis functions generally yields nets which are several orders of magnitude faster in the learning of functional mappings, or models of processes.

All research activities aimed at enhancing the efficiency of the process of function approximation are indeed to be considered computational intelligence research.

3. OPTIMIZATION

Whereas function approximation is concerned with an analytical or computational description of a functional relationship, optimization is concerned with determining that point in the multivariate input space for which the value of function is (say) at a global minimum. This is a computational task also of exponential complexity or worse. Evolutional Programming approaches this task by generating a set of parents. These parents give rise to sets of children, which are mutations of the parents. The children are evaluated. For each set of children, that with the best value is retained as the parent of the new generation, and each new parents generates a set of children for the next generation and so on. We have expanded on this theme in corporating in the process, themes from Genetic Algorithms and from methods of stochastic search with simulated annealing (Aarst and Korst, 1989). In our Guided Evolutionary Programming with Simulated Annealing (GESA) we employ regional guidance and two levels of competition, as well as the practice of simulated annealing, In this paradigm, the optimization procedure starts off with each parent having the same number of children. The children are evaluated competitively to see which child is best selected to serve as the parent using a simulated annealing procedure and the child has a chance of serving as the next parent even if it is somewhat worse than the parent. A temperature parameter is involved.

For each family, all the children are evaluated to determine how many are accepted. This determination is achieved by comparing the value of each child with that of current global optimal attained so far, Regional competition consists of apportioning the relative numbers of children fro the next generation in accordance with how many children were accepted in the previous generation, using the approach of simulated annealing and involving a temperature parameter.

It can be shown that this approach is guaranteed to yield the global minimum, asymptotically (Aarst and Korst, 1989), but our interest is in increasing the efficiency of the search and our results indicate that GESA is indeed a highly efficient algorithm (Yip and Pao, 1993, 1994). GESA is then indeed another example of computational intelligence methodology.

pertubations and drifts

1 monitoring and optimization

’ km model of illpUtS (initial system state) l---l

process . fmdcontiiguration

Process outpuzj which yields

process best perfoml&tnce

parameters d (final results) l monibr

settings input/output

A mabna

oppurtunity to make some intermediate

~ua&e model and

inputs for updated model

Fig. 2 A Schematic Depiction of a Simple Process and an Effective Monitoring and Optimization Approach

4 ASSOCIATIVE MEMORIZATION AND

RECALL

With the Random-Vector Functional-Link net and with GESA, we could learn a model of a process and then proceed to maintain it in the optimal posture. Even complicated and convoluted processes can be modeled and maintained with this procedure. However, unfortunately, this is not all there is to process monitoring and optimization; the human and environmental aspects of the operation enter in interesting and important ways.

It seems that humans remember complexes of information effectively and efficiently in terms of associations between categories of objects, events, and episodes of events, These seem to be coded in linguistic symbols. Numeric or graphic details are indeed also remembered, as appropriate, but seem to be special packages indexed linguistically but with numeric values or graphic details.

27

Large opaque rn~~v~ate-to-rn~~v~ate ~ppings are inapprop~ate for the description of complex processes because such processes are multi-process, hierarchical and sequential in nature and can best be monitored and maintained eficientiy when described with the appropriate conceptual structure. Any attempt to capture everything in one huge opaque mapping is doomed to failure for many reasons, including the fact that humans need to interject themselves into the stream of things, to describe the process in terms of lin~istically described objects and events and transformations. With those tie points, it is then possible to link a process to the outside world; maintenance and optimization tasks are then also much more efficiently performed.

In this area, we have taken advantage of some work on the universal approxi~tio~ capabili~es of fuzzy set systems (Wang 1992) to develop a way in which opaque neural-net computing mappings can be endowed with fuzzy-sets and fuzzy rules interpretations. With the use of GESA we can determine the shapes and positions of membership functions and discover which of various possible fuzzy-set combinations are indeed use%1 valid rules (pao and Mattias, 1994).

5 THE COMPUTATIONAL INTELLIGENCE PERSPECTIVE

A process and the task of monitoring and optimization can be depicted schematically in the manner of Fig. 2, where the initial system state and the process parameter settings are shown as inputs to a mapping for which we know the final results. A computational description of the process is learned from a set of associated input/output pairs of observations and the process can be optimized in the sense that the inputs can be adjusted until the output is optimal. Such a view of the monitoring and optimization task is indeed valid and is ofien practical in that manner. However some human operators ask for op~~uni~ to describe the task more meaningfully in terms of internal intermediate concepts and mechanisms. In that case we can use our GESA/tizzy-set combined procedure to intervene and provide a depiction such as that shown in Fig. 3,

Finally, we recognize the fact that a process can in fact be made up of a number of rather distinct operations, one afier another and some measurements can be made at intervening steps and some adjustments in parameter settings can also be made at intervening steps. In Fig. 4, we exhibit such a scenario of monitoring and optimization steps. Many mappings need to be learned to provide

the basis for rno~~o~ng and control but the principles remain the same. Interjection of intermediate interpretations in terms of fuzzy-sets and fuzzy-&e depictions of internal processes may also be carried, if helpful and as appropriate.

pmsibillty of interaction with humans; disawuing of funy descriptions of sub-proamx; optimization ofdertiptian (hming membership functkm)

Fig 3. A Schematic Depiction of How Opaque Process Descriptiofi Can Be Represented In Terms of Concepts

6. EXAMPLES OF PROCESS MONITOFUNG AND OPTIMIZATION TASKS

We cite two examples of process monitoring and optimization both in the electric power industry

In one case the task is to monitor the opti~~tion of a power plant in an electric power utility to ensure that NO, emission is within legal limits

which heat rate is maintained as low as possible, as depicted schematically in Fig. 5.

A company* using a product incorporating RVFL neural-net computing and GESA Evolutionary Programming is engaged in applying computational intelligence metrologies to mo~to~ng and optimal process planning. We cite some of their work at some Western Pennsylvania and Western New York State electric power utilities. The task is

to reduce heat rate (Btu/kw-hr) while keeping iVOX

emission within legal limits. Heat rate reduction has a direct cost saving component in the fuel (coal) saved. In addition, there are many indirect savings on equipment.

For heat rate two matters are important. The first being the overall heat balance for the plant. This includes the pressures, temperatures and water (steam) flow throughout the plant. The second is the combustion of the coal in the furnace which generates the heat to convert the water to steam in the boiler. NO, is the product of this combustion

process and o~rational p~meters for op~mum combustion change with time.

Inputs include pulverizer flow and temperature measurements which provide indications of primary

* Pegasus Technologies Carp, Painesville, OH.

28

air flow and coal flow, furnace temperatures and windbox pressures. Induction and forced-draft-fan motor amps provide indications of flow balance on primary and secondary air flows respectively and are also inputs. Other miscellaneous inputs include main steam temperature and pressure, condenser backpressure, individual 02 probe readings (in stack gas), feedwater and airheater temperatures, and overfire air positions.

Of prime importance for heat rate and NOX control

is the use of over-fire air ports. In general, opening

the ports increases heat rate while reducing NOX.

The software system is used to find the optimum positions that maintains the best combustion efficiency, These positions tend to be dynamic, ~ons~uently, an o~ti~~tion is run every 10 minutes. The training cycle has yet to be fixed, but could be as often as every hour or only once a day.

The other main parameter optimized is the 02 trim.

These values are constrained based on load, to insure valid recommendations.

At present they lack CO measurements which would provide a direct indication of combustion efficiency. Including that measurement there would be approximately 80 inputs, with 3 outputs to be optimized.

When optimizing, only 9 to 1.5 inputs are allowed to float, the rest are held constant, as they are not changeable to a short time frame by operations {short < 1 hour). In the engineering mode, however, all points are allowed to float and comparing the optimum parameters found versus the control system setpoints provided guidance for tuning of the systems So far, this has led to corrections in 02 trim and calibration, procedures far obtaining the pulverizer flow correction term, Aiso, the heat rate calculation itself may be modified in response to the results of the present inve~igation.

Results to date indicate heat rate can be reduced by 0.5% to 3% depending on load and fuel conditions.

NOX can be reduced by up to 50% for unit without

low NOX burners.

This first example is an instance of the type of application illustrated in Fig. 2. It can be embellished and evolved into an application of the type illustrated in Fig. 4 but it seems that there is no immediate need to use a description of the type illustrated in Fig. 3. In other words, there is no

need for human to interpret the inner details of the combustion process, as yet.

In a second example we are concerned with the process of bill estimation. A more accurate description of the wider task is that of account monitoring including bill estimation. In the tatter case we let the data speak for themselves, i.e. we use self organi~tion method of neural-net computing to identify recognizable categories(fuzzy-sets) and then explore how the desired mapping can be cast in the form of fuzzy rules; the consequences of which are then defuzzified to yield an optimal overall mapping from input circumstances to output resolution of the bill estimation and account maintenance activities. The second example is an example of the type of system shown in Fig. 3.

Fig. 4 A Schematic Depiction a more Typical Process Consisting of a Sequence of Distinct Subprocesses

steam, air and condensate flows and temperatures

boilers, turbine, generator

heat rate -----W

co -----W

NO, emission ---W

t electric equipment parameter settings

Fig. 5 Schematic Illustration of Context of Application of Computational Intelligence to Optimal Process Planning and to Monitoring of Optimal Process

Aarst, E. and Korst, J. (1989). Simulated Anneahg and Boltzman Machines, John Wiley, N. Y.

Bates, D. M. (1988). Nonlinear Regression Analysis and its Application, John Wiley, N. Y.

29

Bezdek, J. C. (1992). On the ~lationship between neural networks, pattern recognition and intelligence, Int. J. Approximate Reasoning, vol. 6, pp. 85-107

Cybenco, G. (1989). Approximation by superposition of a sigmoidal function, Mathematics of Control, Signals and Systems, vol. 2, pp. 303-314

Funahashi (1989). On the approximate realization of continuous mappings by neural networks, ~e~~~~~e~o~~, 2, 183-192.

Homik, K., Stinchcombe (19891, M., and White, H., MuInlayer feedforward networks are universal approximators, Neural Networks, vol.. 2, pp. 359-366.

Igelnik, B. and Pao. Y. H. (1994). Random vector version of the Functional-Link net, Proc. 28th Annual Conference on rn~~r~~~~~~o~ Science and Sysfems, Princeton, N. J.

Kolmogorov, A. N., (1957). Qn the representation of continuous functions of many variables by superposition of continuous functions of one variable and addition, Doklady Akademic Nark USSR, 114 953-956, and Translations American Mathematical Society, vol. 2, pp 55-59

Marks II, R. J. (1993). Intelligence: computational versus artificial, lEEE Trans. of Neural ~e~~r~, vol, 4, pp 737-739

Pao. Y. H., Manias, N. (1994). Automatic optimal design of fuzzy systems based on universal approximation and evolutionary programming, CAISR Internal Report, Case Western Reserve University.

Pao, Y. H., Park, G. H., and Sobajic, D. J. (1994). Learning and generalization characteristics of the random vector ~nctional-lip net, ~~~rocornp~~~ng, Vol. 6, pp 163-180.

Ratkowsky, D. A. (1993). Nonlinear Regression Modeling: a unijed practicai approach, Marcel Dekker, N. Y.

Wang, L. X. (1992). Fuzzy systems are universal approximators, Proc. IEEE Int. ConJ on Fuzzy Systems, San Diego, CA, pp. 1163-I 169

Yip, P. P. C., and Pao, Y. H, (1993). A new approach to the travehng salesman problem, Proceedings of the ~n~erna~~on~~ Join f Confirence on New-a1 Networks, Nagoya, Japan, pp. 1573-1576.

Yip, P. P. C., and Pao, Y. H, (1994). Combinatorial Optimization with use of Guided Evolutionary Simulated Annealing, IEEE Trans. of Neural Networks, (in press),

30