This is the published version: Nguyen, Thi and Peterson, Jim 2008, Enhancing the performance of the fuzzy system approach to prediction, in FSKD 2008 : Proceedings of the Fuzzy Systems and Knowledge Discovery 2008 international conference, IEEE, Piscataway, N.J., pp. 259-265 Available from Deakin Research Online: http://hdl.handle.net/10536/DRO/DU:30054436 ©2008 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE. Copyright: 2008,IEEE

Enhancing the Performance of the Fuzzy System Approach to Prediction

Thi Thanh Nguyen School of Geography and Environmental

Science, Monash University, Australia thi.nguyenthanh@gmail.com

Jim Peterson School of Geography and Environmental

Science, Monash University, Australia jim.peterson@arts.monash.edu.au

Abstract

Using time-series data, and testing the value of incorporating genetic algorithm, momentum technique, event-knowledge, selective presentation learning (under the SEL algorithm) and new training criteria (for financial time series), it is demonstrated that significant improvement in predictability accrues in deployment of the Standard Additive Model (SAM) during application of the fuzzy set approach to forecasting. 1. Introduction

Because the value of a variable often depends on a range of determinants, all varying over time in ways that may or may not be closely related to each other, if at all, prediction can be a complex matter. Even if we can identify all influencing factors of a variable, it is usually impossible to collect time-series data of all factors, and so much of what we might want to know to be confident about establishing trends is not accessible. However, most prediction techniques call for such data.

We can assume that future value xn+1 of variable x depends on its n latest past values x1, x2, …, xn. The value of non-linear function f(x1, x2, …, xn) therefore will be xn+1, the predicted value. Generally, a prediction problem is considered as one of non-linear function approximations wherein the function’s approximated value will be used as the prediction.

Recently, a new approach has become regarded as offering a “universal approximators” path [3]. It is based on deployment of one or other of the fuzzy maths system models. This paper presents use of the Standard Additive Model, especially popular among those looking to apply fuzzy systems to solve prediction problems by function approximation. Reported here is such an application, but designed for improved prediction via incorporation of genetic algorithm, the momentum technique, use of event-knowledge, selective presentation learning, and new training criteria (for financial time series data).

The rest of this paper is organized as follows. The next section presents function approximation with SAM. The third section outlines SAM’s learning process to build the

knowledge base. The fourth section discusses some advanced techniques to improve SAM’s predictive capacity. The performances of these techniques are presented in the fifth section. 2. Function approximation with the Standard Additive Model

A fuzzy system F: Rn Rp stores m if-then rules and can uniformly approximate continuous and bounded measurable functions in the compact domain [3]. This approximation theorem allows any choice of if-part fuzzy sets Aj ⊂ Rn. It also allows any choice of the then-part fuzzy sets B Bj ⊂ R because the system uses only the centroid c

p

j and volume Vj of BjB to compute the output F(x) from the vector input x ∈ Rn.

( )

( )

( )( )∑

==

∑=

∑=

=

∑=

= ⎟⎟⎠

⎞⎜⎜⎝

⎛

m

j jcxjpm

j jVxjajw

m

j jcjVxjajw

m

j jBxjajwCentroidF(x)

11

1

1 (1)

The fuzzy system F: Rn Rp covers the graph of an approximand f with m fuzzy rule patches of the form Aj×Bj ⊂ Rn×Rp or of the word form “If X = Aj then Y = BBj”. If-part set Aj ⊂ Rn has joint set function aj: Rn [0, 1] that factors: . Then-part fuzzy set

Bj

)n(xnj)...a(xja(x)ja 1

1=

B ⊂ Rp has set function bj: Rn [0, 1] and volume (or area) Vj and centroid cj. The convex weights:

( )( )( )∑

=

= m

k kVxkajw

jVxjajwxjp

1

(2)

give the SAM output F(x) as a convex sum of then-part set centroids. We can ignore the rule weights wj if we put w1 = … = wm > 0.

Figure 1 shows the parallel structure of the additive systems and its state-space graph cover. The graph cover leads to an exponential rule explosion. Mitaim and Kosko proposed using metrical joint unfactorable fuzzy sets based on metric and matrix knowledge to overcome this

Fifth International Conference on Fuzzy Systems and Knowledge Discovery

978-0-7695-3305-6/08 $25.00 © 2008 IEEE

DOI 10.1109/FSKD.2008.198

259

drawback (but only partly) [24]. A fuzzy system needs on the order of kn+p-1 rules to approximate a function f: Rn Rp in a compact domain. Optimal rules cover extrema and can help allocate a spare-rule budget in high dimensions [4]. Learning trends to move the rule patches toward the extrema or “bumps” and fill in with rule patches between the bumps. Supervised learning tuned the parameters of the many if-part set functions we tried. It also tuned the then-part volumes and centroids [22, 23, 24].

Figure 1. (a) A parallel structure of SAM. Each input fires each fuzzy rule to some degree to compute F(x). (b) Fuzzy rules define patches in the input-output space. This leads to exponential rule explosion in high dimensions.

3. Learning process in the Standard Additive Model

Learning is an important process of SAM with the aim of constructing a knowledge base that is a structure of if-then fuzzy rules. Learning allows SAM to tune the parameters and check the optimal model of fuzzy rules structure to improve the accuracy in approximation. The SAM learning process includes basic steps such as structure learning, optimizing and parameter learning (Figure 2).

3.1. Identifying centers and shapes of fuzzy sets

Identification of the centers and shapes is performed by deployment of an algorithm designed for unsupervised clustering. We researched and implemented some clustering methods that include Fuzzy C-Means – FCM [10], Adaptive Fuzzy Leader Cluster – AFCL [27], Adaptive Vector Quantization – AVQ [14], Adaptive Vector Quantization Leader – AVQL [27], K-means [16, 26], Gustafson-Kessel – GK [1, 19], and Gustafson-Kessel Leader – GKL [27].

Two classes of fuzzy sets are used: factorable and unfactorable. The factorable fuzzy sets include triangle, trapezoid, Gaussian, Cauchy, Sinc, Laplace, logistic, difference logistic, differential logistic, hyperbolic tangent, difference hyperbolic tangent, hyperbolic secant, and clipped-parabola (quadratic). The unfactorable fuzzy sets encompass symmetric metrical triangle, joint Gaussian, joint Cauchy, and the metrical form of fuzzy sets: parabolic, Cauchy, Laplace, Sinc, logistic, hyperbolic tangent, hyperbolic secant, and difference logistic [24, 27].

In trying to achieve accuracy and stability in application of fuzzy sets in prediction based on time series data, clustering methods AVQ, AVQL and K-means performed best in experiments reported here. Some factorable fuzzy sets such as Sinc, Gaussian, Laplace, Cauchy and the unfactorable fuzzy sets such as metrical triangle, joint Cauchy, metrical Gaussian, and metrical Laplace, showed stability, simplicity and accuracy. Some other membership functions (e.g. trepazoid, difference hyperbolic tangent, metrical difference logistic) often take much time in processing and therefore they are less effective.

3.2. Constructing if-then fuzzy rules

Based on the clustering results, the fuzzy rules system is constructed. The centers of fuzzy rules are identified via quantization vectors. Identifying the parameters of membership functions requires the width of fuzzy sets. Dickerson and Kosko proposed using the ellipsoidal fuzzy rule, and the AVQ clustering algorithm to identify exactly the center and width of fuzzy sets via center, shape and size of ellipsoids [13]. However, this method is complex in implementation. Besides, fuzzy set width can be tuned afterwards in the parameter learning process and therefore, in this step, the width of ith fuzzy set, for example, is set simply via its neighbours by the following

formulae: r

cmim

iσ

−= , where mi is the center of ith fuzzy

set, mc is the center of the closest fuzzy set, and r is the overlap coefficient.

260

3.3. Optimizing fuzzy rule structure

We used the Genetic Algorithm (GA) to optimize a fuzzy rules system. The GA is a population-based, evolutionary optimization method where populations are evolved over generations using the Darwinian principle of survival of the fittest. The GA offers a way of resolving objective functions that are non differentiable, non-continuous, non-linear, noisy, flat, multidimensional, even in the presence of many local minima or constraints [6, 10].

Optimization accrues from SAM application in realizing and/or removing the fuzzy rules that are not necessary in its approximation. The number of fuzzy rules therefore is reduced. That helps to improve the speed of processing and reduce noise in parameter learning.

The fitness function is used:

( ) ( )n

mnσmFitlog2ln += ⎟

⎠⎞⎜

⎝⎛

ε

∑=

−= ⎟⎠⎞⎜

⎝⎛ ⎟

⎠⎞⎜

⎝⎛n

j jxFdjy

nσ

1

212ε

where m is the number of fuzzy rules, and n is the number of learning data sets.

The GA has three basic genetic operators: selection, crossover and mutation. The crossover and mutation probability that were used herein are 0.5 and 0.01 respectively.

3.4. Tuning fuzzy rule parameters

Fuzzy rule parameters are tuned by the supervised learning process [22, 23, 24]. The supervised gradient descent can tune all the parameters in the SAM model. We seek to minimize the squared error:

( ) ( ) ( )( )22

1xFxfxE −=

of the function approximation. The vector function f: Rn Rp has components and so

does the vector function F. Let denote the k

( ) ( ) ( )( )Txpfxfxf ,...,1=

kjξ th

parameter in the set function aj. Then the chain rule gives

the gradient of the error function with respect to , with

respect to the then-part set centroid , and

with respect to the then-part set volume V

kjξ

( )Tpjci

jcjc ...,,=

j. A gradient descent learning law for a SAM parameter

has the form:

ξμξξ

∂

∂−=+

Ettt )()1(

where is the learning rate at iteration t. tμThe momentum technique is also integrated in the

parameters tuning process [20]. The learning fomulae with momentum is as follows:

)(.)()1( tE

ttt ξεξ

μξξ Δ+∂

∂−=+

where ε is the momentum coefficent. 4. Advanced techniques for time series prediction

Figure 2. Data processing path for applying the SAM approach to the learning processes with incorporation of advanced techniques. We need to edit data samples when using event-knowledge (Figure 3). The Genetic algorithm was used to optimize fuzzy rule structure before the process of parameter tuning. Momentum technique, selective presentation learning and new training criteria for financial time series

Momentum technique

Data samples

Clustering using unsupervised algorithm

Identifying centers and shapes of fuzzy sets

Constructing if-then fuzzy rules

Tuning fuzzy rule parameters using supervised algorithm

Selective presentation

learning New training criteria for

financial time series

Using event-knowledge

Optimizing fuzzy rule structure

(7)

(3)

(4)

Genetic algorithm

(5)

(6)

261

could be integrated at the time of tuning parameters.

4.1. Using event-knowledge (EK)

The future value of a variable often depends on its past

values, on the past values of other correlated variables, and on some project-specific non-numerical factors. For example, the future price of a stock depends on political and international events as well as on various economic indicators. Yan et al. proposed a new method to extract features of predictive value from time series data based on a neural network and the K-means clustering method [28]. Gao et al. used a neural network and back-propagation approach to extract events from time series data [7]. Kohara et al. have studied types of prior knowledge which are difficult to insert into initial network structures or to represent in the form of error measurements [18]. They made use of prior knowledge of stock price predictions and newspaper information on domestic and foreign events. EK is extracted from newspaper headlines according to prior knowledge, but reconstruction (hindcasting) of an EK time series, however desirable, is not always possible/feasible. Moreover, the relation between the future values of a variable to its affecting factors is not a simple reflection of some if-then rules based on prior knowledge.

We propose a method to use EK in fuzzy systems. EK in the past can be extracted directly from the time series data of any considered variable. Knowledge of events that influence future value is divided into two types: negative-EK, for events which tend to reduce future values and positive-EK, for events which tend to raise them [17, 18].

The steps by which this method is followed are: o Smoothing time series data [8, 21]. o Establishing the difference values between

smoothed and original values. o Normalizing difference values into [0, 1]. o Assigning normalized value at (t + 1) to EK value

at (t). Smoothing allows the model series-data to be

represented as a really nonlinear function that is assumed to present the trend of time series in normal condition. Holt’s linear exponential smoothing is used herein. The normalized difference value at (t + 1) shows the influence-degree of summarized events at (t) onto time series at (t + 1). The near 1 (one) value presents influence-degree of positive-EK. This dominates the negative-EK. The 0 (zero) value represents the opposite case. An EK value of 0.5 presents the normal condition or the balance of influence degree between positive-EK and negative-EK. The fuzzy rules were modified as follows:

“If input_1(t) = Aj1 and input_2(t) = Aj2 and … and input_n(t) = Ajn and EK(t) = Ej then output(t+1) = Bj”

Application of EK into SAM is modelled in Figure 3.

This method allows experts to use their experiences to estimate the EK value at (t) and consider it as a model’s input to predict value of time series data at (t + 1). We validated this method by using the EK value that is extracted from data samples. The result is presented in Table 1.

Fuzzy system under the Standard

Additive Model

EK (t)

Input n (t)

Input 1 (t)

Input 2 (t)

Output (t + 1) …

Figure 3. EK is an extra input integrated into SAM. The past EK value is extracted directly from time series data. The EK at (t) is estimated by experts and inputted into SAM to predict time series at (t + 1).

4.2. Selective presentation learning (SEL)

Based on the SEL algorithm that Kohara used for neural networks about financial market predictions [17], we applied it in our fuzzy system with some modified parameters, dealing with the following conventional learning problems.

Problems: Generally, the ability to predict large changes is more important than the ability to predict small changes. When all training data are presented equally as in conventional learning, we assume that the systems learn small changes as accurately as large ones and thus cannot learn large changes more effectively than small ones.

To overcome the above drawbacks, the training data was separated into large-change data and small-change data. Large-change data (small-change data) have next-day changes that are larger (smaller) than a preset value. Large-change data are presented to neural networks more often than small-change data. For example, all training data are presented in the first learning cycle, only large-change data are presented in the second cycle, and so forth. The outline of the selective presentation learning algorithm is as follows.

SEL algorithm: o Separate the training data into large-change data

and small-change data.

262

o Train back-propagation networks with more presentations of large-change data than of small-change data.

o Stop network learning at the point satisfying a certain stopping criterion.

4.3. New training criteria for financial time series

The traditional back-propagation training criterion is

based on goodness-of-fit which is also the most popular criterion for forecasting. However, in the context of financial time series forecasting, we are not only concerned at how good the forecasts fit their target. In order to increase the forecastability in terms of profit earning, Yao and Tan proposed a profit-based adjusted weight factor for back-propagation neural network training [15]. Instead of using the traditional least squares error, a factor which contains the profit, direction, and time information is added to the error function. Based on the models, we applied them for training criteria in a fuzzy system. The results show that the new approach does improve the fuzzy system forecastability for the financial application domain (Table 2).

It is not only in applications to financial data that deployment of the Least Squares or Ordinary Least Squares (OLS) error function (Equation 5) may be found wanting. Therefore, the following research hypotheses were considered:

Hypothesis 1: A factor representing the profit could

be added to the OLS function. It is called the Directional Profit (DP) model. The error function is:

( )( )∑ = −= Np poptpDPf

NDPE 12

2

1

( ) ( )( )poptsignptptFpDPf ΔΔ−−= ,,1

( )

⎪⎪

⎩

⎪⎪

⎨

⎧

>Δ<ΔΔ

≤Δ<ΔΔ

>Δ>ΔΔ

≤Δ>ΔΔ

=

σ

σ

σ

σ

ptandpoptifa

ptandpoptifa

ptandpoptifa

ptandpoptifa

pDPf

0*4

0*3

0*2

0*1

∑ = −= ⎟⎠⎞⎜

⎝⎛N

p ptn 11

μσ

where μ is the mean of the target series.

Hypothesis 2: A factor representing the time could be

added to the error function. It is the Discounted Least Squares (DLS) model.

( )( )∑ = −= Np poptpw

NDLSE 12

2

1 (12)

( )⎟⎟⎠

⎞⎜⎜⎝

⎛ −+

=

Napa

e

pw 2

1

1

(13)

The discount rate a will decide the function which actually is a function of the recency of the observation. We use a = 6 as suggested by Refenes et al. in fuzzy system application work [2].

Hypothesis 3: If profit and time are useful factors of

error function to improve the forecastability, an even better result could be achieved by using the combination of both of them. This is the Time dependent Directional Profit (TDP) model.

( )( )∑ = −= Np poptpTDPf

NTDPE 12

2

1 (14)

( ) ( )pwpDPfTDPf *= (15)

The weight changing is the same as the approach used

in the traditional OLS back-propagation except that an adjustment factor is introduced for both DP and TDP models.

Instead of using the OLS error function in Equation 5, EDP, EDLS is modified and the ETDP function is changed from batch mode to incremental mode before using it in application of the SAM fuzzy system. Tuning formulae for all parameters are therefore edited simply by replacing E in Equation 6 with modified EDP, EDLS, and ETDP values. The results of the advanced models are showed in Table 2.

(8)

5. Experimental results (9)

The two of time series data that are used in this study

are described as follows:

(10) Dataset 1: Title Temperatures in England Source Hipel and Mcleod (1994) Frequency Monthly Units oF (Fahrenheit) The number of samples

2976 (from the year of 1723 until 1970)

The number of test samples 500

(11)

263

Dataset 2:

Title Gold price

Source Institute for Economic Research – Ho Chi Minh City, Vietnam

Frequency Daily

Units Thousands of Vietnamese dong (VND)

The number of samples

1485 (from the year of 1996 until 2000)

The number of test samples 300

Computer configuration: Intel(R) Core(TM) CPU

6400 @ 2 x 2.13 GHz, 3.50 GB of RAM. The SAM parameters: the number of inputs: n = 4 (n in equation: xn+1 = f(x1, x2, …, xn)), clustering method: Adaptive Vector Quantitative Leader, parameter learning method: momentum – back-propagation, learning rate: = 0.001, momentum coefficient: ε = 0.9, the number of iteration cycles: t = 1000.

tμ

Table 1. Average error on test samples of applying selective presentation learning (SEL) and event-knowledge (EK).

Dataset 1 Dataset 2 Non-using SEL and EK 0.186 0.172

Using SEL 0.110 0.085 Using EK 0.094 0.051

Table 2. Average error on test samples of applying new training criteria for financial prediction.

Dataset 2 Ordinary Least Squares 0.172 Directional Profit 0.158 Discounted Least Squares 0.147 Time dependent Directional Profit 0.106

6. Conclusions SAM application with incorporation of genetic

algorithm, momentum technique, event-knowledge, selective presentation learning, and new training criteria (the latter for financial time-series data) significantly improved predictability, flexibility and accuracy when applying the SAM approach. This research showed that such a SAM application offers high potential for solving prediction problems. Human knowledge can be modeled

conveniently via use of if-then fuzzy rules. In testing the utility of incorporating the learning process, indicators are that both the SAM and the Artificial Neural Network approach to prediction using time series data can benefit.

Acknowledgements

We acknowledge collaboration with Mr. Hieu Ngoc Duong, Mr. Sang Thanh Do, and express gratitude for use of facilities at the Faculty of Information Technology, University of Technology, Vietnam National University Ho Chi Minh City, Vietnam. 7. References [1] A.E. Gaweda, Optimal data-driven rule extraction using adaptive fuzzy-neural models, Ph.D. Dissertation, University of Louisville, Kentucky, USA, 2002. [2] A.N. Refenes, Y. Bentz, D.W. Bunn, A.N. Burgess, and A.D. Zapranis, “Financial Time Series Modeling with Discounted Least Squares Backpropagation”, Neurocomputing, vol. 14, no. 2, 1997, pp. 123-138. [3] B. Kosko, “Fuzzy systems as universal approximators”, IEEE Transactions on Computers, vol. 43, no. 11, 1994, pp. 1329-1333. [4] B. Kosko, “Optimal fuzzy rules cover extrema”. International Journal of Intelligent Systems, vol. 10, no. 2, 1995, pp. 249-255. [5] B. Kosko, “Global stability of generalized additive fuzzy systems”, IEEE Transactions on Systems, Man, and Cybernetics, vol. 28, no. 3, 1998, pp. 441-452. [6] D.E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning, Addison Wesley, Massachusetts, USA, 1989. [7] D. Gao, Y. Kinouchi, K. Ito, and X. Zhao, “Neural networks for event extraction from time series: a back propagation algorithm approach”, Future Generation Computer Systems, vol. 21, no. 7, 2005, pp. 1096-1105. [8] E.S. Gardner, Jr. and E. McKenzie, “Model Identification in Exponential Smoothing”, Journal of the Operational Research Society, vol. 39, no. 9, 1988, pp. 863-867. [9] G.D. Magoulas, M.N. Vrahatis, and G.S. Androulakis, “Improving the Convergence of the Backpropagation Algorithm Using Learning Rate Adaptation Methods”, Neural Computation, vol. 11, 1999, pp. 1769-1796. [10] G.J. Klir and B. Yuan, Fuzzy Sets and Fuzzy Logic: Theory and Applications, Prentice Hall, Upper Saddle River, NJ, USA, 1995.

264

[11] I.P. Stoyanov, “An improved Backpropagation Neural Network Learning”, Proceedings of the 13

th International

Conference on Pattern Recognition, vol. 4, 1996, pp. 586-588. [12] J.A. Dickerson and B. Kosko, “Fuzzy function approximation with supervised ellipsoidal learning”, World Congress on Neural Networks, vol. 2, 1993, pp. 9-13. [13] J.A. Dickerson and B. Kosko, “Fuzzy function approximation with ellipsoidal rules”, IEEE Transactions Systems, Man, and Cybernetics, vol. 26, no. 4, 1996, pp. 542-560. [14] J.E. Fowler, Adaptive Vector Quantization for the Coding of Nonstationary Sources, SPANN Laboratory Technical Report TR-95-05, The Ohio State University, 1995. [15] J.T. Yao and C.L. Tan, “A Study on Training Criteria for Financial Time Series Forecasting”, Proceedings of International Conference on Neural Information Processing, 2001, pp. 772-777. [16] K. Alsabti, S. Ranka, and V. Singh, “An Efficient K-Means Clustering Algorithm”, IPPS/SPDP Workshop on High Performance Data Mining, IEEE Computer Society Press, 1998. [17] K. Kohara, “Neural Multivariate Prediction Using Event-Knowledge and Selective Presentation Learning”, IEEE International Conference on Neural Networks, vol. 1, 1995, pp. 359-364. [18] K. Kohara, T. Ishikawa, Y. Fukuhara and Y. Nakamura, “Stock Price Prediction Using Prior Knowledge and Neural Networks”, Intelligent Systems in Accounting, Finance and Management, vol. 6, no. 11, 1997, pp. 11-22. [19] L. Yao, K.S. Weng, and C.D. Huang, “A Fuzzy Classifier with Evolution Design of Ellipsoidal Decision Regions”, Proceedings of World Academy of Science, Engineering and Technology, vol. 1, 2005, pp. 449-453. [20] N. Qian, “On the momentum term in gradient descent learning algorithms”, Neural networks, vol. 12, 1999, pp. 145-151. [21] NIST/SEMATECH, Engineering Statistics e-Handbook, http://www.itl.nist.gov/div898/handbook/, 2006. [22] S. Mitaim and B. Kosko, “What is the best shape for a fuzzy set in function approximation?”, Proceeding of the 5

th

IEEE International Conference on Fuzzy Systems, vol. 2, 1996, pp. 1237-1243. [23] S. Mitaim and B. Kosko, “Adaptive joint fuzzy sets for function approximation”, Proceeding of IEEE International Conference on Neural Networks, vol. 1, 1997, pp. 537-542. [24] S. Mitaim and B. Kosko, “The shape of fuzzy sets in adaptive function approximation”, IEEE Transactions on Fuzzy Systems, vol. 9, no. 4, 2001, pp. 637-656.

[25] S.C. Newton, S. Pemmaraju, and S. Mitra, “Adaptive fuzzy leader clustering of complex data sets in pattern recognition”, IEEE Transactions on Neural Networks, vol. 3, no. 5, 1992, pp. 794-800. [26] T. Kanungo, D.M. Mount, N.S. Netanyahu, C.D. Piatko, R. Silverman, and A.Y. Wu, “An Efficient k-Means Clustering Algorithm: Analysis and Implementation”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 7, 2002, pp. 881-892. [27] T.T. Nguyen and S.T. Do, Applying Standard Additive Model for economic prediction problems, Bachelor Dissertation, Faculty of Information Technology, University of Technology, Vietnam National University Ho Chi Minh City, Vietnam, 2006. [28] X.B. Yan, T. Lu, Y.J. Li, and G.B. Cui, “Research on event prediction in time-series data”, Proceedings of International Conference on Machine Learning and Cybernetics, vol. 5, 2004, pp. 2874-2878.

265