A Data Simulation System Using sinx/x and sinx Polynomial Higher Order Neural Networks

A Data Simulation System Using sinx/x and sinx

Polynomial Higher Order Neural Networks

Dr. Ming ZhangDepartment of Physics, Computer Science and

EngineeringCollege of Liberal Arts and Sciences

Christopher Newport UniversityThis research is supported by

Christopher Newport University 2004 Summer Stipend grant

CNU Applied Research Centre 2004 research funding.

SUMMARY Traditional methods of data simulation are high

inaccurate. The solution to choice of the optimal neural network

model for simulation and prediction is still not found. In this paper a data simulation system based on the

sinx/x and sinx Polynomial Higher Order Neural Network (SXSPHONN) has been developed.

The paper also proved that SXSPHONN models are convergence when simulate XOR data.

Rainfall data has been tested in the paper. SXSPHONN model is 0.999% better than Polynomial

Higher Order Neural Network (phonn) model and 0.967% batter than Trigonometric Polynomial Higher Order Neural Network (thonn) model in our rainfall case.

1 INTRODUCTION (1)

Currently, satellite derived precipitation estimates [1] and 3 hour precipitation outlooks for extra-tropical cyclones, and tropical cyclones are computed on the NOAA/NESDIS Interactive Flash Flood Analyser (IFFA) system and transmitted to National Weather Service Forecast Offices, and River Forecast Centres.

1 INTRODUCTION (2)

Artificial neuron network-based models are not yet sufficiently powerful to characterize complex systems. Moreover, a gap exists in the research literature between complex systems and general systems. To characterize complex systems, neuron-based and neural network-based group models are studied.

1 INTRODUCTION (3)

Zhang, Murugesan, and Sadeghi (1995) developed a Polynomial Higher Order Neural Network (PHONN)[12] for data simulation. Ming Zhang, Jing Chun Zhang, and Steve Keen [13] developed a Trigonometric Polynomial Higher Order Neural Network (THONN) system for higher frequency non-linear data simulation and prediction. Lu and Zhang [14] developed a model, called Polynomial and Trigonometric polynomial Higher Order Neural Network (PT-HONN).

1 INTRODUCTION (4)

This paper will develop a new higher order neural network model called sinx/x and sinx Polynomial Higher Order Neural Network (SXSPHONN) model. The sinx/x and sinx functions have been used as neuron activity functions.

For testing SXSPHONN model, a SXSPHONN simulator has been built.

XOR data will be used to test SXSPHONN model convergence.

The comparison experimental results between PHONN, THONN and SXSPHONN models for heavy rainfall estimation will be studied here.

2 SXSPHONN MODEL (1)

The network architecture of SXSPHONN is developed based on the characteristics of THONN models.

The different part of SXSPHONN is, SXSPHONN used sinx/x and sinx functions for the neurons.

It is a multi-layer network that consists of an input layer with input-units, and output layer with output-units, and two hidden layers consisting of intermediate processing units.

2 SXSPHONN MODEL (2) The specify definition of

SXSPHONN is presented in the following:

)))(sin()/)(sin((0,

jiij

n

ji

yxxaZ

2 SXSPHONN MODEL (3)

For a output layer neuron, we have:

)/()()1( okjp

okj

okj wEtwtw

2 SXSPHONN MODEL (4)

For the hidden layer neurons, the weight update equations are formulated as follows:

)/()()1( okjp

okj

okj wEtwtw

2 SXSPHONN MODEL (5)

sinx/x

-0.4-0.2

00.20.40.60.8

11.2

x

sin

x/x

-3π -2π -π 0 π 2π 3π

3. SXSPHONN SIMULATOR

4. XOR TEST (1)

Input 1 Input 2 Output

0 0 1

0 1 0

1 0 0

1 1 1

4. XOR TEST (2)

XOR Input Graph

4. XOR TEST (3)XOR Convergence (12628 epochs)

4. XOR TEST (4)Weights for XOR

4. XOR TEST (5)XOR Final Report from SXSPHONN Simulator

5. HEAVY RAINFALL ESTIMATING (1)

Cloud Top Temperature

Cloud Growth Latitude Degree

Half Hour Rainfall Inches

Cloud Top Temperature

Cloud Growth Latitude Degree

Half Hour Rainfall Inches

> -32 C 2/3 0.05 - 70 C 1/3 0.85 -36 C 2/3 0.20 < - 80 C 1/3 0.95 -46 C 2/3 0.48 > -32 C 0 0.03 -55 C 2/3 0.79 - 36 C 0 0.06 -60 C 2/3 0.94 - 46 C 0 0.11 > - 32 C 1/3 0.05 - 55 C 0 0.22 - 36 C 1/3 0.13 - 60 C 0 0.36 - 46 C 1/3 0.24 -70 C 0 0.49 - 55 C 1/3 0.43 < -80 C 0 0.55 - 60 C 1/3 0.65

Cloud Top Temperature, Cloud Growth, and Rainfall Inches

5. HEAVY RAINFALL ESTIMATING (2)

SXSPHONN: Rainfall Input Data Graph

5. HEAVY RAINFALL ESTIMATING (3)

SXSPHONN: Rainfall Data Convergence (1000 Epochs)

5. HEAVY RAINFALL ESTIMATING (4)

SXSPHONN: Weights for Rainfall

5. HEAVY RAINFALL ESTIMATING (5)

SXSPHONN: Rainfall Estimation Report

5. HEAVY RAINFALL ESTIMATING (6)

PHONN: Rainfall Data Convergence

5. HEAVY RAINFALL ESTIMATING (7)

PHONN: Rainfall Estimation Report

5. HEAVY RAINFALL ESTIMATING (8)

THONN: Rainfall Data Convergence

5. HEAVY RAINFALL ESTIMATING (9)

THONN: Rainfall Estimation Result

6. CONCLUSION

SXSPHONN models are studied in this paper. Using SXSPHONN model, heavy rainfall

estimation could have better results than PHONN and THONN higher order neural network model.

As the next step, the research will more focus on developing automatic higher order neural network models.

ACKNOWLEDGE

The author would like to thank Prof. Douglas Gordon,

Dean of College of Liberal Arts and Science CNU, and

Prof. A. Martin Buoncristiani, Chair of Department of Physics, Computer

Science and Engineering CNU,

for their great support and funding.

REFERENCE R. A. Scofield, The NESDIS operational convective precipitation

estimation technique, Monthly Weather Review, 115, 1982, 1773-1792.

Vicente, G., Scofield, R. A., and Menzel, W. P., The operational GOED infrared rainfall estimation techniques, Bulletin of the American Meteorological Society, 79(9), 1998,1883-1898.

D. R. Lippmann, An introduction to computing with neural nets, IEEE Acoustics, Speech and Signal Processing, 4 (2), 1987, 4-22.

T. Tsao, A group theory approach to neural network computing of 3D rigid motion. Proc. Intl. Joint Conf. Neural Networks. 2, 1989, 275-280.

L. Vecci, F. Piazza, A. Uncini, Learning andapproximation capabilities of adaptive spline activation function neural networks, Neural Networks, 11, 1998, 259-270.

C. T. Chen, W. D. Chang, A feedforward neural network with function shape autotuning, Neural Networks, 9(4), 1996, 627-641.

P. Campolucci, F. Capparelli, S. Guarnieri, F. Piazza, & A. Uncini, Neural networks with adaptive spline activation function. Proceedings of IEEE MELECON 96, Bari, Italy, 1996, 1442-1445.