Neural Information SystemsANSER :Rainfall Estimating System

THONN:Financial Date Simulation System

FACEFLOW: Face Recognition system System

Dr. Ming Zhang, Associate ProfessorDepartment of Physics, Computer Science & Engineering

Christopher Newport University

1 University Place, Newport News, VA 23606, USA

Dr. Ming Zhang

Dr. Ming Zhang 11/1999 – 07/2000: Senior USA NRC Research Associate

NOAA, Funding $70,000. 03/1995 – 11/1999: Ph.D. Supervisor and Senior Lecturer

University of Western Sydney

Fundings: A$203,724 Cash from Fujitsu, ARC, & UWSM 07/1994-03/1995: Ph.D. Supervisor and Lecturer

Monash University, A$50,000 Grant from Fujitsu) 11/1992-07/1994: Project Manager & P.H.D. Supervisor

University of Wollongong, (A$850,000 from SITA) 07/1991-10/1992: USA NRC Postdoctoral Fellow

NOAA, Funding: US$100,000) 07/1989-06/1991: Associate Professor and Postdoctoral Fellow

The Chinese Academy of the Sciences. Funding: RMB$2,000,000

Dr. Ming Zhang

Artificial Neural Network Techniques For Estimating Heravy Rainfall

From Satellite Data

Ming Zhang, Roderick A. Scofield

NOAA/NESDIS/ORA5200 Auth Road, Room 601

Camp Springs, MD 20746, USAEmail: roderick.scofield@noaa.gov

Page 1

Page 2

ANSER System Interface

Artificial Neural network expert System for Estimation of Rainfall from the satellite data

ANSER System (1991-2000)

- 1991-1992:US$66,000 suported by USA National Research Council & NOAA

- 1995-1996:A$11,000 suppouted by Australia Research Council& NOAA

- 1999-2000:US$62,000 suported by USA National Research Council & NOAA

Page 3

Why Develop ANSER ?

- More than $3.5 billion in property is damaged and, more than 225 people are killed by heavy rain and flooding each year

- No rainfall estimating system in GIS system, No real time and working system of rainfall estimation in the world

- Can ANN be used in the weather forecasting area? If yes, how should we use ANN techniques in this area?

Page 4

Why Use Neural Network Techniques ?

- Two Directions of New generation computer Quamtun Computer Artificial Neural Network- Much quicker speed ?- Complicated pattern recognition?- Unknown rule knowledge base?- Self learning reasoning network?- Super position for multip choice?

Page 5

Page 26

ANSER Rainfall Estimation Result

(May 2000)

Time LAT LAN ANSER GAGEMin 0918 37.032 87.906 1.47mm 2.0mmMax 0918 38.765 88.480 6.37mm 6.0mmMin 1207 42.866 90.959 2.45mm 2.0mmMax 1207 42.837 87.403 9.31mm 12.0mmMin 2306 36.599 85.463 0.98mm 1.8mmMax 2306 38.621 86.936 8.82mm 9.0mmMin 2406 34.144 84.349 7.10mm 6.0mmMax 2406 37.148 88.696 27.69mm 33.0mm

Page 29

ANSER Rainfall Estimation Result

9th May 2000Time: 18Z

LAT LANMin 37.032 87.906Max 38.765 88.480

ANSERMin: 1.47 mmMax: 6.37mm

NAVY Min: 2.0mmMax: 6.0mm

Page 30

ANSER Rainfall Estimation Result

12th May 2000Time: 07Z

LAT LANMin 42.866 90.959Max 42.837 87.403

ANSERMin: 2.45 mmMax: 9.31mm

Gage Min: 2.0mmMax: 12.0mm

Page 31

ANSER Rainfall Estimation Result

23th May 2000Time: 06Z

LAT LANMin 36.599 85.463Max 38.621 86.936

ANSERMin: 0.98 mmMax: 8.82mm

Gage Min: 1.8mmMax: 9.0mm

Page 32

ANSER Rainfall Estimation Result24th May 2000Time: 06Z

LAT LANMin 34.144 84.349Max 37.148 88.696

ANSERMin: 7.10 mmMax: 27.69mmGage Min: 6.0mmMax: 23.0mmNAVAMin: 7.0mmMax: 33.0mm

Conclusion- What Approved

Artificial Neural Network Techniques can :

- Much quick speed: 5-10 time quick

- Complicated pattern recognition: cloud merger

- Unknown rule knowledge base: Rainfall

- Reasoning network: rainfall estimation

Page 27

Conclusion- Next Step- Rebuild interface & retraining neural networks

- New neural netowrk models:

more complicated pattern recognition

- Self expending knowledge base:

attract knowledge from real time cases

- Self learning reasoning network: automatic system to

- Study in advance in 15 years: Artificial Neural Network - one of two directions of new generation computer Research

Page 28

Using PT-Honn Models For Multi-polynomial Function Simulation

Bo Lu, Hui Qi, Ming Zhang

University of Western Sydney Macarthur

Campbelltown, NSW 2560, Australia

Roderick A. Scofield

NOAA/NESDIS/ORA

5200 Auth Road,Camp Springs, MD 20746, USA

PHONN Simulator (1994 - 1996)- Polynomial Higher Order Neural Network financial data

simulator

- A$ 105,000 Supported by Fujitsu, Japan

THONN Simulator (1996 - 1998)- Trigonometric polynomial Higher Order Neural Network

financial data simulator

- A$ 10,000 Supported by Australia Research Council

PT-HONN Simulator (1999 - 2000)- Polynomial and Trigonometric polynomial Higher Order

Neural Network financial data simulator

- US$ 46,000 Supported by USA National Research Council

PT-HONN Data Simulator

Simulating by PT-HONN Simulator

Structure of PT-HONN

Page 17

Cloud Merger Operator

MI(i, j) is a black-and-white image which can be used to represent a satellite image.

Label Set: L = {0, 1, 2, …… M}

Each label corresponds to a different cloud merger.

Cloud Merger Recognising operator CMR

CMR: MI(i, j) L

Page 7

Cloud Merger Operator Set The cloud merger recognising operator CMR

is the operator set: CMR = { CMCI, CMR1, CMR2, CMS1,CMS2,CMS3,CMS4, CMM1,CMM2,CMM3,CMM4} Where CMCI: Circle input satellite data cloud

merger recognising operator. …...

Page 8

Ternary Output of Cloud Merger Operator

1, O(Ns,t) 1 – cloud merger L= 2, O(Ns,t) 3 – further test needed 0, O(Ns,t) 2 - cloud not merger where s-th level is the output layer of NN. All other operators (CMR1, CMR2, CMS1,

CMS2, CMS3, CMS4,CMM1, CMM2, CMM3, CMM4) have the same definitions as CMCI.

Page 9

Cloud Merge Using ANN Circle Operator

Page 10

Cloud Merge Using ANN S-Shape 4 Operator

Page 11

Results of Cloud Merger Operator ANN Operator No Merger Merger Circle 0.013398 0.988092 Rectangle 1 0.017790 0.984671 Rectangle 2 0.005251 0.977073 S-Shape 1 0.010566 0.973505 S-Shape 2 0.024652 0.932447 S-Shape 3 0.009757 0.934428 S-Shape 4 0.032293 0.752404 Moon-Shape 1 0.005602 0.967767 Moon-Shape 2 0.038606 0.909732 Moon-Shape 3 0.366631 0.953632 Moon-Shape 4 0.005530 0.932430

Page 12

PT-HONN MODEL

The network architecture of PT-HONN has combined both the characteristics of PHONN and THONN.

It is a multi-layer network that consists of an input layer with input-units, and output layer with output-units, and two hidden layers consisting of intermediate processing units.

Page 15

Definition of PT-HONN

Page 16

))()(

)()((0,

ySinxCosc

yxbyCosxSinaZ

ji

ij

ji

ij

jn

ji

i

ij

Knowledge of Rainfall Half Hour Rainfall Inches

Cloud Top Cloud Growth Latitude Degree

Temperature 2/3 1/3 0

> -32 C 0.05 0.05 0.03 -36 C 0.20 0.13 0.06 -46 C 0.48 0.24 0.11 -55 C 0.79 0.43 0.22 -60 C 0.94 0.65 0.36 -70 C 1.55 0.85 0.49 <-80 C 1.93 0.95 0.55

Page 18

PT-HONN ResultsCloud Top Cloud Growth PHONN PT_HONNTemperature Latitude Degree |Erro|r% |Error| % …… …… ……. …... > - 32 C 1/3 10.47 10.11 - 36 C 1/3 3.50 4.25 - 46 C 1/3 3.52 4.63 - 55 C 1/3 0.22 2.04 - 60 C 1/3 3.21 0.30 - 70 C 1/3 9.01 5.08 < - 80 C 1/3 3.89 1.21 …… …… …… …... Average 6.36% 5.68%

Page 19

CONCLUSION

The results of the comparative experiments show that THONG system is able to simulate higher frequency and higher order non-linear data, as well as being able to simulate discontinuous data.

The THONG model can not only be used for financial simulation, but also for financial prediction.

Using THONG System for Higher Frequency Non-liner Data Simulation & Prediction

FACEFLOW A Robot Vision System

For Moving Face Recognition

Dr. Ming Zhang

FACEFLOW (1992 - 2000) A computer vision system for recognition of

3-dimensional moving faces using GAT model (neural netowrk Groug-based Adaptive tolerance Tree)

A$850,000 supported by SITA (Society Internationale de Telecommunications Aeronautiques)

A$40,500 supported by Australia Research Council A$78,000 supported by Australia Department of

Education. US$160,000 supported by USA National Research

Council.

Page 20

Neuron-Adaptive Neural Network Simulator

* The network architecture of NANN is a multilayer feed-forward network that consists of an input layer with input-units, an output layer with output-units, and one hidden layer consisting of intermediate processing units.* There is no activation function in the input layer and the output neurones are summing units (linear activation)* our activation function for the hidden layer processing units is a Neuron-Adaptive Activation Function (NAAF)

NANN

Page 21

The activation function for the hidden layer processing units is a Neuron-Adaptive Activation Function (NAAF) defined as

where a1,b1,a2,b2,a3 and b3 are real variable which will be adjusted (as well as weights) during training.

xb

xb

ea

eaxbax

3

2

13

21sin1

NAAF

Page 22

Structure of NANN

Figure 1. An NANN with NAAF's for hidden layer

Input

NAAF's

Output

Page 23

NANN Group

Neuron-Adaptive Feedforward Neural network Group (NAFNG) is one kind of neural network group in which each element is a neuron-adaptive feedforward neural network (Fi). We have:

NAFNG ={F1, F2, F3,…... Fi,…...Fn}

Page 24

Feature of NANN Hornik (1991): If the activation function is

continuous, bounded and nonconstant, then standard FNN can approximate any continuous function.

Leshno (1993): A standard FNN can approximate any continuous function if the network's activation function is not a polynomial.

A neuron-adaptive feedforward neural network group with adaptive neurones can approximate any kind of piecewise continuous function.

Page 25

Neuron Network Group Models -complex system GAT Tree Model

- real time and real world face recognition Neuron-Adaptive Neural Network Models

- best match real world data Center Of Motion Model - motion center Second Order Vision Model - motion direction NAAT Tree Model - a possible more powerful

model for face recognition

FACEFLOW: A Robot Vision System

Hornik, K. (1991)

“Whenever the activation function is continuous, bounded and nonconstant, then for an arbitrary compact subset X Rn, standard multilayer feedforward networks can approximate any continuous function on X arbitrarily well with respect to uniform distance, provided that sufficiently many hidden units are available”

Leshno, M. (1993)

“A standard multilayer feedforward network with a locally bounded activation function can approximate any continuous function to any degree of accuracy if and only if the network’s activation function is not a polynomial”

Zhang, Ming (1995)

“ Consider a neural network piecewise function group, in which each member is a standard multilayer feedforward neural network, and which has locally boundded, piecewise continuous (rather than polynomial) activation function and threshold. Eash such group can approximate any king of piecewise continuous function, and to any degree of accuracy”

Knowledge of Rainfall Half Hour Rainfall Inches

Cloud Top Cloud Growth Latitude Degree

Temperature 2/3 1/3 0

> -32 C 0.05 0.05 0.03 -36 C 0.20 0.13 0.06 -46 C 0.48 0.24 0.11 -55 C 0.79 0.43 0.22 -60 C 0.94 0.65 0.36 -70 C 1.55 0.85 0.49 <-80 C 1.93 0.95 0.55

PT-HONN ResultsCloud Top Cloud Growth PHONN PT_HONNTemperature Latitude Degree |Erro|r% |Error| % …… …… ……. …... > - 32 C 1/3 10.47 10.11 - 36 C 1/3 3.50 4.25 - 46 C 1/3 3.52 4.63 - 55 C 1/3 0.22 2.04 - 60 C 1/3 3.21 0.30 - 70 C 1/3 9.01 5.08 < - 80 C 1/3 3.89 1.21 …… …… …… …... Average 6.36% 5.68%

Conclusion- What Approved

Artificial Neural Network Techniques can :

- Much quick speed: 5-10 time quick

- Complicated pattern recognition: cloud merger

- Unknown rule knowledge base: Rainfall

- Reasoning network: rainfall estimation

Conclusion- Next Step- Rebuild interface & retraining neural networks

- New neural netowrk models:

more complicated pattern recognition

- Self expending knowledge base:

attract knowledge from real time cases

- Self learning reasoning network: automatic system to

- Study in advance in 15 years: Artificial Neural Network - one of two directions of new generation computer Research