View
228
Download
0
Tags:
Embed Size (px)
Citation preview
Hybrid Pipeline Structure for Hybrid Pipeline Structure for Self-Organizing Learning ArraySelf-Organizing Learning Array
Yinyin Liu1, Ding Mingwei2 , Janusz A. Starzyk1,
1 School of Electrical Engineering & Computer ScienceOhio University, USA
2 Ross University
ISNN 2007: The 4th International Symposium on Neural Networks
2
OutlineOutline
•RC systems design of SOLAR
•Dimensionality reduction
•Input selection, weighting
•Pipeline structure
• Experimental results
• Conclusions
Broca’sarea
Parsopercularis
Motor cortex Somatosensory cortex
Sensory associativecortex
PrimaryAuditory cortex
Wernicke’sarea
Visual associativecortex
Visualcortex
3
• “…Perhaps the last frontier of science – its ultimate challenge- is to understand the biological basis of consciousness and the mental process by which we perceive, act, learn and remember..” from Principles of Neural Science by E. R. Kandel et al.
E. R. Kandel won Nobel Price in 2000 for his work on physiological basis of memory storage in neurons.
• “…The question of intelligence is the last great terrestrial frontier of science...” from Jeff Hawkins On Intelligence. Jeff Hawkins founded the Redwood Neuroscience Institute devoted to brain research. He co-founded Palm Computing and Handspring Inc.
Intelligence
AI’s holy grailFrom Pattie Maes MIT Media Lab
4
How can we design intelligence?How can we design intelligence?
• We need to know how
• We need means to implement it
• We need resources to build and sustain its operation
5From Ray Kurzwail, The Singularity Summit at Stanford, May 13, 2006
Resources – Evolution of Electronics
6By Gordon E. MooreBy Gordon E. Moore
7
8From Ray Kurzwail, The Singularity Summit at Stanford, May 13, 2006
Clock Speed (doubles every 2.7 years)
9From Ray Kurzwail, The Singularity Summit at Stanford, May 13, 2006
10
OutlineOutline
•RC systems design of SOLAR
•Dimensionality reduction
•Input selection, weighting
•Pipeline structure
• Experimental results
• ConclusionsBroca’sarea
Parsopercularis
Motor cortex Somatosensory cortex
Sensory associativecortex
PrimaryAuditory cortex
Wernicke’sarea
Visual associativecortex
Visualcortex
11
Traditional ANN HardwareTraditional ANN HardwareTraditional ANN HardwareTraditional ANN Hardware
– Limited routing resource.
– Quadratic relationship between the routing and the number of neuron makes classical ANNs wire dominated.
input
output
information flow
hidden
Interconnect is Interconnect is 70% of chip area70% of chip area
12
Biological Neural NetworksBiological Neural Networks Biological Neural NetworksBiological Neural Networks
Cell body
From IFC’s webpage Dowling, 1998, p. 17
13
Sparse StructureSparse Structure
• 1012 neurons in human brain are sparsely connected
• On average, each neuron is connected to other neurons through about 104 synapses
• Sparse structure enables efficient computation and saves energy and cost
14
Why should we care?Why should we care?
Source: SEMATECHSource: SEMATECH
15
0%
20%
40%
60%
80%
100%
1999
2002
2005
2008
2011
2014
% Area Memory
% Area ReusedLogic
% Area New Logic
Percent of die area that must be occupied by memory to maintain SOC design productivity
Design Productivity Gap Design Productivity Gap Low-Value Designs? Low-Value Designs?
Source = Japanese system-LSI industry
16
OutlineOutline
•RC systems design of SOLAR
•Dimensionality reduction
•Input selection, weighting
•Pipeline structure
• Experimental results
• ConclusionsBroca’sarea
Parsopercularis
Motor cortex Somatosensory cortex
Sensory associativecortex
PrimaryAuditory cortex
Wernicke’sarea
Visual associativecortex
Visualcortex
17
SOLAR System DesignSOLAR System Design
• SOLAR Introduction Entropy based self-
organization
– data-driven
– Local connection Dynamical reconfiguration Local and sparse
interconnections Online inputs selection Feature neurons and
merging neurons Pattern recognition,
classification
18
Pipeline OverviewPipeline Overview
node computing ability → “soft” connections
Four modes
1. Idle2. Read3. Process4. Write
19
Pipeline Signal Flow 1Pipeline Signal Flow 1
20
Pipeline Signal Flow 2Pipeline Signal Flow 2
21
Pipeline Signal Flow 3Pipeline Signal Flow 3
22
Node OperationsNode Operations
Implemented with Xilinx picoBlaze
Runs at higher frequency
23
OutlineOutline
•RC systems design of SOLAR
•Dimensionality reduction
•Input selection, weighting
•Pipeline structure
• Experimental resultsExperimental results
• ConclusionsBroca’sarea
Parsopercularis
Motor cortex Somatosensory cortex
Sensory associativecortex
PrimaryAuditory cortex
Wernicke’sarea
Visual associativecortex
Visualcortex
24
Em(x) Simulation ResultsEm(x) Simulation Results
25
Iris Data ProcessingIris Data Processing
4x7 array processing Iris data
Linear growth of HW cost
26
Chip LayoutChip Layout
27
XILINX
XILINX
VIRTEX XCV 1000
VIRTEX XCV 1000
Hardware DevelopmentHardware Development
28
Future WorkFuture Work- System SOLAR- System SOLAR
29
Conclusions & Future workConclusions & Future work
• Sparse coding building in sparsely connected networks
• WTA scheme: local competition accomplish the global competition using primary and secondary layers –efficient hardware implementation
• OTA scheme: local competition produces neuronal activity reduction
• OTA – redundant coding: more reliable and robust
• WTA & OTA: learning memory for developing machine intelligence
Future work:
• Introducing temporal sequence learning
• Building motor pathway on such learning memory
• Combining with goal-creation pathway to build intelligent machine