EE1411

Design of Self-Organizing Learning Design of Self-Organizing Learning Array for Intelligent MachinesArray for Intelligent Machines

Janusz StarzykSchool of Electrical Engineering and Computer Science

Heidi Meeting June 3 2005

Motivation:How a new understanding of the brain will lead to the creation of truly intelligent machines

from J. Hawkins “On Intelligence”

EE1412

Abstract thinking and action planning Capacity to learn and memorize useful things Spatio-temporal memories Ability to talk and communicate Intuition and creativity Consciousness Emotions and understanding others Surviving in complex environment and adaptation Perception Motor skills in relation to sensing and anticipation

Elements of IntelligenceElements of Intelligence

EE1413

Problems of Classical AIProblems of Classical AI

Lack of robustness and generalization No real-time processing Central processing of information by a

single processor No natural interface to environment No self-organization Need to write software

EE1414

Intelligent BehaviorIntelligent Behavior

Emergent from interaction with environment Based on large number of sparsely connected

neurons Asynchronous Self-timed Interact with environment through sensory-

motor system Value driven Adaptive

EE1415

Design principles of intelligent systemsDesign principles of intelligent systems

Design principlessynthetic methodologytime perspectivesemergencediversity/complianceframe-of-reference

from Rolf Pfeifer “Understanding of Intelligence”

Agent designcomplete agent principle

cheap design

ecological balance

redundancy principle

parallel, loosely coupled processes

sensory-motor coordination

value principle

EE1416

The principle of “cheap design”The principle of “cheap design”

intelligent agents: “cheap” exploitation of ecological

niche economical (but redundant) exploitation of specific

physical properties of interaction with real world

EE1417

Principle of “ecological balance”Principle of “ecological balance”

balance / task distribution between morphology neuronal processing (nervous

system) materials environment

balance in complexity given task environment match in complexity of sensory,

motor, and neural system

EE1418

The redundancy principleThe redundancy principle

redundancy prerequisite for adaptive behavior

partial overlap of functionality in different subsystems

sensory systems: different physical processes with “information overlap”

EE1419

Generation of sensory stimulation Generation of sensory stimulation through interaction with environmentthrough interaction with environment

multiple modalities constraints from

morphology and materials

generation of correlations through physical process

basis for cross-modal associations

EE14110

The principle of sensory-motor The principle of sensory-motor coordinationcoordination

self-structuring of sensory data through interaction with environment

physical process —not „computational“

prerequisite for learning

Holk Cruse•no central control•only local neuronal communication•global communication through environment

neuronal connections

EE14111

The principle of parallel, loosely The principle of parallel, loosely coupled processescoupled processes

Intelligent behavior emergent from agent-environment interaction

Large number of parallel, loosely coupled processes

Asynchronous Coordinated through agent’s –sensory-motor system–neural system–interaction with environment

EE1411212

Human Human Brain Brain at Birthat Birth 6 Years Old6 Years Old

14 Years 14 Years OldOld

Neuron Structure and Self-Organizing Principles

EE14113

Neuron Structure and Self-Neuron Structure and Self-Organizing Principles (Cont’d)Organizing Principles (Cont’d)

EE14114

Broca’sarea

Parsopercularis

Motor cortex Somatosensory cortex

Sensory associativecortex

PrimaryAuditory cortex

Wernicke’sarea

Visual associativecortex

Visualcortex

Brain OrganizationBrain Organization

While we learn its functionscan we emulate its operation?

EE14115

V. Mountcastle argues that all regions of the brain perform the same algorithm

SOLAR combines many groups of neurons (minicolumns) in a pseudorandom way

Each microcolumn has the same structure Thus it performs the same computational

algorithm satisfying Mountcastle’s principle

VB Mountcastle (2003). Introduction [to a special issue of Cerebral Cortex on columns]. Cerebral Cortex, 13, 2-4. 

Minicolumn Organization and Minicolumn Organization and Self Organizing Learning ArraysSelf Organizing Learning Arrays

EE14116

“The basic unit of cortical operation is the minicolumn … It contains of the order of 80-100 neurons except in the primate striate cortex, where the number is more than doubled. The minicolumn measures of the order of 40-50 m in transverse diameter, separated from adjacent minicolumns by vertical, cell-sparse zones … The minicolumn is produced by the iterative division of a small number of progenitor cells in the neuroepithelium.” (Mountcastle, p. 2)

 Stain of cortex in planum temporale.

Cortical MinicolumnsCortical Minicolumns

EE14117

 

Groupings of minicolumns seem to form the physiologically observed functional columns. Best known example is orientation columns in V1.

They are significantly bigger than minicolumns, typically around 0.3-0.5 mm and have 4000-8000 neurons

Mountcastle’s summation:

“Cortical columns are formed by the binding together of many minicolumns by common input and short range horizontal connections. … The number of minicolumns per column varies … between 50 and 80. Long range intracortical projections link columns with similar functional properties.” (p. 3)

Groupping of MinicolumnsGroupping of Minicolumns

EE14118

Sparse ConnectivitySparse Connectivity The brain is sparsely connected.

(Unlike most neural nets.) A neuron in cortex may have on the order of 100,000 synapses.

There are more than 1010 neurons in the brain. Fractional connectivity is very low: 0.001%.

Implications:  Connections are expensive biologically since they take up

space, use energy, and are hard to wire up correctly. Therefore, connections are valuable. The pattern of connection is under tight control. Short local connections are cheaper than long ones.

Our approximation makes extensive use of local connections for computation.

EE14119

Introducing Self-Organizing Introducing Self-Organizing Learning Array SOLARLearning Array SOLAR

SOLAR is a regular array of identical processing cells, connected to programmable routing channels. Each cell in the array has ability to self-organize by adapting its functionality in response to information contained in its input signals. Cells choose their input signals from the adjacent routing channels and send their output signals to the routing channels.Processing cells can be structured to implement minicolumns

EE14120

SOLAR Hardware ArchitectureSOLAR Hardware Architecture

EE14121

SOLARSOLAR Routing SchemeRouting Scheme

EE14122

PCB SOLARPCB SOLAR

XILINX

XILINX

VIRTEX XCV 1000

VIRTEX XCV 1000

EE14123

System SOLARSystem SOLAR

EE14124

Wiring in SOLARWiring in SOLAR

Initial wiring and final wiring selection for credit card approval problem

EE14125

SOLAR Classification ResultsSOLAR Classification Results

EE14126

Associative SOLARAssociative SOLAR

EE14127

Associations made in SOLARAssociations made in SOLAR

EE14128

Brain Structure with Value System PropertiesBrain Structure with Value System Properties

Interacts with environment through sensors and actuators

Uses distributed processing in sparsely connected neurons organized in minicolumns

Uses spatio-temporal associative learning Uses feedback for input prediction and screening

input information for novelty Develops an internal value system to evaluate its

state in environment using reinforcement learning Plans output actions for each input to maximize the

internal state value in relation to environment Uses redundant structures of sparsely connected

processing elements

EE14129

Sensors Actuators

Value System

Anticipated ResponseReinf. Signal

Sensory Inputs

Motor Outputs

ActionPlanning

Understanding ImprovementDetection

Expectation

NoveltyDetection

Inhibition Comparison

Possible Minicolumn Organization Possible Minicolumn Organization

EE14130

Learning should be restricted to unexpected situation or reward

Anticipated response should have expected value Novelty detection should also apply to the value

system Need mechanism to improve and compare the value Anticipated response block should learn the response

that improves the value A RL optimization mechanism may be used to learn

the optimum response for a given value system and sensory input

Random perturbation should be applied to the optimum response to explore possible states and learn their the value

New situation will result in new value and WTA will chose the winner

Postulates for Minicolumn OrganizationPostulates for Minicolumn Organization

EE14131

Minicolumn Selective ProcessingMinicolumn Selective Processing

Sensory inputs are represented by more and more abstract features in the sensory inputs hierarchy

Possible implementation is to use winner takes all or Hebbian circuits to select the best match

“Sameness principle” of the observed objects to detect and learn feature invariances

Time overlap of feature neuron activation to store temporal sequences

Random wiring may be used to preselect sensory features

Uses feedback for input prediction and screening input information for novelty

Uses redundant structures of sparsely connected processing elements

EE14132

Minicolumn OrganizationMinicolumn Organization

Positive Reinforcement

Negative Reinforcement

Sensory Inputs

Motor Outputs

Sensory

Value

Motor

superneuron

EE14133

Sensory neurons are primarily responsible for providing information about environment They receive inputs from sensors or other sensory neurons on lower level They interact with motor neurons to represent action and state of

environment They provide an input to reinforcement neurons They help to activate motor neurons

Motor neurons are primarily responsible for activation of motor functions They are activated by reinforcement neurons with the help from sensory

neurons They activate actuators or provide an input to lower level motor neurons They provide an input to sensory neurons

Reinforcement neurons are primarily responsible for building the internal value system They receive inputs from reinforcement learning sensors or other

reinforcement neurons on lower level They receive inputs from sensory neurons They provide an input to motor neurons They help to activate sensory neurons

Minicolumn OrganizationMinicolumn Organization

EE14134

Sensory Neurons FunctionsSensory Neurons FunctionsSensory neurons Represent inputs from environment by

Responding to activation from lower level (summation)Selecting most likely scenario (WTA)

Interact with motor functions byResponding to activation from motor outputs (summation)

Anticipate inputs and screen for novelty byCorrelation to sensory inputs from higher levelInhibition of outputs to higher level

Select useful information byCorrelating its outputs with reinforcement neurons

Identify invariances byMaking spatio-temporal associations between neighbor sensory neurons

WTAWTA

WTA