Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
Large-scale neural models for cognition
Chris EliasmithCentre for Theoretical Neuroscience
University of Waterloo
Three roads to cognition(on Brainstorm)
• Symbolicism
• Connectionism
• Dynamicism
Unify
Spaun• Semantic pointer architecture unified
network (Spaun)
• 8 different percept/cognitive/motor tasks
• 2.5 million neurons
• 8 billion connections
• Uses NEF implemented in Nengo
• Does 8 tasks, including basic perceptual tasks…
And more cognitive tasks...
With no changes to Spaun between tasks...
(8 tasks include: recognition, copy drawing, reinforcement learning, counting, serial working memory, question answering, RVC, RPM)
NEF connection weights
Nengo 2.0
Spaun fly through
Semantic Pointer Architecture
• The SPA uses NEF building blocks for cognitive models
• Four elements described:
• Semantics
• Syntax
• Control
• Learning & memory
Biological Cognition
SPA elements• Communication
protocol:
• Semantic Pointers
• Specific functional claims:
• Motor/perception hierarchies
• Representing structure
• Action selection (BG)
SPA: Semantic Pointers• E.g. The pointer would be the activity of the top
level of a standard hierarchical visual model for object recognition
• This pointer can then support ‘symbol’ manipulation
• It can also be used toreactivate a full visual representation
Serre et al., 2007 PNAS
Surface/deep semantics• Applied to numbers: a) neuron tuning; b) generic SPs;
c) input; d) reconstruction; e) surface semanticsb)
a)a)
c) d)
e)
Charlie Tang
a)Data
Model
Semantic Pointers• Semantic pointers are: Compressed, content-based
‘addresses’ to information in association cortices
• ‘Pointer’ because they are used to recall ‘deep’ semantic information (content-based pointer)
• ‘Semantic’ because they themselves define a ‘surface’ semantic space
a bc
$€
%+=
ê
åëâï
Motor compression• SPs can be used to drive the motor control
hierarchy as a dereferencing operation
Cube Octahedron
8 vertices6 faces
8 faces6 vertices
a) b)Motor Control
Representa!on World
Perception
World Representa!on
World
World
Dom
inantInform
ation Flow
Cognitive compression•Rule: “If vowel, then even”
•Classically: implies(vowel,even)
•What is the antecedent?
Examples of structure
• Rules: implies(vowel, even)
• Concepts: dog
• Lists: 5,3,4,0
Action selection
•Mb compares the cortical state with known SPs•Mc maps selected action to cortical control states
Action selection
Putting it together: Serial working memory
How does it work?
• Compress:
• Map:
• Compress:
• Recurrence:
• Action:
• Decompress:
• Map:
• Decompress:
Perception Cognition Action
Extending Spaun• Spaun: Largely fixed rules in BG
• Built as an expert at a fixed set of tasks
• Limited flexibility for new tasks without model changes
• Two ways to learn new tasks:
• Trial and error
• Explicit instruction
Explicit Instruction
• Flexible perception/action mappings
• Very fast “learning” (to support slow learning)
• Applications:
• Service robots
• Team coordination
Task 1
Rule Manipulation
• Encoding rules:
• If vision=1 then write 3
• If vision=3 then write 9…
Rule Manipulation
• Decoding:
• vision = one
• Action to perform:
Task 2
Rule manipulation
• Encoding/decoding similar:
• Result is loaded into an “executive memory” that keeps track of the task being performed.
• Updates cortical control state instead of simple perception/action map.
Next steps
• Sets of instructions (multiple steps)
• Integrating instruction following into basal ganglia
• Instruct it to perform an entirely novel task
• Parsing instructions from visual stream
Other SPA models• Unifying concepts (Blouw et al, 2016)
• Human-scale conceptual structures (Crawford et al., 2015)
• Instruction following parsing (Choo et al, 2015)
• Intelligence test model (Rasmussen & Eliasmith, 2014)
• Speech perception and generation (Bekolay, 2016)
• Hierarchical reinforcement learning (Rasmussen, 2014)
• N-back task (Gosmann & Eliasmith, 2015)
• Language parsing (Blouw & Eliasmith, 2015)
• Language sequencing (Kroger et al, 2016)
SPA Unifies• Perception (Connectionism)
• Statistical categorization, SPs
• Cognition (Symbolicism)
• Working with structure, SPs
• Action (Dynamicism)
• Brain/body dynamics, SPs & control
Further informationResearch, Papers: http://compneuro.uwaterloo.ca
Nengo, Tutorials, Spaun videos: http://www.nengo.ca
CNRG lab: Terry Stewart, Eric Hunsberger, Brent Komer, Aaron Voelker, Xuan Choo, Sean Aubin, Sugandha Sharma, Mariah Martin-Shein, Peter Blouw, Stacy Gaikovaia, Jan Gosmann, Ivana Kajic, Peter Duggins