The projects or efforts depicted were or are sponsored by the U.S. Army, AFOSR and ONR. The content or information presented does not necessarily reflect the position or the policy of the Government, and no official endorsement should be inferred.

Sigma: Towards a Graphical Architecture for Integrated Cognition

Paul S. Rosenbloom | 7/27/2012

2

The Goal of this Work

A new cognitive architecture – Sigma (𝚺) – based on– The broad yet theoretically elegant power of graphical models– The unifying potential of piecewise continuous functions

As an approach towards integrated cognition– Consolidating the functionality and phenomena implicated in

natural minds/brains and/or artificial cognitive systems That meets two general desiderata

– Grand unified– Functionally elegant

In support of developing functional and robust virtual humans (and intelligent agents/robots)– And ultimately relating to a new unified theory of cognition

3

Example Virtual Humans (USC/ICT)

Ada & Grace

SASO

Gunslinger

INOTS

4

USC/ICT – SASO USC/ISI & UM – IFOR

Cognitive Architecture

Symbolic working memory (x1 ^next x2)(x2 ^next x3)

Long-term memory of rules (a ^next b)(b ^next c)(a ^next c)

Decide what to do next based on preferences generated by rules

Reflect when can’t decide Learn results of reflection Interact with worldSoar 3-8 (CMU/UM/USC)

Fixed structure underlying intelligent behavior– Defines mechanisms for memory, reasoning, learning, interaction, etc.– Intended to yield integrated cognition when add knowledge and skills– May serve as the basis for

A Unified Theory of Cognition Virtual humans, intelligent agents and robots

Induces a language, but not just a language (or toolkit)– Embodies theory of, and constraints on, parts and their combination

Overlaps in aims with what are variously called AGI architectures and intelligent agent/robot architectures

Examples include ACT-R, AuRA, Clarion, Companions, Epic, Icarus, MicroPsi, OpenCog, Polyscheme, RCS, Soar, and TCA

5

Outline of Talk

Desiderata

Sigma’s core

Progress

Wrap up

6

DESIDERATA

7

Unified: Cognitive mechanisms work well together– Share knowledge, skills and uncertainty– Provide complementary functionality

Grand Unified: Extend to non-cognitive aspects– Perception, motor control, emotion, personality, …– Needed for virtual humans, intelligent robots, etc.

Forces important breadth up front– Mixed: General symbolic reasoning with pervasive

uncertainty– Hybrid: Discrete and continuous

Towards synergistic robustness– General combinatoric models– Statistics over large bodies of data

Desideratum I: Grand Unified

Expansive base for mechanism

development and integration

8

Soar 3-8

Hybrid Mixed Short-Term Memory

Learning

Hybrid Mixed Long-Term Memory

Sigma

Decision

Soar 9 (UM)

Broad scope of functionality and applicability– Embodying a superset of existing architectural capabilities

(cognitive, perceptuomotor, emotive, social, adaptive, …) Simple, maintainable, extendible & theoretically elegant

– Functionality from composing a small set of general mechanisms

Desideratum II: Functionally Elegant

9

Candidate Bases for Satisfying Desiderata

Programming languages (C, C++, Java, …)– Little direct support for capability implementation or integration

AI languages (Lisp, Prolog, …)– Neither hybrid nor mixed, nor supportive of integration

Architecture specification languages (Sceptic, …)– Neither hybrid nor mixed, nor sufficiently efficient

Integration frameworks (Storm, …)– Nothing to say about capability implementation

Neural networks– Symbols still difficult, as is achieving necessary capability breadth

Statistical relational languages (Alchemy, BLOG, …)– Exploring a variant tuned to architecture implementation and integration

Based on graphical models with piecewise continuous functions

10

SIGMA’S CORE

11

Enable efficient computation over multivariate functions by decomposing them into products of subfunctions– Bayesian/Markov networks, Markov/conditional random fields, factor graphs

Yield broad capability from a uniform base– State of the art performance across symbols, probabilities and signals via

uniform representation and reasoning algorithm (Loopy) belief propagation, forward-backward algorithm, Kalman filters, Viterbi algorithm, FFT,

turbo decoding, arc-consistency, production match, …

Can support mixed and hybrid processing Several neural network models map onto them

Graphical Models

w

yx

z

up(u,w,x,y,z) = p(u)p(w)p(x|u,w)p(y|x)p(z|x)

f1

w

f3f2

y

x zu

f(u,w,x,y,z) = f1(u,w,x)f2(x,y,z)f3(z)

p(x|u,w)

w

y

x

z

u p(y|x)

p(z|x)

p(u)

p(w)

12

Factor graphs handle arbitrary multivariate functions– Variables in function map onto variable nodes– Factors in decomposition map onto factor nodes– Bidirectional links connect factors with their variables

Summary product alg. processes messages on links– Messages are distributions over link variables (starting w/ evidence)– At variable nodes messages are combined via pointwise product– At factor nodes do products, and summarize out unneeded variables:

122132 ...

y zx

f1 =0 2 4 6 …1 3 5 7 …2 4 6 8 … …

f2 =0 1 2 …1 2 3 …2 3 4 … …

Factor Graphs and the Summary Product Algorithm

A single settling can efficiently yield: Marginals on all variables (integral/sum) Maximum a posterior – MAP (max)Can mix across segments of graph

234...

678...[0 0 0 1 0 …] [0 0 1 0 0 …]

“3” “2”

Based on Kschischang, Frey & Loeliger, 1998

13

Multidimensional continuous functions– One dimension per variable

Approximated as piecewise linear over arrays of rectilinear (orthotopic) regions

Discretize domain for discrete distributions & symbols [1,2)=.2, [2,3)=.5, [3,4)=.3

Booleanize range (and add symbol table) for symbols[0,1)=1 Color(x, Red)=True, [1,2)=0 Color(x, Green)=False

Series10

0.2

0.4

0.6

Mixed Hybrid Representation for Functions/Messages

.7x+.3y+.1

.6x-.2y

1

0

1

1

x+y

.5x+.2

0

x

y

0 .2 .5 .3

Analogous to implementing digital circuits by restricting an inherently continuous underlying technology

14

Object:

WM

Concept:

Join

Pattern

Function

Constant

Constructing SigmaDefining Long-Term and Working Memories

Walker Table Dog Human

.1 .3 .5 .1

CONDITIONAL Concept-Prior Conditions: Object(s,O1) Condacts: Concept(O1,c)

Predicate-based representation– E.g., Object(s,O1), Concept(O1,c)– Arguments are constants in WM but may be variables in LTM

LTM is composed of conditionals (generalized rules)– A conditional is a set of patterns joined with an optional function– Conditionals compile into graph structures

WM comprises nD continuous functions for predicates– Compile to evidence at peripheral factor nodes

LTM Access: Message Passing until Quiescence and then Modify WM

15

Patterns can be conditions, actions or condacts– Conditions and actions embody normal rule semantics

Conditions: Messages flow from WM Actions: Messages flow towards WM

– Condacts embody (bidirectional) constraint/probability semantics Messages flow in both directions: local match + global influence

Pattern networks connect via join nodes– Product (≈ AND for 0/1) enforces variable binding equality

Functions are defined over pattern variables

Object:

WM

Concept:

Join

Pattern

Function

Constant

Walker Table Dog Human

.1 .3 .5 .1

The Structure of ConditionalsCONDITIONAL Concept-Prior Conditions: Object(s,O1) Condacts: Concept(O1,c)

16

Some More Detail on Predicates and Patterns

May be closed world or open world– Do unspecified WM regions default to unknown (1) or false (0)?

Arguments/variables may be unique or universal– Unique act like random variables: P(a)

Distribution over values: [.1 .5 .4] Basis for rating and choice

– Universal act like rule variables: (a ^next b)(b ^next c)(a ^next c) Any/all elements can be true/1: [1 1 0 0 1] Work with all matching values

Key distinctions between Procedural and Declarative Memories

17

Key Questions to be Answered

To what extent can the full range of mechanisms required for intelligent behavior be implemented in this manner?

Can the requisite range of mechanisms all be sufficiently efficient for real time behavior on the part of the whole system?

What are the functional gains from such a uniform implementation and integration?

To what extent can the human mind and brain be modeled via such an approach?

18

PROGRESS

19

Mental imagery [BICA 11a]*– 2D continuous imagery buffer– Transformations on objects

Perception– Edge detection– Object recognition (CRFs) [BICA 11b]

– Localization (of self) [BICA 11b]

Statistical natural language– Question answering (selection)– Word sense disambiguation

Graph integration [BICA 11b]

– CRF + Localization + POMDP

Progress

Memory [ICCM 10]– Procedural (rule)– Declarative (semantic, episodic)– Constraint

Problem solving– Preference based decisions [AGI 11]

– Impasse-driven reflection– Decision-theoretic (POMDP) [BICA 11b]

– Theory of Mind Learning

– Episodic– Gradient descent– Reinforcement

Some of these are very much just beginnings!

20

CONDITIONAL Transitive Conditions: Next(a,b) Next(b,c) Actions: Next(a,c)

(type ’X :constants ‘(X1 X2 X3))(predicate ‘Next ‘((first X) (second X)) :world ‘closed)

0 0 0

1 0 0

0 1 0

0 0 0

1 0 0

0 1 0

0 0 0

1 0 0

0 1 0

0 0

1 0

Procedural if-then Structures Just conditions and actions

– CW and universal variables

Memory (Rules)

WM

Pattern

Join

X2

second

first

X1

X2 X3X1

X3

WM

Next(X1,X2)Next(X2,X3)

Next(a,b)

Next(b,c)X2c

b

X1

X2 X3X1

X3

X2ba

X1

X2 X3X1

X3

a

b

cX2c

a

X1

X2 X3X1

X3

1

1

21

CONDITIONAL Concept-Prior Conditions: Object(s,O1) Condacts: Concept(O1,c)

Naïve Bayes classifier– Prior on concept + CPs on attributes

Just condacts (in pure form)– OW and unique variables

Memory (Semantic)

CONDITIONAL Concept-Weight Conditions: Object(s,O1) Condacts: Concept(O1,c) Weight(O1,w)

w\c Walker Table …

[1,10> .01w .001w …

[10,20> .2-.01w “ …

[20,50> 0 .025-.00025w …

[50,100> “ “ …

Walker Table Dog Human

.1 .3 .5 .1

Object:

WM

Concept:

Join

Pattern

Function

Constant

Given cues, retrieve (predict) object category and missing attributesE.g., Given Color=Silver, Retrieve Category=Walker, Legs=4, Mobile=T, Alive=F, Weight=10

22

Example Semantic Memory Graph

Concept (S)

Legs (D)Mobile (B)

Weight (C) Color (S)

Alive (B)

Just a subset of factor nodes (and no variable nodes)

B: BooleanS: SymbolicD: DiscreteC: Continuous

FunctionWMJoin

T

4

Dog=.21

F=.01, T=.2

Silv

er=.

01,

Bro

wn=

.14,

Whi

te=.

05

[1,50)=.00006w-.00006,

[50,150)=.004-.00003w

23

Local, Incremental, Gradient Descent Learning(w/ Abram Demski & Teawon Han)

Concept (S)

Legs (D)Mobile (B)

Weight (C) Color (S)

Alive (B)T

4

Based on Russell et al., 1995

Gradient defined by feedback to function node Normalize (and subtract out average) Multiply by learning rateAdd to function, (shift positive,) and normalize

24

Procedural vs. Declarative Memories

Similarities All based on WM and LTM All LTM based on conditionals All conditionals map to graph Processing by summary product

Differences Procedural vs. declarative

– Conditions+actions vs. condacts Directionality of message flow

– Closed vs. open world– Universal vs. unique variables

Constraints are actually hybrid: condacts, OW, universalOther variations also possible

25

Mental Imagery

How is spatial information represented and processed in minds?– Add and delete objects from images– Translate, scale and rotate objects– Extract implied properties for further reasoning

In a symbolic architecture either need to– Represent and reason about images symbolically– Connect to an imagery component (as in Soar 9)

Here goal is to use same mechanisms– Representation: Piecewise continuous functions– Reasoning: Conditionals (FGs + SP)

26

2D Imagery Buffer in the Eight Puzzle

The Eight Puzzle is a classic sliding tile puzzle

Represented symbolically in typical AI systems– LeftOf(cell11, cell21), At(tile1, cell11), etc.

Instead represent as a 3D function– Continuous spatial x & y dimensions

(type 'dimension :min 0 :max 3)– Discrete tile dimension (an xy plane)

(type 'tile :discrete t :min 0 :max 9)– Region of plane with tile has value 1

All other regions have value 0 (predicate 'board ’((x dimension) (y dimension) (tile tile !)))

27

Affine Transformations

Translation: Addition (offset)– Negative (e.g., y + -3.1 or y − 3.1): Shift to the left– Positive (e.g., y + 1.5): Shift to the right

Scaling: Multiplication (coefficient)– <1 (e.g. ¼ × y): Shrink– >1 (e.g. 4.37 × y): Enlarge– -1 (e.g., -1 × y or -y): Reflect– Requires translation as well to scale around object center

Rotation (by multiples of 90°): Swap dimensions– x ⇄ y– In general also requires reflections and translations

28

Offset boundaries of regions along a dimensions

Special purpose optimization of a delta function

CONDITIONAL Move-Right Conditions: (selected state:s operator:o) (operator id:o state:s x:x y:y)

(board state:s x:x y:y tile:t) (board state:s x:x+1 y:y tile:0) Actions: (board state:s x:x+1 y:y tile:t) (board – state:s x:x y:y tile:t) (board state:s x:x y:y tile:0) (board – state:s x:x+1 y:y tile:0)

CR

OPPA

D

Translate a Tile

29

Transform a Z Tetromino

CONDITIONAL Rotate-90-Right Conditions: (tetromino x:x y:y) Actions: (tetromino x:4-y y:x)

CONDITIONAL Reflect-Horizontal Conditions: (tetromino x:x y:y) Actions: (tetromino x:4-x y:y)

CONDITIONAL Scale-Half-Horizontal Conditions: (tetromino x:x y:y) Actions: (tetromino x:x/2+1 y:y)

30

Comments on Affine Transformations

Support feature extraction– Edge detection with no fixed pixel size

Support symbolic reasoning– Working across time slices in episodic memory– Working across levels of reflection– Asserting equality of different variables

Need polytopic regions for any-angle rotation

CONDITIONAL Edge-Detector-Left Conditions: (tetromino x:x y:y) (tetromino – x:x-.00001 y:y) Actions: (edge x:x y:y)

×

http://mathworld.wolfram.com/ConvexPolyhedron.html

31

X1 X2 XT2

A1

U2

A2

XT3 X3

U3U1

X0 XT1

A0

Pr

Problem Solving

1 2 3

4 5

7 8 6

1 2 3

4 5

7 8 6

1 2

4 5 3

7 8 6

1 2 3

4 5 6

7 8

1 2 3

8 4

7 6 5

…

In cognitive architectures, the standard approach is combinatoric search for a goal over sequences of operator applications to symbolic states– Architectures like Soar also add control knowledge for decisions

based on associative (rule-driven) retrieval of preferences E.g., operators that move tiles into position are best

Decision-theoretic approach maximizes utility over sequences of operators with uncertain outcomes– E.g., via a partially observable Markov decision process (POMDP)

This work integrates the latter into the former– While exploring (aspect of) grand unification with perception

32

Standard (Soar-like) Problem Solving Base level: Generate, evaluate, select, apply operators

– Generate (retractable): OW actions – LTM(WM) WM– Evaluate (retractable): OW actions + fns – LTM(WM) LM

Link memory (LM) caches last message in both directions– Subsumes Soar’s alpha, beta and preference memories

– Select: Unique variables – LM(WM) WM– Apply (latched): CW actions – LTM(WM) WM

Meta level: Reflect on impasse (not focus here)

Selection

Application

LTM

WM

Generation

LMEvaluation

––

Join Negate WMChanges

+

Decision subgraph

Choice

33

All knowledge encoded as conditionals

Total of 17 conditionals to solve simple problems– 667 nodes (359 variable, 308 factor) and 732 links– Sample problem takes 5541 messages over 7 decisions

792 messages per graph cycle, and .8 msec per message (on iMac)

CONDITIONAL Move-Left ; Move tile left (and blank right) Conditions: (selected state:s operator:left) (operator id:left state:s x:x y:y) (board state:s x:x y:y tile:t) (board state:s x:x-1 y:y tile:0)Actions: (board state:s x:x y:y tile:0) (board – state:s x:x-1 y:y tile:0) (board state:s x:x-1 y:y tile:t) (board – state:s x:x y:y tile:t)

CONDITIONAL Goal-Best ; Prefer operator that moves a tile into its desired location Conditions: (blank state:s cell:cb) (acceptable state:s operator:ct) (location cell:ct tile:t) (goal cell:cb tile:t) Actions: (selected state:s operator:ct) Function: 1

Eight Puzzle Problem Solving

34

Find way in corridor from to G– Locations are discrete, and a map is provided– Vision is local, and feature based rather than object based

Can detect walls (rectangles) and doors (rectangles + circles, colors) Integrates perception, localization, decisions & action

– Both perception and action introduce uncertainty Yielding distributions over objects, locations and action effects

Decision Theoretic Problem Solving + PerceptionChallenge problem

Door 1 Door 3 Door 2

Wal

l WallIG

35

Integrated Graph for Challenge Problem

O0

X0 XT-1

A-1

O-1

X-1 XT-2

A-2

O-2

X-2 XT-3

A-3

X-3

M0M-1M-2 Pr

O0 O-1 O-2 OT-2 OT-1

P1-2

S1-2

P 2-2

S2-2

P3-2

S3-2

P1-1

S1-1

P 2-1

S2-1

P3-1

S3-1

P10

S10

P 20

S20

P30

S30

X1 X2 XT2

A1

U2

A2

XT3 X3

U3U1

XT1

A0

CRF

POMDP

SLAM

Yields distribution over A0 from which best action can be selected

Teawon Han (USC)

Junda Chen (USC)Louis-Philippe Morency (USC/ICT)

Nicole Rafidi (Princeton)David Pynadath (USC/ICT)

Abram Demski (USC/ICT)

36

Comments on Problem Solving & Integrated Graph

Shows decision-theoretic problem solving within same architecture as symbolic problem solving– Ultimately using same preference-based choice mechanism– Capable of reflecting on impasses in decision making

Implemented within graphical architecture without adding CRF, localization and POMDP modules to it– Instead, knowledge is added to LTM and evidence to WM

Distribution on A0 defines operator selection preferences– Just as when solve the Eight Puzzle in standard manner

Total of 25 conditionals– 293 nodes (132 variable, 161 factor) and 289 links– Sample problem takes 7837 messages over 20 decisions

392 messages per graph cycle, and .5 msec per message (on iMac)

37

Reinforcement Learning

Learn values of actions for states from rewards– SARSA: Q(st, at) ← Q(st, at) + α[rt + γQ(st+1, at+1) - Q(st, at)]

Deconstruct in terms of:– Gradient-descent learning– Schematic knowledge for prediction

Synchronic learning/prediction of:– Current reward (R)– Discounted future reward (P)– Q values (Q)– Learn given an action model

Diachronic learning/prediction of:– Action model (transition function) (SN)– Requires addition of intervening decision cycle

At

Pt+1

St+1

Rt+1Q(A)tPtRt

St St+1

R

At

Pt+1

St+1

Rt+1Q(A)tPtRt

St SNt

R

St+1

38

RL in 1D Grid

CONDITIONAL Reward Condacts: (Reward x:x value:r) Function<x,r>: .1:<[1,6)>,*> …

CONDITIONAL Backup Conditions: (Location state:s x:x) (Selected state:s operator:o) (Location*Next state:s x:nx) (Reward x:nx value:r) (Projected x:nx value:p) Actions: (Q x:x operator:o value:.95*(p+r)) (Projected x:x value:.95*(p+r))

CONDITIONAL Transition Conditions: (Location state:s x:x) (Selected state:s operator:o) Condacts: (Location*Next state:s x:nx) Function<x,o,nx>: (.125 * * *)

0 1 2 3 4 5 6 7G

0 1 2 3 4 5 6 702468

LeftRight

0 1 2 3 4 5 6 70

5

10

0 1 2 3 4 5 6 702468

0 1 2 3 4 5 6 7

Reward

Projected

Q

Graphs are of expected values, but learning is of full distributions

Sampling of conditionals

39

Theory of Mind (ToM)(w/ David Pynadath & Stacy Marsella)

Modeling the minds of others– Assessing and predicting complex multiparty situations

My model of her model of …– Building social agents and virtual humans

Can Sigma (elegantly) extend to ToM?– Based on PscyhSim (Pynadath & Marsella)

Decision theoretic problem solving based on POMDPs Recursive agent modeling

– Preliminary work in Sigma on intertwined POMDPs (w/ Nicole Rafidi) Belief revision based on explaining past history

Can cost and quality of ToM be improved? Initial experiments with one-shot, two-person games

– Cooperate vs. defect

40

One-Shot, Two-Person Games

Two players Played only once (not repeated)

– So do not need to look beyond current decision

Symmetric: Players have same payoff matrix Asymmetric: Players have distinct payoff matrices Socially preferred outcome: optimum in some sense Nash equilibrium: No player can increase their

payoff by changing their choice if others stay fixed– Sigma is finding the best Nash equilibrium

Prisoner’s Dilemma

Cooperate

Defect

Cooperate .3 .1(,.4)

Defect .4(,.1) .2A

B

A Cooperate

Defect

Cooperate .1 .2

Defect .3 .1

B Cooperate

Defect

Cooperate .1 .1

Defect .4 .4

41

Symmetric, One-Shot, Two-Person Games

CONDITIONAL Payoff-A-A CONDITIONAL Payoff-B-B Conditions: Choice(A,B,op-b) Conditions: Choice(B,A,op-a) [B’s model of A] Actions: Choice(A,A,op-a) Actions: Choice(B,B,op-b) [B’s model of B] Function: payoff(op-a,op-b) Function: payoff(op-b,op-a)

CONDITIONAL Payoff-A-B CONDITIONAL Payoff-B-A Conditions: Choice(A,A,op-a) Conditions: Choice(B,B,op-b) Actions: Choice(A,B,op-b) Actions: Choice(B,A,op-a) Function: payoff(op-b,op-a) Function: payoff(op-a,op-b)

CONDITIONAL Select-Own-Op Conditions: Choice(ag,ag,op) Actions: Selected(ag,op)

Prisoner’s Dilemma

Cooperate

Defect AResult

BResult

Cooperate .3 .1 .43 .43

Defect .4 .2 .57 .57

StagHunt

Cooperate

Defect AResult

BResult

Cooperate .25 0 .54 .54

Defect .1 .1 .46 .46

602 Messages 962 Messages

Agent A Agent B

42

Graph Structure

Select **

PBA

PAB

PAB

PBA

POR

Actual (Abstracted)

All one predicate

Select BB BA

PAB

PBA

AA AB

PBA

PAB

Select

Nominal

Agent A

Agent B

43

Asymmetric, One-Shot, Two-Person Games

CONDITIONAL Payoff-A-A CONDITIONAL Payoff-B-B Conditions: Choice(A,B,op-b) Conditions: Choice(B,A,op-a) Actions: Choice(A,A,op-a) Actions: Choice(B,B,op-b) Function: payoff(A,op-a,op-b) Function: payoff(B,op-b,op-a)

CONDITIONAL Payoff-A-B CONDITIONAL Payoff-B-A Conditions: Choice(A,A,op-a) Conditions: Choice(B,B,op-b) Model(m) Model(m) Actions: Choice(A,B,op-b) Actions: Choice(B,A,op-a) Function: payoff(m,op-b,op-a) Function: payoff(m,op-a,op-b)

CONDITIONAL Select-Own-Op Conditions: Choice(ag,ag,op) Actions: Selected(ag,op)

A Cooperate

Defect

Cooperate .1 .2

Defect .3 .1

B Cooperate

Defect

Cooperate .1 .1

Defect .4 .4374 Messages 636 Messages

CorrectOther

AResult

BResult

Cooperate .51 .29

Defect .49 .71

Other asSelf

AResult

BResult

Cooperate .47 .29

Defect .53 .71

44

WRAP UP

45

Closed vs. open world functionsUniversal vs. unique variablesDiscrete vs. continuous variablesBoolean vs. numeric function values

Uni- vs. bi-directional linksMax vs. sum summarizationLong- vs. short-term memoryProduct vs. affine factors

f1

w

f3f2

y

x zu

f(u,w,x,y,z) = f1(u,w,x)f2(x,y,z)f3(z)

Factor graphs w/ Summary Product

0

x+.3y

0

1

.5y

6x

x-y

1

Piecewise Continuous Functions

Rule memory Preference-based decisionsEpisodic memory POMDP-based decisionsSemantic memory LocalizationMental imagery …Edge detectors

➤➤➤➤➤

➤➤➤

Broad Set of Capabilities from Space of VariationsHighlighting Functional Elegance and Grand Unification

Knowledge above architecture also involved– Conditionals that are compiled into subgraphs

46

Conclusion Sigma is a novel graphical architecture

– With potential to support integrated cognition and the development of virtual humans (and intelligent agents/robots)

– Focus so far is not on a unified theory of human cognition However, makes interesting points of contact with existing theories

Grand unification– Demonstrated mixed processing

Both general symbolic problem solving and probabilistic reasoning– Demonstrated hybrid processing

Including forms of perception integrated directly with cognition– Need much more on perception, plus action, emotion, …

Functional elegance– Demonstrated aspects of memory, learning, problem solving,

perception, imagery, Theory of Mind [and natural language]– Based on factor graphs and piecewise continuous functions

47

PublicationsRosenbloom, P. S. (2009). Towards a new cognitive hourglass: Uniform implementation of cognitive architecture via factor graphs.  Proceedings of the

9th International Conference on Cognitive Modeling.Rosenbloom, P. S. (2009).  A graphical rethinking of the cognitive inner loop.  Proceedings of the IJCAI International Workshop on Graphical Structures

for Knowledge Representation and Reasoning.Rosenbloom, P. S. (2009).  Towards uniform implementation of architectural diversity.  Proceedings of the AAAI Fall Symposium on Multi-

Representational Architectures for Human-Level Intelligence.Rosenbloom, P. S. (2010). An architectural approach to statistical relational AI.  Proceedings of the AAAI Workshop on Statistical Relational AI.Rosenbloom, P. S. (2010). Speculations on leveraging graphical models for architectural integration of visual representation and reasoning. 

Proceedings of the AAAI-10 Workshop on Visual Representations and Reasoning.Rosenbloom, P. S. (2010). Combining procedural and declarative knowledge in a graphical architecture.  Proceedings of the 10th International

Conference on Cognitive Modeling.Rosenbloom, P. S. (2010). Implementing first-order variables in a graphical cognitive architecture.  Proceedings of the First International Conference on

Biologically Inspired Cognitive Architectures.Rosenbloom, P. S. (2011). Rethinking cognitive architecture via graphical models.  Cognitive Systems Research, 12, 198-209.Rosenbloom, P. S. (2011). From memory to problem solving: Mechanism reuse in a graphical cognitive architecture.  Proceedings of the Fourth

Conference on Artificial General Intelligence. Winner of the 2011 Kurzweil Award for Best AGI Idea.Rosenbloom, P. S. (2011). Mental imagery in a graphical cognitive architecture.  Proceedings of the Second International Conference on Biologically

Inspired Cognitive Architectures.Chen, J., Demski, A., Han, T., Morency, L-P., Pynadath, P., Rafidi, N. & Rosenbloom, P. S. (2011). Fusing symbolic and decision-theoretic problem

solving + perception in a graphical cognitive architecture.  Proceedings of the Second International Conference on Biologically Inspired Cognitive Architectures.

Rosenbloom, P. S. (2011). Bridging dichotomies in cognitive architectures for virtual humans.  Proceedings of the AAAI Fall Symposium on Advances in Cognitive Systems.

Rosenbloom, P. S. (2012). Graphical models for integrated intelligent robot architectures. Proceedings of the AAAI Spring Symposium on Designing Intelligent Robots: Reintegrating AI.

Rosenbloom, P. S. (2012). Towards a 50 msec cognitive cycle in a graphical architecture. Proceedings of the 11th International Conference on Cognitive Modeling.

Rosenbloom, P. S. (2012). Towards functionally elegant, grand unified architectures. Proceedings of the 21st Behavior Representation in Modeling & Simulation (BRIMS) Conference. Abstract for panel on “Accelerating the Evolution of Cognitive Architectures,” K. A. Gluck (organizer).