Goal�ComputationallyE�cientKnowledge

RepresentationSystems

Knowledgerepresentationsystem�

Knowledgebasecontainsfactsabouttheworld�

�Commonsenseknowledge�

Inferencemechanism

infersnew

factsfrom

storedones�

Makeimplicitknowledgeexplicit�

Modulardesignforintelligentsystems�

Modulesforperception�action�etc�query

querytheknowledgerepresentationsystem

asneeded�

�

RepresentingKnowledge

Logic�propositional��rst�order�terminological����

Declarative�

Wellunderstoodsemantics�

Usedforcommonsensetheories�

Ontologyofliquids�Hayes�����

Qualitativeprocesstheory�Forbus�����

Centralproblem�

Inferenceisintractable�

�

ExpressivenessversusComplexity

Directtradeo�betweenexpressivenessand

tractabilityofarepresentationlanguage�

�LevesqueandBrachman�����

CompareF

irst�orderlogic�expressivebutintractable�

Relationaldatabases�restrictedbute�cient�

Problem�

Standarddatabasesexpressivelyinadequatefor

disjunctiveandorincompleteinformation�

W

etnessh���Episodeh�h���

M

ereweth��Dryingh��Spreadingh�

�Hayes�����

�

DealingwithComplexity

��Restrictingthelanguage

�LevesqueandBrachman�����Nebeletal�

����

Example�restrictedterminologicallogics�

Disadvantage�sublanguageoftennotsu�ciently

expressive�

��Incompletereasoning�non�standardsemantics

�Levesque�����Frisch����

Example�four�valuedsemanticsnomodusponens��

Disadvantage�inferencemechanism

oftentooweak�

�

��Incompletereasoning�resourcebounded

�DoyleandPatil�����

Example�runtheorem

proverforlimitedamount

oftime�

Disadvantage�unclearwhatcancannotbeinferred�

��Vividreasoning

�Levesque����

Example�usedefaultstoremovedisjunctive

information��Etheringtonet

al�����Selman����

Disadvantage�unsound�

Alternativeapproach�

�KnowledgeCompilation

�

KnowledgeCompilation

Translateknowledgegiveninsomegeneralrepresentation

languageintoatractable�restrictedlanguage�

sourcelanguage��

targetlanguage

Exacttranslationoftennotpossible�

Canapproximateoriginaltheory

yetretainsoundness�

completeness

inansweringqueries�

�

Outline

Propositionalcase

De�ningHornapproximations

Properties

Algorithmsandcomplexity

Extensions

Othertractabletargetlanguages

Terminologicallogics

�

PropositionalTheories

Source�clausalpropositionaltheories�

Inference�NP�Complete�

Target�Horntheories�

Inference�lineartime�

Notation

Clause�disjunctionofliterals�

Clausaltheory�conjunctionofclauses�CNF��

Hornclause�atmostonepositiveliteral�

Example���a�

�b�

c�

Equivalently��a�

b��

c

Negativeclause���a�

�b�

Model�ofatheory��atruthassignment�underwhich

thetheoryevaluatesto true���

�

HornApproximations�ModelTheory

modelsof�lb

modelsof�

modelsof�ub

�

originalCNFtheory�

lb

�

Lower�boundHornapproximation�

ub

�

Upper�boundHornapproximation�

De�nition�HornBounds

Where

isasetofclauses

andM

�isthesetofmodelsof

satisfyingtruthassignments�

Dene

lbisaHornLower�boundof

ubisaHornUpper�boundof

i� lband ubaresetsofHornclausesand

M

lb��M

��M

ub�

equivalently

lbj�

j�

ub

Lowerbound�

fewermodels�

logicallystronger

Upperbound�

moremodels�

logicallyweaker

glbisaGreatestHornlower�bound�GLB�

glbisaHornlower�bound�and

Noset �ofHornclausessuchthat

M

glb��M

���M

��

Equivalently�aweakestHorntheorythatimplies �

Notuniquefor �

lubisaLeastHornupper�bound�LUB�

lubisaHornupper�bound�and

Noset �

ofHornclausessuchthat

M

��M

���M

lub��

Equivalently�strongestHorntheoryimpliedby �

Isuniquefor �

glband lubareHornapproximationsof �

�

Example

�

�a�c���b�c��a�b�

Hornlower�bound�a�b�c

GLBs�

a�c

and

b�c

Hornupper�bound��a�c���b�c�

LUB�

c

a�b�c

j�

a�c

j�

j�

c

j�

�a�c���b�c�

GLB

LUB

�

UsingApproximationsforQueryAnswering

j�

��

If lubj�

�then j�

��

Lineartime��

If glbj�

�then j�

��

Lineartime��

Otherwise�use

directly�

Orreturn�don�tknow���

Queriesansweredinlineartimeleadto

improvementinoverallresponsetime

toaseriesofqueries�

�

QueryLanguages

Querylanguagecanbemoreexpressivethan

targetlanguage�

HornApproximations�

QuerycanbearbitraryCNFformula�

Answerinlineartime�

�

PropertiesofHornApproximations

LUB

canbeviewedasanabstraction�

Considerbackgroundknowledge�

doctorX��professionalX�

lawyerX��professionalX�

Fact�

doctorJill��laywerJill�

LUB

isprofessionalJill�

doctorX��professionalX�

lawyerX��professionalX�

�

abstractionoffacts�

originalbackgroundknowledge�

Generalizes�BorgidaandEtherington�����

�

GLB

canbeviewedasaspecialization�

doctorJill��laywerJill�becomesdoctorJill�

inoneoftheGLBs��

Asacounterexample�

glbj�

lawyerJill�implies

j�

lawyerJill�

glbj�

doctorJill�providesnoinformation�

Comparegeometrytheorem

proving�Gelernter����

Asapositiveexample�

JumptoconclusionthatDoctorJill��

Notsoundwhenusedinthisway�

GLB

isasetofmodels�sogeneralizesvividreasoning

�Levesque�����mentalmodels�Johnson�Laird�����

�

ComputingHornApproximations

Theorem�Let

beasetofclauses�TheGLB

of

is

consistenti�

isconsistent�SimilarlyforLUB��

ComputingGLB

orLUB

isintractable�

ProvidedP�

NP�

Approximationscanbeusedtocheckconsistency�

View

ascompilationprocess�

Costamortizedovertotalsetofqueries�

�

ComputingtheGLB

Horn�strengthening�

p�q��rhasHorn�strengthenings

p��rand

q��r

Horn�strengtheningofatheory�

p�q��r��s�t�hasstrengthening

p��r��s

amongothers��

Theorem�EachGLB

of

isequivalenttosome

Horn�strengtheningof �

Algorithm�searchspaceofHorn�strengthenings�

�

Lemma

Where H

isaHorntheory�andC

isanyclause�

If H

entailsC�

then H

entailssomeHorn�strengtheningofC�

Proof

Bycompletenessofresolution�thereissomeclauseC�that

followsfrom

H

byresolution�suchthatC��C�

BecausetheresolventofHornclausesisHorn�C�

isHorn�

ThereforethereissomeCH

thatisaHorn�strengtheningof

C

suchthatC�

�CH�C�andso Hj�

CH�

�

GLB

�

W

eakestHorn�strengthening

glbisHorn�sobylemma

glbentailssomeHorn�strengthening �of �

glbj�

�j�

But glbisagreatestweakest�Hornlower�bound�

Therefore� glb� ��

�

procedureGLB �

�ComputessomeHorngreatestlower�boundof

�

begin

L��

thelexicographically�rst

Horn�strengtheningof

beginloop

L�

��

lexicographicallynext

Horn�strengtheningof

ifnoneexiststhenexit�

ifLj�

L�thenL��

L�

endloop

removesubsumedclausesfrom

L

returnL

end

��

ExampleofGLB

Algorithm

�

�a�c���a�b�c�d�

Horn�strengthenings�

L��

�a�c���a�b�

L��

�a�c���a�c�

L��

�a�c���a�d�

BecauseL

�j�

L�

L�j�

L�

algorithm

returnsL�

��

PropertiesoftheGLB

algorithm

Anytime�

Algorithm

maybestoppedatanytimeto

�ndalower�boundnotnecessaryaGLB��

Lower�boundimprovesovertime�

LengthofGLB

�lengthoforiginaltheory�

��

ComputingtheLUB

BasicStrategy�

Computeallresolventsoforiginaltheory�

andcollectallHornresolvents�

Problem�E

venaHorntheorycanhaveexponentially

manyHornresolvents�

Solution�R

esolveonlypairsofclausescontaining

atleastonenon�Hornclause�

Methodiscomplete��SelmanandKautz�����

��

ExampleofComputingLUB

�

�a�b���b�c��a�b�

Resolvents�

������

b

������

a�c

Answeris�

a�b���b�c��b

�b�c

Algorithm

doesnotresolve�����

��

PropertiesoftheLUB

algorithm

Anytime�

Nospaceblow�upforHorn�

Canconstructnon�Horntheorieswith

exponentiallylargerLUB�

New

letterscansometimesreducesize�

��

Explosion�anexample

is

CompSci�Phil�Psych��CogSci

ReadsMcCarthy�CompSci�CogSci�

ReadsDennett�Phil�CogSci�

ReadsKosslyn�Psych�CogSci�

LUB

isCompSci�Phil�Psych��CogSci

CompSci�Phil�ReadsKosslyn��CogSci

CompSci�ReadsDennett�Psych��CogSci

���

ReadsMcCarthy�ReadsDennett�ReadsKosslyn��CogSci

SizeLUB

O�n�

Nosmallerequivalentsetofclausesexists�

��

ShrinkingLUB�introducenew

concepts

�

CompSciBuff�CompSci�ReadsMcCarthy�

PhilBuff�Phil�ReadsDennett�

PsychBuff�Psych�ReadsKosslyn�

LUB

becomes

CompSciBuff�PhilBuff�PsychBuff��CogSci

CompSci�CompSciBuff

ReadsMcCarthy�CompSciBuff

Phil�PhilBuff

ReadsDennett�PhilBuff

Psych�PsychBuff

ReadsKosslyn�PsychBuff

SizeLUB

On��

��

New

LUB

andoriginalLUB

areequivalentonqueries

inLang ��

New

conceptscapturewhatcertainpairsofpropositions

haveincommon�

E�g��CompSciBuff�CompSci�ReadsMcCarthy�

So�new

conceptsareusefulgeneralizationsforobtaining

atractableapproximationoftheoriginaltheory�

Formingconceptsforfastinference�

QUESTION�doesasmallperhapsnon�obvious��representation

oftheLUB

alwaysexist�

�

W

hatDoW

eMeanby�Size��

Therearemanyequivalentclausaltheories

� ��yet �exponentiallylarger

Perhaps��size�willmeansizeof

smallestequivalentsetofclauses

Notsu�cientforprovingthatsomething

isinherentlylarge�theremaybe

cleverwaystoencodealargesetofclauses

Structuresharing�

Schemas�etc�

ConsiderthenanyrepresentationoftheHornLUB

thatenablespolytimeinference

�

TheAnswer�

TheredoexisttheorieswhoseHornLUB

isinherentlylarge�

AnyrepresentationoftheLUB

thatenables

polynomialtimeinferenceis

exponentiallylargerthanthetheory�

��

Proof�CircuitComplexity

Non�uniform

P�foreveryn�thereissomepolysize

circuitforinputsoflengthn

equiv�somepolytimealgorithm�

�

samepolynomialforalln�

ProofofinherentlyintractableLUB�s�

Canconstructasingleparticulartheory

whoseLUB

canbeusedtosolve��SAT

foranyformulaoverm

variables

foreachm��

��

�m��inputs�specifyany

LUB �

��SAT

overm

variables

�pm����pm����pm

�p��p��p�

p��p��p�

��

unsatis�able

Circuitcomputes���i�LUB �entails

thatnotalltheinputssetto���canhold

simultaneously�

��

Universal�SAT

Theory

For��SAT

formulasoveragivensetofm

variables�

de�ne�

�

Vfx�y�z��ixyz

jx�y�xareliteralsg

wherethe�i�variablesarenew

Any��CNFformulaoverm

variablescanbe

speci�edasasingleclausemadeup

oftheiinput�variables

��

Example

p���p��p����p��p���p��isunsatis�able

i�

p���p��p���ipp�p� ���p��p���p���ip�p�p� �j�

�ipp�p�

��ip�p�p�

i�

LUB �j�

�ipp�p�

��ip�p�p�

Note�queryisHorn�soLUB

iscomplete�

��

RelationtothePolynomialHierarchy

ExistenceofsmallLUB�sfortheseuniversaltheories

wouldimplyNP�non�uniform

P

WeakerconditionthanP�NP�

Polynomialhierarchycollapsesto �

Stillconsideredtobeprettyunlikely

��

Implications

MaychooseLUB

from

adi�erenttractable

class�thatdoesguaranteeitissmall

k�Horn�boundlengthofHornclauses

Onk

�maxsize

��SAT

�conjunction�literalclauses

NotHorn�p�q�okay

On�

�maxsize

Couldtrytocompiletoseveraldi�erenttractable

classes�pickmostpowerfulclassthatissmall

fortheparticularinputtheory

GLB

andLUB

maybedi�erentkindsoftheories

��

Extensions

Othertargetlanguages�

��SAT

On��maxsize��

k�HornOnk�maxsize��

Clausesnotcontaininggivensetof

�irrelevant�propositions�

�Compilingaway�partsoforiginaltheory�

Similarto�SubramanianandGenesereth������

First�ordersourceandtargetlanguages�

Algorithmsmaynotterminate�

GLB

algorithm�interleavecomparisonofHorn�

strengtheningsandsearch�

��

OtherFormalisms�TerminologicalLogics

TerminologicallogicsClassic�Brachmanetal��

FL�intractable�

FL�

�tractablenorolerestrictions��

CompileFLconceptstoFL�

concepts�

person

�ANDperson�ALL�RESTRfriendmale�

�ANDdoctor�SOMEspecialty���

personwhoseeverymalefriendisadoctorwithaspecialty�

�ANDperson�ALLfriend�ANDdoctor�SOMEspecialty�����

QueryLanguage�TerminologicalLogics

RecentworkbyLenzerini�etal������showsthat

Candeterminesubsumptionbetween

FLconceptand

FL�

concept

inpolynomialtime�

Again�querylanguagecanbemoreexpressive

thantargetlanguage�

�

DoesKnowledgeCompilationReallyW

ork�

Cantheboundsanswer�hard�queries�

oraresuchquerieseasyfororiginaltheory�

Isthereempiricalevidenceofsavings�

Classesconsidered�

Hard�randomlygeneratedtheories

�relativelyunstructureddata�

Planningproblems

�highlystructureddata�

Arecostsalwaysshiftedtocompilationtime�

orcanthecompilationprocessitselfbe�paido����

�

FormalArgumentforSavings

Trivialcase�ifKB

isinconsistent�compilation

detectsthis�allqueriesthenare�free��

A

moreinterestingcase�SupposeKB

isconsistent�

logicallyequivalenttoaHorntheory�but

isnotinHornform�

Therecannotexistatheorem

proverthat

e�cientlyhandlesthisspecialcase

�ValiantandVazirani�����

However�aftercompilationallqueries

canbeansweredinlineartime�

Therefore�KC

doesnotjust�skim�easyqueries�

��

HardRandom

Theories

Testset���random

��CNFtheories�rangingin

sizefrom

��to���variables�

Ratioof���clausespervariableyields

computationallyhardformulas�

�Mitchell�Selman�andLevesque�����

WecomputedtheunitclauseLUB

andGLB

WeakerthantheHornLUB

andGLB�buteasier

tocomputeandanalyze�

Note�unitclauseboundsarealsoHornbounds�

butnotthebestHornbounds�

��

PercentofQueriesAnswered

Basedonthesizeoftheboundsobtained�wecan

exactlycomputethepercentageofall

randomlygeneratedqueriesthatareanswered

bytheboundsalone�

vars

clauses

sizeunit

sizeunit

percentqueriesanswered

LUB

GLB

unit

binary

ternary

��

���

��

��

��

���

���

�

��

��

��

��

��

���

��

��

�

���

��

�

�

�

�

���

���

��

���

���

��

ExecutionTime

Implementedqueryalgorithm�usingtheprogram

Tableau�CrawfordandAuton����

aversionoftheDavis�Putnam

procedure�

tohandlequeriesonwhichboundsfail�

Timeinsecondstoanswer����random

queries

SGIChallenge��

vars

clauses

boundsandtableau

tableauonly

binary

ternary

binary

ternary

��

���

��

��

���

���

�

��

��

��

��

���

��

��

�

��

���

���

�

�

��

��

����

���

��

Random

Formulas�Summary

Knowledgecompilationmightnotbeexpected

toworkonunstructured�random

formulas�

However�evenunitclauseboundsgavegreat

computationalsavings�onaverage�

over���Xfasteron��literalqueries�

On���variabletheories�compilationtime

approx��hour�completelypaidfor

after�����literalqueries�

outperformedoriginalgoalofsimply

shiftingexecutiontimeo��line�

��

PlanningProblems

Domain�Robotmovinginagraph�likeenvironment�

�Pollack�����Hendler����

Movingtocertainnodesconsumesresources�

makingothernodesinaccessible�forbiddenpairs���

Encodedintheplanningassatis�abilityframework

�KautzandSelman�������

Planscorrespondtomodelsofthetheory�

PlanningisNP�completenotjustshortest�path��

��

Example� In going from a to g in at most � steps�

can the robot visit j�

Answer� No by length of shortest path��

MAZE

a c

d

e

h

f

j

g

ki

l

b

��

Example� In going from a to g in at most � steps�

can the robot visit b�

Answer� Yes a model contains path a�b�d�e�f�g��

MAZE

a c

d

e

h

f

j

g

ki

l

b

�

Example� In going from a to g in at most �� steps�

must the robot visit e�

Answer� Yes forbidden pairs eventually block all routes

through area labeled �MAZE���

MAZE

a c

d

e

h

f

j

g

ki

l

b

�

CompilingSAT

EncodingofPlanningDomain

Axiomsformapworldshowninprevious�guresuse

���variables�

������clauses�

Compilingunitboundstakes���hours�

Querytestsets�

RandBin����random

binaryqueries�

RandEver����random

binaryqueries�restricted

topredicatesoftheform

�istheroboteverataspeci�ed�point��

Hand��hand�constructed�non�obvious�queries��

�

PlanningResults

RandBin

RandEver

Hand

numberofqueries

�

�

�

theory

�

onlytime

����

����

����

KC

Querytime

���

��

��LUB

time

��

��

��

KCusing��LUB

time

���

��

��

boundsonlytime

�

�

numberansweredbybounds

��

���

�

Timesinseconds�onanSGIChallenge�

Theory�

has��variables������clauses�

��LUB

time��usingTableauonconjunction

oforiginaltheoryanditsLUB�

��

Observations

BasicKC

Queryalgorithm

increasedspeedby�X

to�X�

Similarbene�tgainedbysimplyconjoiningtheory

anditsLUB�andusingacompletetheorem

prover�

Bestperformance��rsttestagainstbounds if

boundsfail�testagainsttheory�LUB

�

��Xspeedup�

Ifwillingtoignorequeriesnotansweredbythe

bounds�����Xspeedup�handles��!

���!

of

thequeries�

Substitutesensingfortheorem

proving�

��

Conclusions

Wehavebeguntoevaluatecomputationalsavings

possiblewithknowledgecompilationbytheory

approximation�

Boundsanswermanyqueriesthatwouldbehard

toanswerwithanycompletetheorem

prover�

successinshiftingcomputationalcosto��line�

Signi�cantspeed�upoccursonbothunstructured

random�andstructuredplanning�problems�

Goodperformanceobtainedwithunitclause

approximations�

Openquestion�isadditionalcostofcomputing

strongerHornboundsworthwhile�

��

SummaryandConclusions

Introducedknowledgecompilation�

A

proposaltowardsobtaininge�cient

knowledgerepresentationsystems�

Features�

Norestrictionsonexpressivenessofsourcelanguage�

Approximationsbasedontwodelimitingbounds�

Generalizesotherworkonabstractionand

�model�based�reasoning�

Soundandcompleteinference�

E�ciencyimprovesovertime�

Generalityanyextensionalsemantics��

��