Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
Goal�ComputationallyE�cientKnowledge
RepresentationSystems
Knowledgerepresentationsystem�
Knowledgebasecontainsfactsabouttheworld�
�Commonsenseknowledge�
Inferencemechanism
infersnew
factsfrom
storedones�
Makeimplicitknowledgeexplicit�
Modulardesignforintelligentsystems�
Modulesforperception�action�etc�query
querytheknowledgerepresentationsystem
asneeded�
�
RepresentingKnowledge
Logic�propositional��rst�order�terminological����
Declarative�
Wellunderstoodsemantics�
Usedforcommonsensetheories�
Ontologyofliquids�Hayes�����
Qualitativeprocesstheory�Forbus�����
Centralproblem�
Inferenceisintractable�
�
ExpressivenessversusComplexity
Directtradeo�betweenexpressivenessand
tractabilityofarepresentationlanguage�
�LevesqueandBrachman�����
CompareF
irst�orderlogic�expressivebutintractable�
Relationaldatabases�restrictedbute�cient�
Problem�
Standarddatabasesexpressivelyinadequatefor
disjunctiveandorincompleteinformation�
W
etnessh���Episodeh�h���
M
ereweth��Dryingh��Spreadingh�
�Hayes�����
�
DealingwithComplexity
��Restrictingthelanguage
�LevesqueandBrachman�����Nebeletal�
����
Example�restrictedterminologicallogics�
Disadvantage�sublanguageoftennotsu�ciently
expressive�
��Incompletereasoning�non�standardsemantics
�Levesque�����Frisch����
Example�four�valuedsemanticsnomodusponens��
Disadvantage�inferencemechanism
oftentooweak�
�
��Incompletereasoning�resourcebounded
�DoyleandPatil�����
Example�runtheorem
proverforlimitedamount
oftime�
Disadvantage�unclearwhatcancannotbeinferred�
��Vividreasoning
�Levesque����
Example�usedefaultstoremovedisjunctive
information��Etheringtonet
al�����Selman����
Disadvantage�unsound�
Alternativeapproach�
�KnowledgeCompilation
�
KnowledgeCompilation
Translateknowledgegiveninsomegeneralrepresentation
languageintoatractable�restrictedlanguage�
sourcelanguage��
targetlanguage
Exacttranslationoftennotpossible�
Canapproximateoriginaltheory
yetretainsoundness�
completeness
inansweringqueries�
�
Outline
Propositionalcase
De�ningHornapproximations
Properties
Algorithmsandcomplexity
Extensions
Othertractabletargetlanguages
Terminologicallogics
�
PropositionalTheories
Source�clausalpropositionaltheories�
Inference�NP�Complete�
Target�Horntheories�
Inference�lineartime�
Notation
Clause�disjunctionofliterals�
Clausaltheory�conjunctionofclauses�CNF��
Hornclause�atmostonepositiveliteral�
Example���a�
�b�
c�
Equivalently��a�
b��
c
Negativeclause���a�
�b�
Model�ofatheory��atruthassignment�underwhich
thetheoryevaluatesto true���
�
HornApproximations�ModelTheory
modelsof�lb
modelsof�
modelsof�ub
�
originalCNFtheory�
lb
�
Lower�boundHornapproximation�
ub
�
Upper�boundHornapproximation�
De�nition�HornBounds
Where
isasetofclauses
andM
�isthesetofmodelsof
satisfyingtruthassignments�
Dene
lbisaHornLower�boundof
ubisaHornUpper�boundof
i� lband ubaresetsofHornclausesand
M
lb��M
��M
ub�
equivalently
lbj�
j�
ub
Lowerbound�
fewermodels�
logicallystronger
Upperbound�
moremodels�
logicallyweaker
glbisaGreatestHornlower�bound�GLB�
glbisaHornlower�bound�and
Noset �ofHornclausessuchthat
M
glb��M
���M
��
Equivalently�aweakestHorntheorythatimplies �
Notuniquefor �
lubisaLeastHornupper�bound�LUB�
lubisaHornupper�bound�and
Noset �
ofHornclausessuchthat
M
��M
���M
lub��
Equivalently�strongestHorntheoryimpliedby �
Isuniquefor �
glband lubareHornapproximationsof �
�
Example
�
�a�c���b�c��a�b�
Hornlower�bound�a�b�c
GLBs�
a�c
and
b�c
Hornupper�bound��a�c���b�c�
LUB�
c
a�b�c
j�
a�c
j�
j�
c
j�
�a�c���b�c�
GLB
LUB
�
UsingApproximationsforQueryAnswering
j�
��
If lubj�
�then j�
��
Lineartime��
If glbj�
�then j�
��
Lineartime��
Otherwise�use
directly�
Orreturn�don�tknow���
Queriesansweredinlineartimeleadto
improvementinoverallresponsetime
toaseriesofqueries�
�
QueryLanguages
Querylanguagecanbemoreexpressivethan
targetlanguage�
HornApproximations�
QuerycanbearbitraryCNFformula�
Answerinlineartime�
�
PropertiesofHornApproximations
LUB
canbeviewedasanabstraction�
Considerbackgroundknowledge�
doctorX��professionalX�
lawyerX��professionalX�
Fact�
doctorJill��laywerJill�
LUB
isprofessionalJill�
doctorX��professionalX�
lawyerX��professionalX�
�
abstractionoffacts�
originalbackgroundknowledge�
Generalizes�BorgidaandEtherington�����
�
GLB
canbeviewedasaspecialization�
doctorJill��laywerJill�becomesdoctorJill�
inoneoftheGLBs��
Asacounterexample�
glbj�
lawyerJill�implies
j�
lawyerJill�
glbj�
doctorJill�providesnoinformation�
Comparegeometrytheorem
proving�Gelernter����
Asapositiveexample�
JumptoconclusionthatDoctorJill��
Notsoundwhenusedinthisway�
GLB
isasetofmodels�sogeneralizesvividreasoning
�Levesque�����mentalmodels�Johnson�Laird�����
�
ComputingHornApproximations
Theorem�Let
beasetofclauses�TheGLB
of
is
consistenti�
isconsistent�SimilarlyforLUB��
ComputingGLB
orLUB
isintractable�
ProvidedP�
NP�
Approximationscanbeusedtocheckconsistency�
View
ascompilationprocess�
Costamortizedovertotalsetofqueries�
�
ComputingtheGLB
Horn�strengthening�
p�q��rhasHorn�strengthenings
p��rand
q��r
Horn�strengtheningofatheory�
p�q��r��s�t�hasstrengthening
p��r��s
amongothers��
Theorem�EachGLB
of
isequivalenttosome
Horn�strengtheningof �
Algorithm�searchspaceofHorn�strengthenings�
�
Lemma
Where H
isaHorntheory�andC
isanyclause�
If H
entailsC�
then H
entailssomeHorn�strengtheningofC�
Proof
Bycompletenessofresolution�thereissomeclauseC�that
followsfrom
H
byresolution�suchthatC��C�
BecausetheresolventofHornclausesisHorn�C�
isHorn�
ThereforethereissomeCH
thatisaHorn�strengtheningof
C
suchthatC�
�CH�C�andso Hj�
CH�
�
GLB
�
W
eakestHorn�strengthening
glbisHorn�sobylemma
glbentailssomeHorn�strengthening �of �
glbj�
�j�
But glbisagreatestweakest�Hornlower�bound�
Therefore� glb� ��
�
procedureGLB �
�ComputessomeHorngreatestlower�boundof
�
begin
L��
thelexicographically�rst
Horn�strengtheningof
beginloop
L�
��
lexicographicallynext
Horn�strengtheningof
ifnoneexiststhenexit�
ifLj�
L�thenL��
L�
endloop
removesubsumedclausesfrom
L
returnL
end
��
ExampleofGLB
Algorithm
�
�a�c���a�b�c�d�
Horn�strengthenings�
L��
�a�c���a�b�
L��
�a�c���a�c�
L��
�a�c���a�d�
BecauseL
�j�
L�
L�j�
L�
algorithm
returnsL�
��
PropertiesoftheGLB
algorithm
Anytime�
Algorithm
maybestoppedatanytimeto
�ndalower�boundnotnecessaryaGLB��
Lower�boundimprovesovertime�
LengthofGLB
�lengthoforiginaltheory�
��
ComputingtheLUB
BasicStrategy�
Computeallresolventsoforiginaltheory�
andcollectallHornresolvents�
Problem�E
venaHorntheorycanhaveexponentially
manyHornresolvents�
Solution�R
esolveonlypairsofclausescontaining
atleastonenon�Hornclause�
Methodiscomplete��SelmanandKautz�����
��
ExampleofComputingLUB
�
�a�b���b�c��a�b�
Resolvents�
������
b
������
a�c
Answeris�
a�b���b�c��b
�b�c
Algorithm
doesnotresolve�����
��
PropertiesoftheLUB
algorithm
Anytime�
Nospaceblow�upforHorn�
Canconstructnon�Horntheorieswith
exponentiallylargerLUB�
New
letterscansometimesreducesize�
��
Explosion�anexample
is
CompSci�Phil�Psych��CogSci
ReadsMcCarthy�CompSci�CogSci�
ReadsDennett�Phil�CogSci�
ReadsKosslyn�Psych�CogSci�
LUB
isCompSci�Phil�Psych��CogSci
CompSci�Phil�ReadsKosslyn��CogSci
CompSci�ReadsDennett�Psych��CogSci
���
ReadsMcCarthy�ReadsDennett�ReadsKosslyn��CogSci
SizeLUB
O�n�
Nosmallerequivalentsetofclausesexists�
��
ShrinkingLUB�introducenew
concepts
�
CompSciBuff�CompSci�ReadsMcCarthy�
PhilBuff�Phil�ReadsDennett�
PsychBuff�Psych�ReadsKosslyn�
LUB
becomes
CompSciBuff�PhilBuff�PsychBuff��CogSci
CompSci�CompSciBuff
ReadsMcCarthy�CompSciBuff
Phil�PhilBuff
ReadsDennett�PhilBuff
Psych�PsychBuff
ReadsKosslyn�PsychBuff
SizeLUB
On��
��
New
LUB
andoriginalLUB
areequivalentonqueries
inLang ��
New
conceptscapturewhatcertainpairsofpropositions
haveincommon�
E�g��CompSciBuff�CompSci�ReadsMcCarthy�
So�new
conceptsareusefulgeneralizationsforobtaining
atractableapproximationoftheoriginaltheory�
Formingconceptsforfastinference�
QUESTION�doesasmallperhapsnon�obvious��representation
oftheLUB
alwaysexist�
�
W
hatDoW
eMeanby�Size��
Therearemanyequivalentclausaltheories
� ��yet �exponentiallylarger
Perhaps��size�willmeansizeof
smallestequivalentsetofclauses
Notsu�cientforprovingthatsomething
isinherentlylarge�theremaybe
cleverwaystoencodealargesetofclauses
Structuresharing�
Schemas�etc�
ConsiderthenanyrepresentationoftheHornLUB
thatenablespolytimeinference
�
TheAnswer�
TheredoexisttheorieswhoseHornLUB
isinherentlylarge�
AnyrepresentationoftheLUB
thatenables
polynomialtimeinferenceis
exponentiallylargerthanthetheory�
��
Proof�CircuitComplexity
Non�uniform
P�foreveryn�thereissomepolysize
circuitforinputsoflengthn
equiv�somepolytimealgorithm�
�
samepolynomialforalln�
ProofofinherentlyintractableLUB�s�
Canconstructasingleparticulartheory
whoseLUB
canbeusedtosolve��SAT
foranyformulaoverm
variables
foreachm��
��
�m��inputs�specifyany
LUB �
��SAT
overm
variables
�pm����pm����pm
�p��p��p�
p��p��p�
��
unsatis�able
Circuitcomputes���i�LUB �entails
thatnotalltheinputssetto���canhold
simultaneously�
��
Universal�SAT
Theory
For��SAT
formulasoveragivensetofm
variables�
de�ne�
�
Vfx�y�z��ixyz
jx�y�xareliteralsg
wherethe�i�variablesarenew
Any��CNFformulaoverm
variablescanbe
speci�edasasingleclausemadeup
oftheiinput�variables
��
Example
p���p��p����p��p���p��isunsatis�able
i�
p���p��p���ipp�p� ���p��p���p���ip�p�p� �j�
�ipp�p�
��ip�p�p�
i�
LUB �j�
�ipp�p�
��ip�p�p�
Note�queryisHorn�soLUB
iscomplete�
��
RelationtothePolynomialHierarchy
ExistenceofsmallLUB�sfortheseuniversaltheories
wouldimplyNP�non�uniform
P
WeakerconditionthanP�NP�
Polynomialhierarchycollapsesto �
Stillconsideredtobeprettyunlikely
��
Implications
MaychooseLUB
from
adi�erenttractable
class�thatdoesguaranteeitissmall
k�Horn�boundlengthofHornclauses
Onk
�maxsize
��SAT
�conjunction�literalclauses
NotHorn�p�q�okay
On�
�maxsize
Couldtrytocompiletoseveraldi�erenttractable
classes�pickmostpowerfulclassthatissmall
fortheparticularinputtheory
GLB
andLUB
maybedi�erentkindsoftheories
��
Extensions
Othertargetlanguages�
��SAT
On��maxsize��
k�HornOnk�maxsize��
Clausesnotcontaininggivensetof
�irrelevant�propositions�
�Compilingaway�partsoforiginaltheory�
Similarto�SubramanianandGenesereth������
First�ordersourceandtargetlanguages�
Algorithmsmaynotterminate�
GLB
algorithm�interleavecomparisonofHorn�
strengtheningsandsearch�
��
OtherFormalisms�TerminologicalLogics
TerminologicallogicsClassic�Brachmanetal��
FL�intractable�
FL�
�tractablenorolerestrictions��
CompileFLconceptstoFL�
concepts�
person
�ANDperson�ALL�RESTRfriendmale�
�ANDdoctor�SOMEspecialty���
personwhoseeverymalefriendisadoctorwithaspecialty�
�ANDperson�ALLfriend�ANDdoctor�SOMEspecialty�����
QueryLanguage�TerminologicalLogics
RecentworkbyLenzerini�etal������showsthat
Candeterminesubsumptionbetween
FLconceptand
FL�
concept
inpolynomialtime�
Again�querylanguagecanbemoreexpressive
thantargetlanguage�
�
DoesKnowledgeCompilationReallyW
ork�
Cantheboundsanswer�hard�queries�
oraresuchquerieseasyfororiginaltheory�
Isthereempiricalevidenceofsavings�
Classesconsidered�
Hard�randomlygeneratedtheories
�relativelyunstructureddata�
Planningproblems
�highlystructureddata�
Arecostsalwaysshiftedtocompilationtime�
orcanthecompilationprocessitselfbe�paido����
�
FormalArgumentforSavings
Trivialcase�ifKB
isinconsistent�compilation
detectsthis�allqueriesthenare�free��
A
moreinterestingcase�SupposeKB
isconsistent�
logicallyequivalenttoaHorntheory�but
isnotinHornform�
Therecannotexistatheorem
proverthat
e�cientlyhandlesthisspecialcase
�ValiantandVazirani�����
However�aftercompilationallqueries
canbeansweredinlineartime�
Therefore�KC
doesnotjust�skim�easyqueries�
��
HardRandom
Theories
Testset���random
��CNFtheories�rangingin
sizefrom
��to���variables�
Ratioof���clausespervariableyields
computationallyhardformulas�
�Mitchell�Selman�andLevesque�����
WecomputedtheunitclauseLUB
andGLB
WeakerthantheHornLUB
andGLB�buteasier
tocomputeandanalyze�
Note�unitclauseboundsarealsoHornbounds�
butnotthebestHornbounds�
��
PercentofQueriesAnswered
Basedonthesizeoftheboundsobtained�wecan
exactlycomputethepercentageofall
randomlygeneratedqueriesthatareanswered
bytheboundsalone�
vars
clauses
sizeunit
sizeunit
percentqueriesanswered
LUB
GLB
unit
binary
ternary
��
���
��
��
��
���
���
�
��
��
��
��
��
���
��
��
�
���
��
�
�
�
�
���
���
��
���
���
��
ExecutionTime
Implementedqueryalgorithm�usingtheprogram
Tableau�CrawfordandAuton����
aversionoftheDavis�Putnam
procedure�
tohandlequeriesonwhichboundsfail�
Timeinsecondstoanswer����random
queries
SGIChallenge��
vars
clauses
boundsandtableau
tableauonly
binary
ternary
binary
ternary
��
���
��
��
���
���
�
��
��
��
��
���
��
��
�
��
���
���
�
�
��
��
����
���
��
Random
Formulas�Summary
Knowledgecompilationmightnotbeexpected
toworkonunstructured�random
formulas�
However�evenunitclauseboundsgavegreat
computationalsavings�onaverage�
over���Xfasteron��literalqueries�
On���variabletheories�compilationtime
approx��hour�completelypaidfor
after�����literalqueries�
outperformedoriginalgoalofsimply
shiftingexecutiontimeo��line�
��
PlanningProblems
Domain�Robotmovinginagraph�likeenvironment�
�Pollack�����Hendler����
Movingtocertainnodesconsumesresources�
makingothernodesinaccessible�forbiddenpairs���
Encodedintheplanningassatis�abilityframework
�KautzandSelman�������
Planscorrespondtomodelsofthetheory�
PlanningisNP�completenotjustshortest�path��
��
Example� In going from a to g in at most � steps�
can the robot visit j�
Answer� No by length of shortest path��
MAZE
a c
d
e
h
f
j
g
ki
l
b
��
Example� In going from a to g in at most � steps�
can the robot visit b�
Answer� Yes a model contains path a�b�d�e�f�g��
MAZE
a c
d
e
h
f
j
g
ki
l
b
�
Example� In going from a to g in at most �� steps�
must the robot visit e�
Answer� Yes forbidden pairs eventually block all routes
through area labeled �MAZE���
MAZE
a c
d
e
h
f
j
g
ki
l
b
�
CompilingSAT
EncodingofPlanningDomain
Axiomsformapworldshowninprevious�guresuse
���variables�
������clauses�
Compilingunitboundstakes���hours�
Querytestsets�
RandBin����random
binaryqueries�
RandEver����random
binaryqueries�restricted
topredicatesoftheform
�istheroboteverataspeci�ed�point��
Hand��hand�constructed�non�obvious�queries��
�
PlanningResults
RandBin
RandEver
Hand
numberofqueries
�
�
�
theory
�
onlytime
����
����
����
KC
Querytime
���
��
��LUB
time
��
��
��
KCusing��LUB
time
���
��
��
boundsonlytime
�
�
numberansweredbybounds
��
���
�
Timesinseconds�onanSGIChallenge�
Theory�
has��variables������clauses�
��LUB
time��usingTableauonconjunction
oforiginaltheoryanditsLUB�
��
Observations
BasicKC
Queryalgorithm
increasedspeedby�X
to�X�
Similarbene�tgainedbysimplyconjoiningtheory
anditsLUB�andusingacompletetheorem
prover�
Bestperformance��rsttestagainstbounds if
boundsfail�testagainsttheory�LUB
�
��Xspeedup�
Ifwillingtoignorequeriesnotansweredbythe
bounds�����Xspeedup�handles��!
���!
of
thequeries�
Substitutesensingfortheorem
proving�
��
Conclusions
Wehavebeguntoevaluatecomputationalsavings
possiblewithknowledgecompilationbytheory
approximation�
Boundsanswermanyqueriesthatwouldbehard
toanswerwithanycompletetheorem
prover�
successinshiftingcomputationalcosto��line�
Signi�cantspeed�upoccursonbothunstructured
random�andstructuredplanning�problems�
Goodperformanceobtainedwithunitclause
approximations�
Openquestion�isadditionalcostofcomputing
strongerHornboundsworthwhile�
��
SummaryandConclusions
Introducedknowledgecompilation�
A
proposaltowardsobtaininge�cient
knowledgerepresentationsystems�
Features�
Norestrictionsonexpressivenessofsourcelanguage�
Approximationsbasedontwodelimitingbounds�
Generalizesotherworkonabstractionand
�model�based�reasoning�
Soundandcompleteinference�
E�ciencyimprovesovertime�
Generalityanyextensionalsemantics��
��