Encoding Generalized Quantifiers in
Dependency-based Compositional Semantics
Yubing Dong โ University of Southern California
Ran Tian โ Tohoku University
Yusuke Miyao โ National Institute of Informatics, Japan
Generalized Quantifiers (GQ)
Most students like noodles.
Property-denoting noun phrase
Generalized Quantifier
Generalized Quantifiers (GQ)
Most students like noodles.
Property-denoting noun phrase
PredicateGeneralized Quantifier
Generalized Quantifiers (GQ)
Most (Student) (LikeNoodles) โ {0,1}
DenotationsStudent โ ๐
LikeNoodles โ ๐Binary Relation over ๐
Generalized Quantifiers (GQ)
Most (Student) (LikeNoodles)
iff
๐๐ญ๐ฎ๐๐๐ง๐ญ โฉ ๐๐ข๐ค๐๐๐จ๐จ๐๐ฅ๐๐ฌ
๐๐ญ๐ฎ๐๐๐ง๐ญ> 80%
The relation imposed by a GQ is usually based on the notion โ of set cardinalities
Recognizing Textual Entailment (RTE)
Example:โข ๐1: Mary loves every dog.โข ๐2: Tom has a dog.โข ๐ป: Tom has an animal that Mary loves.โข ๐1, ๐2 โ ๐ป i.e. ๐1 and ๐2 entails ๐ป
Definition: โ๐ entails ๐ป" (๐ โ ๐ป) if, typically, a human
reading ๐ would infer that ๐ป is most likely trueโข Relatively loose, compared to logical entailment
GQ in RTE
The FraCaS Corpus:โข Built in mid-1990sโข A set of hand-crafted entailment problems covering
wide range of semantic phenomena
Section 1 - Generalized Quantifiers:โข 74 problems:
โข 44 have single premise sentenceโข 30 have multiple premise sentence
GQ in RTE
SystemAccuracy
Single Multi Overall
NatLogMacCartney07 84.1%
N/AMacCartney08 97.7%
CCG-DistParser Syntax 70.5% 50.0% 62.2%
Gold Syntax 88.6% 80.0% 85.1%
Accuracies of previous systems on Section 1 of FraCaS corpus
GQ in RTE
SystemAccuracy
Single Multi Overall
NatLogMacCartney07 84.1%
N/AMacCartney08 97.7%
CCG-DistParser Syntax 70.5% 50.0% 62.2%
Gold Syntax 88.6% 80.0% 85.1%
TIFMO
Baseline 79.5% 86.7% 82.4%
Selection 90.9% 93.3% 91.9%
Relation 88.6% 93.3% 90.5%
Selection+Relation 93.2% 96.7% 94.6%
Accuracies of previous systems on Section 1 of FraCaS corpus
Properties of GQsProblem with encoding the โperfect semanticsโ
Most (Student) (LikeNoodles)
iff
๐๐ญ๐ฎ๐๐๐ง๐ญ โฉ ๐๐ข๐ค๐๐๐จ๐จ๐๐ฅ๐๐ฌ
๐๐ญ๐ฎ๐๐๐ง๐ญ> 80%
Challenge: set cardinalities are difficult to perfectly encode
Properties of GQs
Compromise: only encode major GQ propertiesโข Interaction with universal and existential quantificationsโข Conservativityโข Monotonicity
Properties of GQsInteraction with universal and existential quantifications
Case 1:๐ด โ ๐ต โ ๐น ๐ด ๐ต โ ๐ด โฉ ๐ต โ โ
Example: โmostโ
All students like noodles.
Most students like noodles.
There are students who like noodles.
Properties of GQsInteraction with universal and existential quantifications
Case 2:๐ด โ ๐ต โ ๐น ๐ด ๐ต โ ๐ด โฉ ๐ต โ โ
Example: โa lot ofโ
All students like noodles.
A lot of students like noodles.
There are students who like noodles.
Properties of GQsInteraction with universal and existential quantifications
Case 3:๐ด โ ๐ต โ ๐น ๐ด ๐ต โ ๐ด โฉ ๐ต โ โ
Example: โat most nโ
All students like noodles.
At most 5 students like noodles.
There are students who like noodles.
Properties of GQsConservativity
The โdomain restrainingโ role of the noun argumentโข Eliminates objects that do not have the noun propertyโข Only need to consider which of the rest has the predicate property
๐น ๐ด ๐ต โบ ๐น(๐ด)(๐ด โฉ ๐ต)
Example:โข โFew apples are toxic.โโบโFew apples are toxic apples.โโข We donโt care non-apples toxicants, e.g. toxic oranges
Properties of GQsMonotonicity
A GQ ๐น โ โ is upward entailing in the noun argument if:๐น ๐ดโฒ ๐ต โ ๐น ๐ด ๐ต โ๐ดโฒ โ ๐ด
Similarly, a GQ can also beโข downward entailing in the noun argument, and โข upward/downward entailing in the predicate argument
Properties of GQsMonotonicity
At most 5 students like noodles.
At most 5 Japanese students like udon noodles.
Example: โat most ๐โ is downward entailing in each argument
Properties of GQsMonotonicity
Example: โat least ๐โ is upward entailing in each argument
At least 5 students like noodles.
At least 5 Japanese students like udon noodles.
Properties of GQsMonotonicity
Example: โmostโ is neither upward nor downward entailing in the noun argument
Most students like noodles.
Most Japanese students like noodles.
Properties of GQsMonotonicity
Example: but is upward entailing in the predicate argument
Most students like noodles.
Most students like udon noodles.
DCS for RTE
DCS tree for โAll students like udon noodlesโ
Abstract Denotations:
๐ง๐จ๐จ๐๐ฅ๐ โ ๐๐ฎ๐๐จ๐ง โ ๐๐ฌ๐ญ๐ฎ๐๐๐ง๐ญ โ ๐๐ฅ๐ข๐ค๐ โ ๐ ร๐
DCS for RTE
๐ท1 = ๐ง๐จ๐จ๐๐ฅ๐ โฉ ๐ฎ๐๐จ๐ง
DCS tree for โAll students like udon noodlesโ
โudon noodlesโ
DCS for RTE
๐ท1 = ๐ง๐จ๐จ๐๐ฅ๐ โฉ ๐ฎ๐๐จ๐ง
๐ท2 = ๐ฅ๐ข๐ค๐ โฉ ๐๐๐ต๐ฝ ร ๐ท1 ๐๐ต๐ฝ
DCS tree for โAll students like udon noodlesโ
โlike udon noodlesโ
DCS for RTE
๐ท1 = ๐ง๐จ๐จ๐๐ฅ๐ โฉ ๐ฎ๐๐จ๐ง
๐ท2 = ๐ฅ๐ข๐ค๐ โฉ ๐๐๐ต๐ฝ ร ๐ท1 ๐๐ต๐ฝ
๐ท3 = ๐๐๐ต๐ฝ ๐ท2
DCS tree for โAll students like udon noodlesโ
โsubjects who like udon noodlesโ
DCS for RTE
๐ท1 = ๐ง๐จ๐จ๐๐ฅ๐ โฉ ๐ฎ๐๐จ๐ง
๐ท2 = ๐ฅ๐ข๐ค๐ โฉ ๐๐๐ต๐ฝ ร ๐ท1 ๐๐ต๐ฝ
๐ท3 = ๐๐๐ต๐ฝ ๐ท2๐ท4 = ๐โ
๐๐ต๐ฝ ๐ท3, ๐ฌ๐ญ๐ฎ๐๐๐ง๐ญ
DCS tree for โAll students like udon noodlesโ
qโr R,C โก x โ โ Rโฉ x รWr โ x รCr
If ๐ and ๐ถ have the same dimension,โข ๐โ
๐ ๐ , ๐ถ = โ (0-dimension point set) when ๐ถ โ ๐ ,โข ๐โ
๐ ๐ , ๐ถ = โ otherwise
wide reading of โโโ
DCS for RTE
๐ท1 = ๐ง๐จ๐จ๐๐ฅ๐ โฉ ๐ฎ๐๐จ๐ง
๐ท2 = ๐ฅ๐ข๐ค๐ โฉ ๐๐๐ต๐ฝ ร ๐ท1 ๐๐ต๐ฝ
๐ท3 = ๐๐๐ต๐ฝ ๐ท2๐ท4 = ๐โ
๐๐ต๐ฝ ๐ท3, ๐ฌ๐ญ๐ฎ๐๐๐ง๐ญ
๐ท5 = ๐โ๐๐ต๐ฝ
๐ท2, ๐ฌ๐ญ๐ฎ๐๐๐ง๐ญ
DCS tree for โAll students like udon noodlesโ
qโr R,C โก x โ โ Rโฉ x รWr โ x รCr
If ๐ and ๐ถ have the same dimension,โข ๐โ
๐ ๐ , ๐ถ = โ (0-dimension point set) when ๐ถ โ ๐ ,โข ๐โ
๐ ๐ , ๐ถ = โ otherwise
narrow reading of โโโ(โthe set of udon noodles that all student likeโ)
DCS for RTE
๐ท1 = ๐ง๐จ๐จ๐๐ฅ๐ โฉ ๐ฎ๐๐จ๐ง
๐ท2 = ๐ฅ๐ข๐ค๐ โฉ ๐๐๐ต๐ฝ ร ๐ท1 ๐๐ต๐ฝ
๐ท3 = ๐๐๐ต๐ฝ ๐ท2๐ท4 = ๐โ
๐๐ต๐ฝ ๐ท3, ๐ฌ๐ญ๐ฎ๐๐๐ง๐ญ
๐ท5 = ๐โ๐๐ต๐ฝ
๐ท2, ๐ฌ๐ญ๐ฎ๐๐๐ง๐ญ
DCS tree for โAll students like udon noodlesโ
Prove statement
โข ๐ท4 โ โ (wide reading) orโข ๐ท5 โ โ (narrow reading)
using forward chaining
DCS for RTE
Basic operators / functions:โข ร - Cartesian product of setsโข โฉ - Set intersectionโข ๐๐ - Projection onto domain of semantic role ๐โข ๐๐ - Relabelingโข ๐โ
๐ - Division
Basic types of statements:โข Non-emptiness: ๐ด โ โ โข Subsumption: ๐ด โ ๐ต
DCS for RTE: the selection operator
โข Introduced as an extension to represent the generalized selection operation in relational algebra
โข Marked on a DCS tree nodeโข Wrap the abstract denotation ๐ท to form a new abstract
denotation ๐ ๐ ๐ท
โข The properties of ๐ ๐ ๐ท can be user defined
Example:
the set of highest mountains: ๐ โ๐๐โ๐๐ ๐ก(๐ฆ๐จ๐ฎ๐ง๐ญ๐๐ข๐ง)
Encoding GQs as Selections
We encode a GQ ๐น using selection ๐ ๐น as:
๐น ๐ด ๐ต โก ๐ ๐น ๐ด โ ๐ต
Basic requirement:โข ๐น should be upward-entailing in the predicate
argument ๐ตโข A major limitation
Encoding GQs as Selections
๐น ๐ด ๐ต โก ๐ ๐น ๐ด โ ๐ต
โข Entailment from universal quantification now written as:๐ด โ ๐ต โ ๐ ๐น ๐ด โ ๐ต
โข Conservativity as:๐ ๐น ๐ด โ ๐ด โฉ ๐ต โ ๐ ๐น ๐ด โ ๐ต
โข Both hold if we add axiom:๐ ๐น ๐ด โ ๐ด
Encoding GQs as Selections
๐น ๐ด ๐ต โก ๐ ๐น ๐ด โ ๐ต
โข Entailment to existence quantification now written as:๐ ๐น ๐ด โ ๐ต โ ๐ด โฉ ๐ต โ โ
โข Holds if we add axiom:๐ ๐น ๐ด โฉ ๐ด โ โ
Encoding GQs as Selections
๐น ๐ด ๐ต โก ๐ ๐น ๐ด โ ๐ต
โข Monotonicity in the noun argument ๐ด (e.g. upward) now written as:
A โ Aโฒ โง ๐ ๐น ๐ด โ ๐ต โ ๐ ๐น ๐ดโฒ โ ๐ต
โข Holds if we add axiom:A โ Aโฒ โ ๐ ๐น ๐ด โ ๐ ๐น ๐ดโฒ
DCS tree for โAt least 5 students like udon noodles.โwhere the GQ โat least 5โ is encoded as selection ๐ ๐ด๐ก๐ฟ๐๐๐ ๐ก 5
Encoding GQs as Selections
Example: at least ๐
โข Satisfied: upward-entailing in predicate argument
โข Entails existential quantification:โ๐ด ๐ ๐ด๐ก๐ฟ๐๐๐ ๐ก 5 ๐ด โฉ ๐ด โ โ
โข Upward-entailing in noun argument:โ๐ด, ๐ดโฒ ๐ . t. A โ Aโฒ
๐ ๐ด๐ก๐ฟ๐๐๐ ๐ก 5 ๐ด โ ๐ ๐ด๐ก๐ฟ๐๐๐ ๐ก 5 ๐ดโฒ
Encoding GQs as SelectionsExample:
โAt least 5 Japanese students like udon noodles.โ
โ โ At least 5 students like noodles.โ
๐ท1 = ๐ง๐จ๐จ๐๐ฅ๐ โฉ ๐ฎ๐๐จ๐ง
๐ท2 = ๐ฅ๐ข๐ค๐ โฉ ๐๐๐ต๐ฝ ร ๐ท1 ๐๐ต๐ฝ
๐ท3 = ๐๐๐ต๐ฝ ๐ท2
๐ท3โฒ = ๐๐๐ต๐ฝ ๐ฅ๐ข๐ค๐ โฉ ๐๐๐ต๐ฝ ร ๐ง๐จ๐จ๐๐ฅ๐๐๐ต๐ฝ
Encoding GQs as RelationsIntro to Relations
โข Review: GQ can be seen as binary relation over 2๐
โข Therefore, we introduce a new extension: relationโข A new type of statementโข A relation ๐๐น ๐ด, ๐ต can represent arbitrary custom
relation between abstract denotations ๐ด and ๐ต
Encoding GQs as RelationsIntro to Relations
Relation ๐๐น ๐ด, ๐ต
โข The inference engine keeps track of which term pairs are labeled with which relationsโข Does ๐ด and ๐ต have relation ๐๐น?โข What terms have relation ๐๐น to ๐ด?
โข Supports custom axioms for a relationโข What entails ๐๐น ๐ด, ๐ต ?โข What does ๐๐น ๐ด, ๐ต entail?
Encoding GQs as Relations
We intuitively encode a GQ ๐น using relation ๐๐น as:
๐น ๐ด ๐ต โก r๐น ๐ด, ๐ต
๐ท1 = ๐ง๐จ๐จ๐๐ฅ๐ โฉ ๐ฎ๐๐จ๐ง
๐ท2 = ๐ฅ๐ข๐ค๐ โฉ ๐๐๐ต๐ฝ ร ๐ท1 ๐๐ต๐ฝ
๐ท3 = ๐๐๐ต๐ฝ ๐ท2
Statement:๐๐ด๐ก๐๐๐ ๐ก 5 ๐ฌ๐ญ๐ฎ๐๐๐ง๐ญ, ๐ท3
Encoding GQs as Relations
๐น ๐ด ๐ต โก r๐น ๐ด, ๐ต
โข Entailment from universal quantification:๐ด โ ๐ต โ ๐๐น ๐ด, ๐ต
โข Entailment to existential quantification:๐๐น ๐ด, ๐ต โ ๐ด โฉ ๐ต โ โ
โข Monotonicity (e.g. downward in both arguments):๐๐น ๐ด, ๐ต โง ๐ด โ ๐ดโฒ โง ๐ต โ ๐ตโฒ โ ๐๐น ๐ดโฒ, ๐ตโฒ
Encoding GQs as Relations
๐น ๐ด ๐ต โก r๐น ๐ด, ๐ต
โข Conservativity:๐๐น ๐ด, ๐ต โ ๐๐น ๐ด, ๐ด โฉ ๐ต
โข How about the other direction?๐๐น ๐ด, ๐ด โฉ ๐ต โ ๐๐น ๐ด, ๐ต
Encoding GQs as Relations
๐๐น ๐ด, ๐ด โฉ ๐ต โ ๐๐น ๐ด, ๐ต
Challenge:โข The inference engine is based on forward chaining:
โข Always try to deduce all possible implications from given premisesโข Efficientโข Opens the possibility of adapting DCS for entailment
generation
Encoding GQs as Relations
๐๐น ๐ด, ๐ด โฉ ๐ต โ ๐๐น ๐ด, ๐ต
Challenge:โข The inference engine is based on forward chainingโข Therefore itโs infeasible to enumerate all forms ๐ = ๐ด โฉ ๐ต
when ๐๐น ๐ด, ๐ is claimedโข Number of possibilities explodes exponentially
โข e.g. ๐ = ๐ โฉ ๐ถ โ๐ถ, ๐ = ๐ด โฉ ๐ต โฉ ๐ถ = ๐ด โฉ ๐ต โฉ ๐ถ
Encoding GQs as Relations
๐๐น ๐ด, ๐ด โฉ ๐ต โ ๐๐น ๐ด, ๐ต
Implementation: limit search using conditions ๐ โ ๐ด โง ๐ โ ๐ต
If ๐๐น ๐ด, ๐ and ๐ โ ๐ด:โข For each ๐ต โ ๐:
โข Check if ๐ = ๐ด โฉ ๐ต
We emphasize this detail because formal semantic researchers are often not aware of these difficulties.
Encoding GQs as RelationsLimitations
๐น ๐ด ๐ต โก r๐น ๐ด, ๐ต
Limitation:
Relations in DCS trees are always explained as having the widest scope, hence cannot deal with multiple relations in a sentence.
Encoding GQs as RelationsLimitations
Example:๐: At most 10 commissioners spend a lot of time at home.
We want to state๐๐ด๐ก๐๐๐ ๐ก 10 ๐๐จ๐ฆ๐ข๐ฌ๐ฌ๐ข๐จ๐ง๐๐ซ๐ฌ, ๐ท
where ๐ท = โpeople who spend a lot of time at homeโ
But this is impossible if โa lot ofโ is also encoded as a relation
Encoding GQs as RelationsLimitations
Example:๐๐ด๐ก๐๐๐ ๐ก 10 ๐๐จ๐ฆ๐ข๐ฌ๐ฌ๐ข๐จ๐ง๐๐ซ๐ฌ, ๐ท
๐ท = "people who spend a lot of time at home"
Workaround:Since โa lot ofโ is upward-entailing in predicate argument, we can encode it using selection ๐ ๐ด๐ฟ๐๐ก๐๐, while still encode โat
most 10โ using ๐๐ด๐ก๐๐๐ ๐ก 10
Encoding GQs as RelationsLimitations
Example:๐๐ด๐ก๐๐๐ ๐ก 10 ๐๐จ๐ฆ๐ข๐ฌ๐ฌ๐ข๐จ๐ง๐๐ซ๐ฌ, ๐ท
๐ท = ๐โ๐๐ต๐ฝ
๐ทโฒ, ๐ ๐ด๐ฟ๐๐ก๐๐ ๐ญ๐ข๐ฆ๐
where
๐ทโฒ = ๐ฌ๐ฉ๐๐ง๐ โฉ ๐๐๐ต๐ฝ ร๐๐๐ต๐ฝ ร ๐ก๐จ๐ฆ๐๐๐๐ท
(โspend at homeโ)
EvaluationSet-up
The FraCaS Corpus:โข Built in mid-1990sโข A set of hand-crafted entailment problems covering
wide range of semantic phenomena
Section 1 - Generalized Quantifiers:โข 74 problems:
โข 44 have single premise sentenceโข 30 have multiple premise sentence
EvaluationSet-up
Settings:โข Baseline
โข Simply drop GQsโข Same tree structure as follows
โข Selectionโข Relationโข Selection+Relation
EvaluationSet-up
Settings:โข Baselineโข Selection
โข Implement all GQs as selections, even for those that are downward-entailing in predicate argument
โข Relationโข Selection+Relation
EvaluationSet-up
Settings:โข Baselineโข Selectionโข Relation
โข Implement all GQs as relationsโข Selection+Relation
EvaluationSet-up
Settings:โข Baselineโข Selectionโข Relationโข Selection+Relation
โข Use relations to encode GQs that are downward-entailing in predicate argument
โข Encode the rest with selections
Evaluation
SystemAccuracy
Single Multi Overall
NatLogMacCartney07 84.1%
N/AMacCartney08 97.7%
CCG-DistParser Syntax 70.5% 50.0% 62.2%
Gold Syntax 88.6% 80.0% 85.1%
TIFMO
Baseline 79.5% 86.7% 82.4%
Selection 90.9% 93.3% 91.9%
Relation 88.6% 93.3% 90.5%
Selection+Relation 93.2% 96.7% 94.6%
Accuracies of previous systems on Section 1 of FraCaS corpus
Conclusion
โข Generalized Quantifiers are important (for RTE)
โข We explored ways of encoding GQs in DCS for RTEโข via selection extensionโข via relation extension (newly proposed)
โข Significant improvement in performance, but not perfectโข which suggests towards more powerful logical systems