Upload
yubing-dong
View
265
Download
2
Embed Size (px)
Citation preview
Encoding Generalized Quantifiers in
Dependency-based Compositional Semantics
Yubing Dong – University of Southern California
Ran Tian – Tohoku University
Yusuke Miyao – National Institute of Informatics, Japan
Generalized Quantifiers (GQ)
Most students like noodles.
Property-denoting noun phrase
Generalized Quantifier
Generalized Quantifiers (GQ)
Most students like noodles.
Property-denoting noun phrase
PredicateGeneralized Quantifier
Generalized Quantifiers (GQ)
Most (Student) (LikeNoodles) ∈ {0,1}
DenotationsStudent ⊆ 𝑊
LikeNoodles ⊆ 𝑊Binary Relation over 𝑊
Generalized Quantifiers (GQ)
Most (Student) (LikeNoodles)
iff
𝐒𝐭𝐮𝐝𝐞𝐧𝐭 ∩ 𝐋𝐢𝐤𝐞𝐍𝐨𝐨𝐝𝐥𝐞𝐬
𝐒𝐭𝐮𝐝𝐞𝐧𝐭> 80%
The relation imposed by a GQ is usually based on the notion ⋅ of set cardinalities
Recognizing Textual Entailment (RTE)
Example:• 𝑇1: Mary loves every dog.• 𝑇2: Tom has a dog.• 𝐻: Tom has an animal that Mary loves.• 𝑇1, 𝑇2 ⇒ 𝐻 i.e. 𝑇1 and 𝑇2 entails 𝐻
Definition: “𝑇 entails 𝐻" (𝑇 ⇒ 𝐻) if, typically, a human
reading 𝑇 would infer that 𝐻 is most likely true• Relatively loose, compared to logical entailment
GQ in RTE
The FraCaS Corpus:• Built in mid-1990s• A set of hand-crafted entailment problems covering
wide range of semantic phenomena
Section 1 - Generalized Quantifiers:• 74 problems:
• 44 have single premise sentence• 30 have multiple premise sentence
GQ in RTE
SystemAccuracy
Single Multi Overall
NatLogMacCartney07 84.1%
N/AMacCartney08 97.7%
CCG-DistParser Syntax 70.5% 50.0% 62.2%
Gold Syntax 88.6% 80.0% 85.1%
Accuracies of previous systems on Section 1 of FraCaS corpus
GQ in RTE
SystemAccuracy
Single Multi Overall
NatLogMacCartney07 84.1%
N/AMacCartney08 97.7%
CCG-DistParser Syntax 70.5% 50.0% 62.2%
Gold Syntax 88.6% 80.0% 85.1%
TIFMO
Baseline 79.5% 86.7% 82.4%
Selection 90.9% 93.3% 91.9%
Relation 88.6% 93.3% 90.5%
Selection+Relation 93.2% 96.7% 94.6%
Accuracies of previous systems on Section 1 of FraCaS corpus
Properties of GQsProblem with encoding the “perfect semantics”
Most (Student) (LikeNoodles)
iff
𝐒𝐭𝐮𝐝𝐞𝐧𝐭 ∩ 𝐋𝐢𝐤𝐞𝐍𝐨𝐨𝐝𝐥𝐞𝐬
𝐒𝐭𝐮𝐝𝐞𝐧𝐭> 80%
Challenge: set cardinalities are difficult to perfectly encode
Properties of GQs
Compromise: only encode major GQ properties• Interaction with universal and existential quantifications• Conservativity• Monotonicity
Properties of GQsInteraction with universal and existential quantifications
Case 1:𝐴 ⊆ 𝐵 ⇒ 𝐹 𝐴 𝐵 ⇒ 𝐴 ∩ 𝐵 ≠ ∅
Example: “most”
All students like noodles.
Most students like noodles.
There are students who like noodles.
Properties of GQsInteraction with universal and existential quantifications
Case 2:𝐴 ⊆ 𝐵 ⇒ 𝐹 𝐴 𝐵 ⇒ 𝐴 ∩ 𝐵 ≠ ∅
Example: “a lot of”
All students like noodles.
A lot of students like noodles.
There are students who like noodles.
Properties of GQsInteraction with universal and existential quantifications
Case 3:𝐴 ⊆ 𝐵 ⇒ 𝐹 𝐴 𝐵 ⇒ 𝐴 ∩ 𝐵 ≠ ∅
Example: “at most n”
All students like noodles.
At most 5 students like noodles.
There are students who like noodles.
Properties of GQsConservativity
The “domain restraining” role of the noun argument• Eliminates objects that do not have the noun property• Only need to consider which of the rest has the predicate property
𝐹 𝐴 𝐵 ⟺ 𝐹(𝐴)(𝐴 ∩ 𝐵)
Example:• “Few apples are toxic.”⟺“Few apples are toxic apples.”• We don’t care non-apples toxicants, e.g. toxic oranges
Properties of GQsMonotonicity
A GQ 𝐹 ⋅ ⋅ is upward entailing in the noun argument if:𝐹 𝐴′ 𝐵 ⇒ 𝐹 𝐴 𝐵 ∀𝐴′ ⊆ 𝐴
Similarly, a GQ can also be• downward entailing in the noun argument, and • upward/downward entailing in the predicate argument
Properties of GQsMonotonicity
At most 5 students like noodles.
At most 5 Japanese students like udon noodles.
Example: “at most 𝑛” is downward entailing in each argument
Properties of GQsMonotonicity
Example: “at least 𝑛” is upward entailing in each argument
At least 5 students like noodles.
At least 5 Japanese students like udon noodles.
Properties of GQsMonotonicity
Example: “most” is neither upward nor downward entailing in the noun argument
Most students like noodles.
Most Japanese students like noodles.
Properties of GQsMonotonicity
Example: but is upward entailing in the predicate argument
Most students like noodles.
Most students like udon noodles.
DCS for RTE
DCS tree for “All students like udon noodles”
Abstract Denotations:
𝐧𝐨𝐨𝐝𝐥𝐞 ⊆ 𝑊𝐮𝐝𝐨𝐧 ⊆ 𝑊𝐬𝐭𝐮𝐝𝐞𝐧𝐭 ⊆ 𝑊𝐥𝐢𝐤𝐞 ⊆ 𝑊 ×𝑊
DCS for RTE
𝐷1 = 𝐧𝐨𝐨𝐝𝐥𝐞 ∩ 𝐮𝐝𝐨𝐧
𝐷2 = 𝐥𝐢𝐤𝐞 ∩ 𝑊𝑆𝐵𝐽 × 𝐷1 𝑂𝐵𝐽
DCS tree for “All students like udon noodles”
“like udon noodles”
DCS for RTE
𝐷1 = 𝐧𝐨𝐨𝐝𝐥𝐞 ∩ 𝐮𝐝𝐨𝐧
𝐷2 = 𝐥𝐢𝐤𝐞 ∩ 𝑊𝑆𝐵𝐽 × 𝐷1 𝑂𝐵𝐽
𝐷3 = 𝜋𝑆𝐵𝐽 𝐷2
DCS tree for “All students like udon noodles”
“subjects who like udon noodles”
DCS for RTE
𝐷1 = 𝐧𝐨𝐨𝐝𝐥𝐞 ∩ 𝐮𝐝𝐨𝐧
𝐷2 = 𝐥𝐢𝐤𝐞 ∩ 𝑊𝑆𝐵𝐽 × 𝐷1 𝑂𝐵𝐽
𝐷3 = 𝜋𝑆𝐵𝐽 𝐷2𝐷4 = 𝑞⊆
𝑆𝐵𝐽 𝐷3, 𝐬𝐭𝐮𝐝𝐞𝐧𝐭
DCS tree for “All students like udon noodles”
q⊆r R,C ≡ x ∅≠R∩ x ×Wr ⊆ x ×Cr
If 𝑅 and 𝐶 have the same dimension,• 𝑞⊆
𝑟 𝑅, 𝐶 = ∗ (0-dimension point set) when 𝐶 ⊆ 𝑅,• 𝑞⊆
𝑟 𝑅, 𝐶 = ∅ otherwise
wide reading of “⊆”
DCS for RTE
𝐷1 = 𝐧𝐨𝐨𝐝𝐥𝐞 ∩ 𝐮𝐝𝐨𝐧
𝐷2 = 𝐥𝐢𝐤𝐞 ∩ 𝑊𝑆𝐵𝐽 × 𝐷1 𝑂𝐵𝐽
𝐷3 = 𝜋𝑆𝐵𝐽 𝐷2𝐷4 = 𝑞⊆
𝑆𝐵𝐽 𝐷3, 𝐬𝐭𝐮𝐝𝐞𝐧𝐭
𝐷5 = 𝑞⊆𝑆𝐵𝐽
𝐷2, 𝐬𝐭𝐮𝐝𝐞𝐧𝐭
DCS tree for “All students like udon noodles”
q⊆r R,C ≡ x ∅≠R∩ x ×Wr ⊆ x ×Cr
If 𝑅 and 𝐶 have the same dimension,• 𝑞⊆
𝑟 𝑅, 𝐶 = ∗ (0-dimension point set) when 𝐶 ⊆ 𝑅,• 𝑞⊆
𝑟 𝑅, 𝐶 = ∅ otherwise
narrow reading of “⊆”(“the set of udon noodles that all student like”)
DCS for RTE
𝐷1 = 𝐧𝐨𝐨𝐝𝐥𝐞 ∩ 𝐮𝐝𝐨𝐧
𝐷2 = 𝐥𝐢𝐤𝐞 ∩ 𝑊𝑆𝐵𝐽 × 𝐷1 𝑂𝐵𝐽
𝐷3 = 𝜋𝑆𝐵𝐽 𝐷2𝐷4 = 𝑞⊆
𝑆𝐵𝐽 𝐷3, 𝐬𝐭𝐮𝐝𝐞𝐧𝐭
𝐷5 = 𝑞⊆𝑆𝐵𝐽
𝐷2, 𝐬𝐭𝐮𝐝𝐞𝐧𝐭
DCS tree for “All students like udon noodles”
Prove statement
• 𝐷4 ≠ ∅ (wide reading) or• 𝐷5 ≠ ∅ (narrow reading)
using forward chaining
DCS for RTE
Basic operators / functions:• × - Cartesian product of sets• ∩ - Set intersection• 𝜋𝑟 - Projection onto domain of semantic role 𝑟• 𝑙𝑟 - Relabeling• 𝑞⊆
𝑟 - Division
Basic types of statements:• Non-emptiness: 𝐴 ≠ ∅• Subsumption: 𝐴 ⊆ 𝐵
DCS for RTE: the selection operator
• Introduced as an extension to represent the generalized selection operation in relational algebra
• Marked on a DCS tree node• Wrap the abstract denotation 𝐷 to form a new abstract
denotation 𝑠𝑓 𝐷
• The properties of 𝑠𝑓 𝐷 can be user defined
Example:
the set of highest mountains: 𝑠ℎ𝑖𝑔ℎ𝑒𝑠𝑡(𝐦𝐨𝐮𝐧𝐭𝐚𝐢𝐧)
Encoding GQs as Selections
We encode a GQ 𝐹 using selection 𝑠𝐹 as:
𝐹 𝐴 𝐵 ≡ 𝑠𝐹 𝐴 ⊆ 𝐵
Basic requirement:• 𝐹 should be upward-entailing in the predicate
argument 𝐵• A major limitation
Encoding GQs as Selections
𝐹 𝐴 𝐵 ≡ 𝑠𝐹 𝐴 ⊆ 𝐵
• Entailment from universal quantification now written as:𝐴 ⊆ 𝐵 ⇒ 𝑠𝐹 𝐴 ⊆ 𝐵
• Conservativity as:𝑠𝐹 𝐴 ⊆ 𝐴 ∩ 𝐵 ⇔ 𝑠𝐹 𝐴 ⊆ 𝐵
• Both hold if we add axiom:𝑠𝐹 𝐴 ⊆ 𝐴
Encoding GQs as Selections
𝐹 𝐴 𝐵 ≡ 𝑠𝐹 𝐴 ⊆ 𝐵
• Entailment to existence quantification now written as:𝑠𝐹 𝐴 ⊆ 𝐵 ⇒ 𝐴 ∩ 𝐵 ≠ ∅
• Holds if we add axiom:𝑠𝐹 𝐴 ∩ 𝐴 ≠ ∅
Encoding GQs as Selections
𝐹 𝐴 𝐵 ≡ 𝑠𝐹 𝐴 ⊆ 𝐵
• Monotonicity in the noun argument 𝐴 (e.g. upward) now written as:
A ⊆ A′ ∧ 𝑠𝐹 𝐴 ⊆ 𝐵 ⇒ 𝑠𝐹 𝐴′ ⊆ 𝐵
• Holds if we add axiom:A ⊆ A′ ⇒ 𝑠𝐹 𝐴 ⊇ 𝑠𝐹 𝐴′
DCS tree for “At least 5 students like udon noodles.”where the GQ “at least 5” is encoded as selection 𝑠𝐴𝑡𝐿𝑒𝑎𝑠𝑡 5
Encoding GQs as Selections
Example: at least 𝑛
• Satisfied: upward-entailing in predicate argument
• Entails existential quantification:∀𝐴 𝑠𝐴𝑡𝐿𝑒𝑎𝑠𝑡 5 𝐴 ∩ 𝐴 ≠ ∅
• Upward-entailing in noun argument:∀𝐴, 𝐴′ 𝑠. t. A ⊆ A′
𝑠𝐴𝑡𝐿𝑒𝑎𝑠𝑡 5 𝐴 ⊇ 𝑠𝐴𝑡𝐿𝑒𝑎𝑠𝑡 5 𝐴′
Encoding GQs as SelectionsExample:
“At least 5 Japanese students like udon noodles.”
⇒ “ At least 5 students like noodles.”
𝐷1 = 𝐧𝐨𝐨𝐝𝐥𝐞 ∩ 𝐮𝐝𝐨𝐧
𝐷2 = 𝐥𝐢𝐤𝐞 ∩ 𝑊𝑆𝐵𝐽 × 𝐷1 𝑂𝐵𝐽
𝐷3 = 𝜋𝑆𝐵𝐽 𝐷2
𝐷3′ = 𝜋𝑆𝐵𝐽 𝐥𝐢𝐤𝐞 ∩ 𝑊𝑆𝐵𝐽 × 𝐧𝐨𝐨𝐝𝐥𝐞𝑂𝐵𝐽
Encoding GQs as RelationsIntro to Relations
• Review: GQ can be seen as binary relation over 2𝑊
• Therefore, we introduce a new extension: relation• A new type of statement• A relation 𝑟𝐹 𝐴, 𝐵 can represent arbitrary custom
relation between abstract denotations 𝐴 and 𝐵
Encoding GQs as RelationsIntro to Relations
Relation 𝑟𝐹 𝐴, 𝐵
• The inference engine keeps track of which term pairs are labeled with which relations• Does 𝐴 and 𝐵 have relation 𝑟𝐹?• What terms have relation 𝑟𝐹 to 𝐴?
• Supports custom axioms for a relation• What entails 𝑟𝐹 𝐴, 𝐵 ?• What does 𝑟𝐹 𝐴, 𝐵 entail?
Encoding GQs as Relations
We intuitively encode a GQ 𝐹 using relation 𝑟𝐹 as:
𝐹 𝐴 𝐵 ≡ r𝐹 𝐴, 𝐵
𝐷1 = 𝐧𝐨𝐨𝐝𝐥𝐞 ∩ 𝐮𝐝𝐨𝐧
𝐷2 = 𝐥𝐢𝐤𝐞 ∩ 𝑊𝑆𝐵𝐽 × 𝐷1 𝑂𝐵𝐽
𝐷3 = 𝜋𝑆𝐵𝐽 𝐷2
Statement:𝑟𝐴𝑡𝑀𝑜𝑠𝑡 5 𝐬𝐭𝐮𝐝𝐞𝐧𝐭, 𝐷3
Encoding GQs as Relations
𝐹 𝐴 𝐵 ≡ r𝐹 𝐴, 𝐵
• Entailment from universal quantification:𝐴 ⊆ 𝐵 ⇒ 𝑟𝐹 𝐴, 𝐵
• Entailment to existential quantification:𝑟𝐹 𝐴, 𝐵 ⇒ 𝐴 ∩ 𝐵 ≠ ∅
• Monotonicity (e.g. downward in both arguments):𝑟𝐹 𝐴, 𝐵 ∧ 𝐴 ⊇ 𝐴′ ∧ 𝐵 ⊇ 𝐵′ ⇒ 𝑟𝐹 𝐴′, 𝐵′
Encoding GQs as Relations
𝐹 𝐴 𝐵 ≡ r𝐹 𝐴, 𝐵
• Conservativity:𝑟𝐹 𝐴, 𝐵 ⇒ 𝑟𝐹 𝐴, 𝐴 ∩ 𝐵
• How about the other direction?𝑟𝐹 𝐴, 𝐴 ∩ 𝐵 ⇒ 𝑟𝐹 𝐴, 𝐵
Encoding GQs as Relations
𝑟𝐹 𝐴, 𝐴 ∩ 𝐵 ⇒ 𝑟𝐹 𝐴, 𝐵
Challenge:• The inference engine is based on forward chaining:
• Always try to deduce all possible implications from given premises• Efficient• Opens the possibility of adapting DCS for entailment
generation
Encoding GQs as Relations
𝑟𝐹 𝐴, 𝐴 ∩ 𝐵 ⇒ 𝑟𝐹 𝐴, 𝐵
Challenge:• The inference engine is based on forward chaining• Therefore it’s infeasible to enumerate all forms 𝑋 = 𝐴 ∩ 𝐵
when 𝑟𝐹 𝐴, 𝑋 is claimed• Number of possibilities explodes exponentially
• e.g. 𝑋 = 𝑋 ∩ 𝐶 ∀𝐶, 𝑋 = 𝐴 ∩ 𝐵 ∩ 𝐶 = 𝐴 ∩ 𝐵 ∩ 𝐶
Encoding GQs as Relations
𝑟𝐹 𝐴, 𝐴 ∩ 𝐵 ⇒ 𝑟𝐹 𝐴, 𝐵
Implementation: limit search using conditions 𝑋 ⊆ 𝐴 ∧ 𝑋 ⊆ 𝐵
If 𝑟𝐹 𝐴, 𝑋 and 𝑋 ⊆ 𝐴:• For each 𝐵 ⊇ 𝑋:
• Check if 𝑋 = 𝐴 ∩ 𝐵
We emphasize this detail because formal semantic researchers are often not aware of these difficulties.
Encoding GQs as RelationsLimitations
𝐹 𝐴 𝐵 ≡ r𝐹 𝐴, 𝐵
Limitation:
Relations in DCS trees are always explained as having the widest scope, hence cannot deal with multiple relations in a sentence.
Encoding GQs as RelationsLimitations
Example:𝑃: At most 10 commissioners spend a lot of time at home.
We want to state𝑟𝐴𝑡𝑀𝑜𝑠𝑡 10 𝐜𝐨𝐦𝐢𝐬𝐬𝐢𝐨𝐧𝐞𝐫𝐬, 𝐷
where 𝐷 = “people who spend a lot of time at home”
But this is impossible if “a lot of” is also encoded as a relation
Encoding GQs as RelationsLimitations
Example:𝑟𝐴𝑡𝑀𝑜𝑠𝑡 10 𝐜𝐨𝐦𝐢𝐬𝐬𝐢𝐨𝐧𝐞𝐫𝐬, 𝐷
𝐷 = "people who spend a lot of time at home"
Workaround:Since “a lot of” is upward-entailing in predicate argument, we can encode it using selection 𝑠𝐴𝐿𝑜𝑡𝑂𝑓, while still encode “at
most 10” using 𝑟𝐴𝑡𝑀𝑜𝑠𝑡 10
Encoding GQs as RelationsLimitations
Example:𝑟𝐴𝑡𝑀𝑜𝑠𝑡 10 𝐜𝐨𝐦𝐢𝐬𝐬𝐢𝐨𝐧𝐞𝐫𝐬, 𝐷
𝐷 = 𝑞⊆𝑂𝐵𝐽
𝐷′, 𝑠𝐴𝐿𝑜𝑡𝑂𝑓 𝐭𝐢𝐦𝐞
where
𝐷′ = 𝐬𝐩𝐞𝐧𝐝 ∩ 𝑊𝑆𝐵𝐽 ×𝑊𝑂𝐵𝐽 × 𝐡𝐨𝐦𝐞𝑀𝑂𝐷
(“spend at home”)
EvaluationSet-up
The FraCaS Corpus:• Built in mid-1990s• A set of hand-crafted entailment problems covering
wide range of semantic phenomena
Section 1 - Generalized Quantifiers:• 74 problems:
• 44 have single premise sentence• 30 have multiple premise sentence
EvaluationSet-up
Settings:• Baseline
• Simply drop GQs• Same tree structure as follows
• Selection• Relation• Selection+Relation
EvaluationSet-up
Settings:• Baseline• Selection
• Implement all GQs as selections, even for those that are downward-entailing in predicate argument
• Relation• Selection+Relation
EvaluationSet-up
Settings:• Baseline• Selection• Relation
• Implement all GQs as relations• Selection+Relation
EvaluationSet-up
Settings:• Baseline• Selection• Relation• Selection+Relation
• Use relations to encode GQs that are downward-entailing in predicate argument
• Encode the rest with selections
Evaluation
SystemAccuracy
Single Multi Overall
NatLogMacCartney07 84.1%
N/AMacCartney08 97.7%
CCG-DistParser Syntax 70.5% 50.0% 62.2%
Gold Syntax 88.6% 80.0% 85.1%
TIFMO
Baseline 79.5% 86.7% 82.4%
Selection 90.9% 93.3% 91.9%
Relation 88.6% 93.3% 90.5%
Selection+Relation 93.2% 96.7% 94.6%
Accuracies of previous systems on Section 1 of FraCaS corpus