Statistical Methods and Linguistics - Steven Abney 1998. 09. 24. Thur. POSTECH Computer Science NLP Lab 9425021 Shim Jun-Hyuk

Statistical Methods and Statistical Methods and Linguistics Linguistics - Steven Abney- Steven Abney

1998. 09. 24. Thur. 1998. 09. 24. Thur.

POSTECH Computer SciencePOSTECH Computer Science

NLP Lab 9425021NLP Lab 9425021Shim Jun-HyukShim Jun-Hyuk

2

CS730B - Statistical NLP

Contents Contents

IntroductionIntroduction

Linguistics Review under Statistical methodsLinguistics Review under Statistical methods Language Acquisition Language Change Language Variation

Language Structure and PerformanceLanguage Structure and Performance Language Property Grammaticality and Ambiguity v. Performance Non-Linguistic Factors for Performance Grammaticality and Acceptability Grammar and Computation The Frictionless Plane, Autonomy and Isolation Holy Grail

3


ContentsContents

How Statistics HelpsHow Statistics Helps Disambiguation Degrees of Grammaticality Naturalness Structure Preferences Error Tolerance Learning on the Fly Lexical Acquisition

ObjectionsObjections Are Stochastic Methods only for engineers? Did not Chomsky debunk all this ages ago?

ConclusionConclusion

4


IntroductionIntroduction

LinguisticsLinguistics Computation Linguistics

Performance Practical Application little concerned with human language processing Rationale by the Statistical Method

Theoretical Linguistics Competence Theoretical Research with grammars and structures concerned with human language processing

ObjectivesObjectives Theoretical Background of Statistical analyses Review in the view of Linguistics Importance of Weighted Grammar

5


1.1. Linguistics Review under Statistical Models Linguistics Review under Statistical Models (1)(1)

ObjectiveObjective Linguistics Issues in terms of population of grammar General population of grammar can be usefully examined by the Statistical

Models

Language Acquisition (LA)Language Acquisition (LA) Probabilistic(stochastic) or weighted grammar in Children’s LA Process Co-existence and decay in grammars Algebraic(Non-stochastic) grammar as supplementation

6


1.1. Linguistics Review under Statistical Models Linguistics Review under Statistical Models (2)(2)

Language Change Language Change Change in Probability of Language Construction

EX) Rule, Parameter setting

Not “Abrupt”, but “Gradual” Statistical Co-existence and Decay

“Adult monolingual speaker” - finally the grammar is stochastic in community

Language VarianceLanguage Variance Dialectology

Arbitrary continuum of language made by geographic distance Contact Frequency and intelligibility

Typology EX) Language Feature, Conditional Probability distributions

Statistical Modeling using the stochastic grammar

7


2.2. Language Structure and Performance Language Structure and Performance (1)(1)

LanguageLanguage Algebraic Properties

Idealization - Adult monolingual Speaker theoretical syntax - Linguistics Data Structure judgments for competence

Statistical Properties Stochastic Model - Performance data adjustments on structure-judgement data for “performance effects” grammaticality and ambiguity judgments about the sentences as opposed

to structure

8



Grammaticality and Ambiguity v. PerformanceGrammaticality and Ambiguity v. Performance Example

The a are of I The cows are grazing in the meadow John saw Mary Ambiguity Problem under Grammatical structures

Genuine ambiguities and Spurious ambiguities Problem Is not ungrammatical but undesired analyses case1 - elided sentence case2 - rare Usage The Problem is how to identify the correct structure form the

possible. Can be solved by the use of weighted grammars in computational

linguistics

9



Non-Linguistic Factors for PerformanceNon-Linguistic Factors for Performance Perception is the problem of Performance and It needs Non-Linguistic

Factors with Grammaticality

Grammaticality and Acceptability perceptions of grammaticality and Ambiguity - Performance data What is “Performance data” - find some choice of words and

context to get a clear positive judgment (Acceptability) Grammar and Computation

The Problem how can we compute the linguistic data simply and absolutely

Competence v. Computation Autonomy of syntax - not same as isolation and not be reduced to semantics

Holy Grail The larger picture and ultimate goal of Generative linguistics is to

make sense of language production, comprehension, acquisition, variation, and change

10


3. How Statistics Helps 3. How Statistics Helps (1)(1)

Disambiguation (Disambiguation ( 모호성 해소모호성 해소 )) Describing an algorithm to compute the correct parse among the possible correct parse - the parse that human perceive various statistical methods exist 예 ) “John walks” - Context-free grammar with weights of rules

Degrees of GrammaticalityDegrees of Grammaticality Gradations of acceptability Degrees of error in speech production Measure of goodness is a global measure that combine the degrees of

grammaticality with naturalness and structural preference By parameter Estimation, we can get the measure of “ degrees of

grammaticality”

11



NaturalnessNaturalness plausibility - in the sense of selectional preferences collocational knowledge - “how do you say it” statistical method are applied to collocations and selectional restrictions

Structural PreferenceStructural Preference One of the parsing strategies longest-match preference make an important role in the dispreference for the structure

Error toleranceError tolerance Detecting the error in sentences and select the best analysis Primary motivations for Shannon’s noisy channel model

12



Learning on the FlyLearning on the Fly much like the error correction to admit a space of learning operations

assigning a new part of speech to a word adding a new subcategorization frame to verb, etc

Lexical AcquisitionLexical Acquisition the absolute richness of natural language grammars and lexica primary area of application for distributional and statistical approaches to a

cquisition Example of distributional Approaches

acquisition of Part-of-Speech Collocation selectional restriction and ETC.

13


4. 4. Objections to Statistical MethodsObjections to Statistical Methods

Are Stochastic Models only for Engineers?Are Stochastic Models only for Engineers? Are the stochastic models practically always a stopgap approximation? With a complex deterministic system and the initial conditions we can com

pute the state at all time In fact, more insight and successful than identifying every deterministic fac

tors

What Chomsky really proves?What Chomsky really proves? syntactic Structures (1957)

Chomsky : grammatical(s) Pn(s) > E• no choice for “n” and “E”

• Pn(s) : best n-th order approximation to English

Shannon’s MM : grammatical(s) lim(noo) Pn(s) > E• n increase, then erroneously assigned non-zero probability decease

Handbook of Mathematical Psychology (1963)

14


5. Conclusion5. Conclusion

Statistical method Statistical method weighted grammars, distributional induction methods relevant to Linguistics

Performance v. CompetencePerformance v. Competence Performance is not a goal but a useful tool of Computational Linguistics Competence is needed to understand the algebraic properties of language Algebraic methods are inadequate for understanding the human language The Age of Computational Linguistics using Statistical Technology

Documents

Statistical Methods and Linguistics - Steven Abney 1998. 09. 24. Thur. POSTECH Computer Science NLP Lab 9425021 Shim Jun-Hyuk