Computational LinguisticsComputational LinguisticsComputational LinguisticsComputational Linguistics

Lecture 1: IntroductionLecture 1: IntroductionLecture 1: IntroductionLecture 1: Introduction

What is Linguistics?Prescriptive Grammar:Descriptive Grammar:Some Linguistic MethodsGrammatical TheoryLinguistics Beyond Grammarlinguistics What is computational linguistics?Dr. Saeed Dr. Saeed

RahatiRahatiDr. Saeed Dr. Saeed RahatiRahati

What is Linguistics?What is Linguistics?

Linguistics is the study of human language, broadly construed.

Linguistics is a scientific discipline with established theories, analytic methods, and real-world applications.

Linguists often study individual languages, but...

When linguists study individual languages, they have larger issues in mind.

3

Linguistics is descriptive, not prescriptive

… What is Linguistics? … What is Linguistics?

4

Rules against certain usages.Few if any rules for what is

allowed.Condemns forms generally in

use.Explicitly normative enterprise.

Prescriptive Grammar:Prescriptive Grammar:

Rules characterizing what people do say.

Tries to do so in a way that reflects internalized generalizations that people have made.

Linguists are fundamentally concerned with linguistic knowledge.

Descriptive Grammar:Descriptive Grammar:

6

• parkway vs. driveway• maternity dress vs. paternity

suit• bathing trunks (pl) vs. bikini

(sing)• you are vs. *you is• Aren’t I clever? vs. *I aren’t

clever.

Anyway, language isn’t logical:Anyway, language isn’t logical:

7

◮ Fieldwork◮ Formal analysis of patterns in data sets◮ Psycholinguistic experiments◮ Computational modeling◮ Corpus analysis

Some Linguistic MethodsSome Linguistic Methods

8

◮ Phonetics: The study of speech sounds◮ Phonology: The study of sound systems◮ Morphology: The study of word structure◮ Syntax: The study of sentence structure◮ Semantics: The study of linguistic meaning◮ Pragmatics: The study of language use

Grammatical TheoryGrammatical Theory

9

Phonetics: The Study of Speech SoundsPhonetics: The Study of Speech Sounds

10

Phonology: The Study of Sound SystemsPhonology: The Study of Sound Systems

11

◮ missile: ‘ICBM’◮ anti-tank-missile: ‘missile targetting tanks’◮ anti-aircraft-missile: ‘missile targetting aircraft’◮ anti-missile-missile: ‘missile targetting ICBMs’

Morphology: The Study Of Word StructureMorphology: The Study Of Word Structure

12

◮ Rule: Anti-X-missile is a missile targetting Xs.◮ What kind of missile targets anti-missile-missiles?◮ anti-anti-missile-missile-missile◮ anti-anti-anti-missile-missile-missile-missile:‘missile targetting anti-anti-missile-missile-missiles’

Morphological RulesMorphological Rules

13

◮ I saw the woman with the telescope.I [saw [the woman] [with the telescope]].I [saw [[the woman] [with the telescope]]].

◮ Put the block in the box on the table in the bedroom.◮ Put the block in the box on the table in the bedroom near the kitchen.

Syntax: The Study of Sentence StructureSyntax: The Study of Sentence Structure

14

◮ Structural Ambiguity produces semantic ambiguity.◮ Both in morphology and syntax.◮ Lexical Ambiguity: We screened the candidates.◮ Both Together: I saw her duck.

Semantics: The Study of Linguistic MeaningSemantics: The Study of Linguistic Meaning

15

Q: Is Palin a Republican?A: Is the Pope Catholic?◮ Why don’t you move up to the City?◮ Why should I stand here and listen to this?◮ Do you think I’m saying this just to hear the sound of my own voice?

Pragmatics: The Study of Language UsePragmatics: The Study of Language Use

16

◮ Historical Linguistics: How languages change over time.◮ Sociolinguistics: How languages vary socially. How language is used as a social resource.◮ Psycholinguistics: What goes on in people’s heads as they use language.◮ Language Acquisition: How people learn language. (first language acquisition; second language acquisition)◮ Computational Linguistics: Making computers process (generate/‘understand’/translate...) human languages.

Linguistics Beyond GrammarLinguistics Beyond Grammar

17

more

rig

oro

us

less

rigoro

us

more

fl

akey

literarycriticism

physicsbiology

chemistrypsychology

computational linguistics

neuropsychologylinguistics?

Computational LinguisticsComputational Linguistics

18

What defines the rigor of a field? Whether results are reproducible Whether theories are

testable/falsifiable Whether there are a common set of

methods for similar problems Whether approaches to problems

can yield interesting new questions/answers

Computational Linguistics

19Linguistics

Computational Linguistics

20

more

rig

oro

us

less

rigoro

us

literarycriticism

engineering sociologylinguistics

Linguistics

more

rig

oro

us le

ss

rigoro

us

oth

er a

reas o

f so

ciolin

gu

istics(e

.g. D

eb

ora

h

Tan

nen

)“th

eore

tical”

ling

uistics

(e.g

. m

inim

alist

synta

x)

som

e a

reas o

f so

ciolin

gu

istics(e

.g. B

ill Lab

ov)

psy

cholin

gu

istics

exp

erim

en

tal p

hon

etics

histo

rical lin

gu

istics

“theore

tical”

ling

uistics

(e.g

. lexica

l-fun

ction

al

gra

mm

ar)

The true situation with linguistics

22

What is computational linguistics? Text normalization/segmentation Morphological analysis Automatic word pronunciation prediction Transliteration Word-class prediction: e.g. part of speech tagging Parsing Semantic role labeling Machine translation Dialog systems Topic detection Summarization Text retrieval Bioinformatics Language modeling for automatic speech recognition Computer-aided language learning (CALL)

23Computational linguistics

Often thought of as natural language engineering

But there is also a serious scientific component to it.

24

Goals of Computational Linguistics/ Natural Language Processing To get computers to deal with

language the way humans do: They should be able to understand

language and respond appropriately in language

They should be able to learn human language the way children do

They should be able to perform linguistic tasks that skilled humans can do, such as translation

Yeah, right

25Some interesting themes…

Finite-state methods: Many application areas Raises many interesting questions about how

“regular” language is Grammar induction:

Linguists have done a poor job at their stated goal of explaining how humans learn grammar

Computational models of language change: Historical evidence for language change is only

partial. There are many changes in language for which we have no direct evidence.

26Why CL may seem ad hoc

Wide variety of areas (as in linguistics)

If it’s natural language engineering the goal is often just to build something that works

Techniques tend to change in somewhat faddish ways… For example: machine learning

approaches fall in and out of favor