Upload
quito
View
49
Download
3
Embed Size (px)
DESCRIPTION
LINGUISTICA GENERALE E COMPUTAZIONALE, PARTE 2. Lezione 1: Cos’e ’ la Linguistica Computazionale , Introduzione al corso. LINGUISTICA COMPUTAZIONALE. - PowerPoint PPT Presentation
Citation preview
LINGUISTICA GENERALE E COMPUTAZIONALE,
PARTE 2
Lezione 1: Cos’e’ la Linguistica Computazionale,
Introduzione al corso 1
2
LINGUISTICA COMPUTAZIONALE
• Questa seconda parte di LG&C è un’introduzione alla LINGUISTICA COMPUTAZIONALE (COMPUTATIONAL LINGUISTICS): lo studio di modelli computazionali e statistici dell’INTERPRETAZIONE del linguaggio– Normalmente distinta da CORPUS LINGUISTICS
(uso di modelli computazionali e statistici x analizzare CORPORA)
QUESTA LEZIONE
• Riassunto dei concetti rilevanti di linguistica generale
• Interpretazione: quali sono i problemi?• Applicazioni di linguistica computazionale• Piano del corso
Livelli di analisi linguistica – un rapido riassunto
5
LIVELLI DI ANALISI LINGUISTICA
• Fonetica e fonologia– “cat” = /k/ + /æ/ + /t/
• Parole– Parti del discorso– Morfologia
• Sintassi• Semantica • Discorso
PARTI DEL DISCORSO
• NOMI (tavolo, Simona)• VERBI (camminare, mangiare, colpire)• AGGETTIVI (rosso, rapido)• AVVERBI (probabilmente, subito)• PRONOMI (io, lui, ci)• ARTICOLI (il, la, un)• PREPOSIZIONI (di, a, con)• CONGIUNZIONI (e, ma, o)• [Italiano]: INTERIEZIONI (ahi! )
MORFOLOGIA
• Le parole non sono unita’ ‘atomiche’: (in Italiano almeno) si possono quasi sempre scomporre in unita’ piu’ piccole: i MORFEMI
• Un MORFEMA e’ “la minima unita’ linguistica dotata di un significato proprio”
DUE ESEMPI
REPURIFICARE
`ripetizione’ `privo di contaminanti’
RE- + PUR- + -IFICARE
`rendere’
STRUTTURA DELLE PAROLE
• INGLESE: RADICE + AFFISSI– RADICE (boy)– AFFISSI (-s in boy+s)
• ITALIANO: TEMA + AFFISSI– RADICE (ragazz-)– TEMA (radice + vocale tematica – e.g., ragazzo)– AFFISSI (-i in ragazz+i)
13
SINTASSI• Words are organized in PHRASES
– I put THE BAGELS in the freezer– I put THE BAGELS THAT WE HAD NOT EATEN in the freezer
• Phrases are classified according to their main CONSTITUENT, or HEAD:– Noun phrases:
• the bagels, the homeless old man that I tried to help yesterday• Mary, she, one of them
– Verb phrases:• Mary went to the store and bought a bagel
– Adjective Phrases:• John is tall / very tall / quite certain to succeed
– Sentences
14
Marking Phrase Constituents
• BRACKETING:– [S [NP The children] [VP ate [NP the cake]]]
• TREES:
SNP
VP
ATNNS
VBD NP
AT NNthe
children ate
thecake
Sintassi: obiettivo
Sintassi
“(io) Nel mezzo del cammin di nostra vita mi ritrovai per una selva oscura ” NP PP VP PP
“[Oscura per mezzo] [nel selva] [del nostra] [mi] [ritrovai] [di cammin vita una]” ?? PP ?? NP VP ??
Una frase italiana con la struttura (NP PP VP PP) è corretta
Una frase italiana con la struttura (?? PP ?? NP VP ??) è scorretta
Riconoscere i costituenti Riconoscere una struttura corretta
[ ][ ][[ ][ ]]
SINTASSI
SEMANTICA
• Due tipi di conoscenza semantica sulle parole:– Conoscenza ‘denotazionale’– Conoscenza ‘composizionale’
• Quattro tipi di teorie:– Referenziale– Cognitivo / mentalista
• Teoria dei prototipi– Strutturale / relazionale
Conoscenza denotazionale e conoscenza composizionale
• Conoscenza DENOTAZIONALE: conoscenza sulla ‘parola in se’:– Il CAVALLO e’ un ANIMALE dalla lunga criniera …– (Il tipo di conoscenza tipicamente trovata nelle
definizioni)• Conoscenza COMPOSIZIONALE: conoscenza
sul come la parola si combina con altre parole
CONOSCENZA COMPOSIZIONALE
• Dal punto di vista composizionale si possono fare almeno due distinzioni :– Tra PREDICATI ed ARGOMENTI– Tra parole FUNZIONALI e parole ‘CONTENUTO’
PREDICATI ED ARGOMENTI
Maria ha noleggiato una macchina
PREDICATO
ARGOMENTI
NLE 21
Discourse
• Anaphora– John arrived late. He always does that. – My car didn’t start this morning. There was some
problem with the engine fan. • Discourse relations:
– My car didn’t start this morning BECAUSE there was some problem with the engine fan.
Dave Bowman: “Open the pod bay doors, HAL”
HAL 9000: “I’m sorry Dave. I’m afraid I can’t do that.”
ANLE 232004/05
LA LINGUISTICA COMPUTAZIONALE NEL 2014: DOVE DOVREMMO ESSERE ..
Amer. Good afternoon, Hal. How's everything going?Hal. Good afternoon, Mr Amer. Everything is going extremely well.Amer. Hal, you have an enormous responsibility on this mission, in many ways perhaps the greatest responsibility of any single mission element. You are the brain and central nervous system of the ship, and your responsibilities include watching over the men in hibernation. Does this ever cause you any - lack of confidence?Hal. Let me put it this way, Mr Amer. The 9000 series is the most reliable computer ever made. No 9000 computer has ever made a mistake or distorted information. We are all, by any practical definition of the words, foolproof and incapable of error.Amer. Hal, despite your enormous intellect, are you ever frustrated by your dependence on people to carry out actions?Hal. Not in the slightest bit. I enjoy working with people. I have a stimulating relationship with Dr Poole and Dr Bowman. My mission responsibilities range over the entire operation of the ship, so I am constantly occupied. I am putting myself to the fullest possible use, which is all, I think, that any conscious entity can ever hope to do.
ELN 24
… E DOVE SIAMO
• A Febbraio del 2011 il sistema WATSON sviluppato da IBM ha vinto a Jeopardy! Battendo tre dei piu’ noti campioni del passato– http://www.youtube.com/watch?v=otBeCmpEKTs
2011/12
Modelli di interpretazione nella linguistica computazionale
NLE 26
INTERPRETAZIONE: IL MODELLO A ‘PIPELINE’
PREPROCESSINGLEXICAL
PROCESSINGSYNTACTIC
PROCESSING
SEMANTIC PROCESSING
DISCOURSE PROCESSING
NLE 27
INTERPRETAZIONE: IL MODELLO A PIPELINE
PREPROCESSINGLEXICAL
PROCESSINGSYNTACTIC
PROCESSING
SEMANTIC PROCESSINGDISCOURSE
When did Watson won Jeopardy?
TOKENIZATION
POS TAGGING / WORDSENSE IDENTIFY
PHRASES
PREDICATE/ARGUMENT
ANAPHORA
LA STRUTTURA DI WATSON
TOKENIZZAZIONE
C’ERA UNA VOLTA UN PEZZO DI LEGNO.
C’ERA | UNA | VOLTA | UN | PEZZO | DI | LEGNO. |
C’ | ERA | UNA | VOLTA | UN | PEZZO | DI | LEGNO | . |
PARTI DEL DISCORSO
Television/NN has/HVZ yet/RB to/TO work/VB out/RP a/AT living/RBG arrangement/NN with/IN jazz/NN ,/, which/VDT comes/VBZ to/IN the/AT medium/NN more/QL as/CS an/AT uneasy/JJ guest/NN than/CS as/CS a/AT relaxed/VBN member/NN of/IN the/AT family/NN ./.
LING 2000 - 2006 NLP 36
ANALISI SINTATTICA CON CONTEXT-FREE GRAMMARS
V PP
VP
S
NP
the
the mat
satcat
onNPPrep
The cat sat on the mat
DetN
Det N
LING 2000 - 2006 NLP 37
Processing Steps, IV: Semantic Processing
• John went to the book store. John store1, go(John, store1)
• John bought a book. buy(John,book1)
• John gave the book to Mary. give(John,book1,Mary)
• Mary put the book on the table. put(Mary,book1,table1)
DOVE STA IL PROBLEMA?
• Rumore (typos, linguaggio sgrammaticato, etc)
• Ambiguità
• Il ruolo del senso comune
BAD ENGLISH (E ITALIANO) ON THE WEB
CHINGLISH: To take notice of safe: The slippery are very crafty (“Take care, slippery”)Note that the level of gap(“Mind the gap”)
LANGUAGE CHANGE:I brought two apple'sBlack is different to white
SPAM:Buongiornosono sempre in attesa delle vostre informazioni affinché possa rapidamente le trasmetta al mio avvocato perché possa rapidamente fare l’analisi della vostra cartella più rapidamente che il possibile.Grazie rapidamente di me gli inviati.
AMBIGUITA’ NELLA CLASSIFICAZIONE GRAMMATICALE
• Molte forme di parola possono essere associate con parti del discorso diverse:– STATO sia sostantivo (LO STATO ITALIANO) che
verbo (NON SONO STATO IO)
AMBIGUITA’ DI PARTE DEL DISCORSO: LEGGE1
1 Norma, espressa dagli organi legislativi dello Stato, che stabilisce diritti e doveri dei cittadini Legge delega, che viene emessa dal potere esecutivo su delega del potere legislativo entro un ambito ben precisato Legge ponte, emessa in attesa di un'altra più organica A norma, a termini di legge, secondo ciò che la legge prescrive.2 (est.) Complesso delle norme costituenti l'ordinamento giuridico di uno Stato: la legge è uguale per tutti Essere fuori della legge, non essere garantito dalla legge o non sentirsi a essa soggetto Dettar legge, imporre a tutti la propria volontà.3 Scienza giuridica: laurea in legge; dottore in legge; facoltà di legge Uomo di legge, specialista nella scienza giuridica.4 Autorità giudiziaria: ricorrere alla legge In nome della legge, formula con cui i rappresentanti dell'autorità giudiziaria intimano a qc. di obbedire a un comando della stessa: in nome della legge, aprite!5 (est.) Ogni norma che regola la condotta individuale o sociale degli uomini: le leggi della società.6 (est.) Regola fondamentale di una tecnica, di un'arte e sim.: le leggi della pittura.7 Relazione determinata e costante fra le quantità variabili che entrano in un fenomeno: le leggi della matematica, della fisica.
LEGGE2
leggerev. tr. (pres. io lèggo, tu lèggi; pass. rem. io lèssi, tu leggésti; part. pass. lètto)1 Riconoscere dai segni della scrittura le parole e comprenderne il significato: imparare, insegnare a leggere; leggere a voce alta (ass.) Fare lettura, dedicarsi alla lettura: trascorro gran parte della giornata leggendo.2 Interpretare certi segni convenzionali o naturali: i ciechi leggono con le dita; leggere un diagramma (fig.) Leggere la mano, ricavare dati sul carattere e sul destino di qc. basandosi sulle linee della mano.3 (lett.) Interpretare uno scritto, un passo: i critici dell'Ottocento leggevano erroneamente questa strofa (est.) Interpretare, valutare scritti, eventi e sim. secondo particolari criteri: leggere un film in chiave ironica.4 (fig.) Intuire i pensieri e le intenzioni di qc.: gli si legge il terrore sul volto.
STATISTICHE SULL’AMBIGUITA’ NEL B.C.
Unambiguous (1tag) 35,340Ambiguous (2-7 tags) 4,100
2 tags 3,7603 tags 2644 tags 615 tags 126 tags 27 tags 1 (“still”)
Part of Speech Tagging and Word Sense Disambiguation
• [verb Duck ] ! [noun Duck] is delicious for dinner
• I went to the bank to deposit my check. I went to the bank to look out at the river. I went to the bank of windows and chose the
one dealing with last names beginning with “d”.
S S
NP VP NP VP
I V NP VP I V NP
made her V made det N
duck her duck
Syntactic Disambiguation
• Structural ambiguity:
LING 2000 - 2006 NLP 46
SemanticsSame event - different sentences
John broke the window with a hammer.
John broke the window with the crack.
The hammer broke the window.
The window broke.
NLE 47
Scope ambiguity
IL RUOLO DEL SENSO COMUNE
• Winograd (1974):– The city council refused the women a
permit because they feared violence.– The city council refused the women a
permit because they advocated violence
ANLE 492004/05
NLP APPLICATIONS• Mature, everyday technology that hardly anybody notices anymore
– E.g., tokenization, normalization, regular expression search• Solid technology that is intensively used but can (and is) still be improved
– E.g., lemmatization; spelling correctors; IR / Web search; Speech synthesis• Used in real applications, but substantial improvements still desired
– E.g., POS tagging; term extraction; summarization; speech recognition; text classification (e.g., for spam detection); sentiment analysis
– Spoken dialogue systems for simple information seeking (railways, phone)• ‘Almost there’ technology – exists in prototype form
– E.g., information extraction, generation systems, simple speech translation systems• Pie in the sky
– Full machine translation, more advanced dialogue
ANLE 502004/05
Part I: Mature Technologies
• Research in NLE has been going on for many years and in many forms – e.g., as part of compiler technology, information retrieval, etc.
• The results of this work are a number of well-established technologies that are hardly considered ‘research’ anymore
ANLE 512004/05
Basic Word Processing
TOKENISATION:
StringTokenizer st = new StringTokenizer("this is a test"); while (st.hasMoreTokens()) { System.out.println(st.nextToken()); }prints out: this
is a test
WORD COUNTING / FREQUENCIES
ANLE 522004/05
Regular Expressions for Search, Validation and Parsing
• Basics (e.g., search in Google)– cat OR dog – “Regular * in Java”
• More advanced (e.g., regular expressions in PERL, Java, etc.) (for advanced search, user input validation, etc.)
– /[Ww]ordnet/– /colou?r/– /Mas*imo Poesio/– [a-z|A-Z]*– [^A-Z]– /$[0-9]+\.[0-9][0-9]
• Note also: SUBSTITUTION– s/colour/color/
• ELIZA: – s/.* YOU ARE (depressed|sad) .*/I AM SORRY TO HEAR YOU ARE \1/
ANLE 532004/05
Part II: Solid Technology that could still use improvements
• Over the last ten to twenty years new applications have appeared which are by now fairly well established, but whose results are still not 100% accurate (nor is clear they ever will!)
ANLE 542004/05
Stemming, lemmatization and morphological analysis
• Stemming: – FOXES -> FOX
• Lemmatization:– 'screeching, screeches, screeched,' and 'screech' -> 'screech'
+ING – 'were' -> 'be‘ +PAST
• (Sometimes) used for: Information Retrieval– But: not in GOOGLE
• More general morphological analysis:– Wissenschaftlichemitarbeiter -> Wissenschaft + mitarbeiter Scientific collaborator (Researcher)– Uygarlastiramadiklarimizdanmiscasina ->– Uygar +las +tir +ama +dik Civilized +BEC +CAUSE +NEGABLE +PPART – +lar +imiz +dan +mis +siniz +casina +PL +P1PL +ABL +PAST +2PL +AsIf– `(behaving) as if you are among those whom we could not civilize’
ANLE 552004/05
Morphological Analysis: the Xerox tools
ANLE 562004/05
Word Prediction
• Systems that can complete the current word / sentence (e.g., to help people with disabilities)
• E.g., the Aurora System• Or textHelp!
ANLE 572004/05
Spelling correction
• Word:– Gettin -> getting– Alway -> always – But :
• olways -/-> always • Definittely -/-> definitely
• Some shells: > set correct = cmd
> lz /usr/bin CORRECT>ls /usr/bin (y|n|e|a)?
ANLE 582004/05
Part of Speech Tagging
• Assign a PART OF SPEECH to each word:– ‘dog’ -> NOUN– ‘eat’ -> VERB
• Book that flight VB DT NN• Applications: all over the place!
– IR– IE– Translation
TEXT CLASSIFICATION: SPAM DETECTION
From: "" <[email protected]>Subject: real estate is the only way... gem oalvgkay
Anyone can buy real estate with no money down
Stop paying rent TODAY !
There is no need to spend hundreds or even thousands for similar courses
I am 22 years old and I have already purchased 6 properties using the
methods outlined in this truly INCREDIBLE ebook.
Change your life NOW !
=================================================
Click Below to order:http://www.wholesaledaily.com/sales/nmd.htm================================================
=
Dear Hamming Seminar Members
The next Hamming Seminar will take place on Wednesday 25th May and the details are as follows - Who: Dave Robertson Title: Formal Reasoning Gets Social Abstract: For much of its history, formal knowledge representation has aimed to describe knowledge independently of the personal and social context in which it is used, with the advantage that we can automate reasoning with such knowledge using mechanisms that also are context independent. This sounds good until you try it on a large scale and find out how sensitive to context much of reasoning actually is. Humans, however, are great hoarders of information and sophisticated tools now make the acquisition of many forms of local knowledge easy. The question is: how to combine this beyond narrow individual use, given that knowledge (and reasoning) will inevitably be contextualised in ways that may be hidden from the people/systems that may interact to use it? This is the social side of knowledge representation and automated reasoning. I will discuss how the formal reasoning community has adapted to this new view of scale. When: 4pm, Wednesday 25 May 2011 Where: Room G07, Informatics Forum There will be wine and nibbles afterwards in the atrium café area.
SENTIMENT ANALYSIS
Id: Abc123 on 5-1-2008 “I bought an iPhone a few days ago. It is such a nice phone. The touch screen is really cool. The voice quality is clear too.
It is much better than my old Blackberry, which was a terrible phone and so difficult to type with its tiny keys. However, my mother was mad with me as I did not tell her before I bought the phone. She also thought the phone was too expensive, …”
SENTIMENT ANALYSIS
Id: Abc123 on 5-1-2008 “I bought an iPhone a few days ago. It is such a nice phone. The touch screen is really cool. The voice quality is clear too.
It is much better than my old Blackberry, which was a terrible phone and so difficult to type with its tiny keys. However, my mother was mad with me as I did not tell her before I bought the phone. She also thought the phone was too expensive, …”
SENTIMENT ANALYSIS
Id: Abc123 on 5-1-2008 “I bought an iPhone a few days ago. It is such a nice phone. The touch screen is really cool. The voice quality is clear too.
It is much better than my old Blackberry, which was a terrible phone and so difficult to type with its tiny keys. However, my mother was mad with me as I did not tell her before I bought the phone. She also thought the phone was too expensive, …”
Stylometry: Who wrote this?
“On the far side of the river valley the road passed through a stark black burn. Charred and limbless trunks of trees stretching away on every side. Ash moving over the road and the sagging hands of blind wire strung from the blackened lightpoles whining thinly in the wind.”
Stylometry: Who wrote this?
“On the far side of the river valley the road passed through a stark black burn. Charred and limbless trunks of trees stretching away on every side. Ash moving over the road and the sagging hands of blind wire strung from the blackened lightpoles whining thinly in the wind.”
Cormac McCarthy
ANLE 652004/05
Speech Synthesis
• Speech Synthesis (the automatic production of speech from text or other computer-encoded source) is much easier than speech RECOGNITION and is currently a very hot area in industry
• For a British example, check out Rhetorical Systems
• US: AT&T
Parte 3: tecnologie più avanzate
• Da tecnologie usate per anni ma ancora problematiche, a tecnologie solo disponibili in forma prototipale
• (Non ci occuperemo di queste tecnologie nel corso)
ANLE 682004/05
Speech Recognition
• Speech Recognition fairly solid (and works very well for digits)– E.g., IBM’s Via Voice:
• http://www4.ibm.com/software/speech/enterprise/dcenter/demo_0.html
Summarization
• Summarization is the production of a summary either from a single source (single-document summarization) or from a collection of articles (multi-document summarization)
• An example is the Columbia Newblaster
Machine Translation
• Machine translation is one of the earliest attempts at language technology (from the ’40s)
• Still mostly useful to get a quick idea of the content of a text, but can sometimes works reasonably well
• An example: Newstran.com
Machine Translation
ANLE 722004/05
Machine Translation
INFORMATION EXTRACTION: REFERENCES TO (NAMED) ENTITIES
LOC
SITE
CULTURE
EXAMPLE OF IE APPLICATION: FINDING JOBS FROM THE WEB
foodscience.com-Job2
JobTitle: Ice Cream Guru
Employer: foodscience.com
JobCategory: Travel/Hospitality
JobFunction: Food Services
JobLocation: Upper Midwest
Contact Phone: 800-488-2611
DateExtracted: January 8, 2001
Source: www.foodscience.com/jobs_midwest.html
OtherCompanyJobs: foodscience.com-Job1
CONTENUTO DEL CORSO
• Il pacchetto NLTK, implementato in Python, permette di sperimentare tecniche CL anche a chi ha poca esperienza di programmazione
• Durante il corso– introdurremo Python ricapitolando gli aspetti della
linguistica computazionale gia’ introdotti in IDUL– Useremo NLTK per sperimentare
• POS tagging• Parsing• classificazione
• Seguiremo abbastanza fedelmente il testo di Bird Klein & Loper
Il Testo
http://www.nltk.org/book
ALTRE INFORMAZIONI
• Sito:– http://clic.cimec.unitn.it/massimo/Teach/ELN/
• Esame:– Sviluppare un progettino in Python, da presentare
all’orale• Ricevimento:
– Su appuntamento
SCARICARE Python e NLTK
• Primo compito: Seguire le istruzioni a
http://www.nltk.org/download