73
Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

  • View
    215

  • Download
    1

Embed Size (px)

Citation preview

Page 1: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

Enabling MT for Languages with Limited Resources

Alon LavieLanguage Technologies Institute

Carnegie Mellon University

Page 2: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 2

Progression of MT• Started with rule-based systems

– Very large expert human effort to construct language-specific resources (grammars, lexicons)

– High-quality MT extremely expensive only for handful of language pairs

• Along came EBMT and then SMT…– Replaced human effort with extremely large volumes of

parallel text data– Less expensive, but still only feasible for a small number of

language pairs– We “traded” human labor with data

• Where does this take us in 5-10 years?– Large parallel corpora for maybe 25-50 language pairs

• What about all the other languages?• Is all this data (with very shallow representation of

language structure) really necessary?• Can we build MT approaches that learn deeper levels of

language structure and how they map from one language to another?

Page 3: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 3

Why Machine Translation for Languages with Limited Resources?

• We are in the age of information explosion– The internet+web+Google anyone can get the

information they want anytime…• But what about the text in all those other

languages?– How do they read all this English stuff?– How do we read all the stuff that they put online?

• MT for these languages would Enable:– Better government access to native indigenous and

minority communities– Better minority and native community participation in

information-rich activities (health care, education, government) without giving up their languages.

– Civilian and military applications (disaster relief)– Language preservation

Page 4: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 4

The Roadmap to Learning-based MT

• Automatic acquisition of necessary language resources and knowledge using machine learning methodologies:– Learning morphology (analysis/generation)– Rapid acquisition of broad coverage word-to-word

and phrase-to-phrase translation lexicons– Learning of syntactic structural mappings

• Tree-to-tree structure transformations [Knight et al], [Eisner], [Melamed] require parse trees for both languages

• Learning syntactic transfer rules with resources (grammar, parses) for just one of the two languages

– Automatic rule refinement and/or post-editing• Effective integration of acquired knowledge

with statistical/distributional information

Page 5: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 5

CMU’s AVENUE Approach• Elicitation: use bilingual native informants to produce a

small high-quality word-aligned bilingual corpus of translated phrases and sentences

• Transfer-rule Learning: apply ML-based methods to automatically acquire syntactic transfer rules for translation between the two languages– Learn from major language to minor language– Translate from minor language to major language

• XFER + Decoder:– XFER engine produces a lattice of all possible transferred

structures at all levels– Decoder searches and selects the best scoring combination

• Rule Refinement: refine the acquired rules via a process of interaction with bilingual informants

• Morphology Learning• Word and Phrase bilingual lexicon acquisition

Page 6: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 6

AVENUE Architecture

Learning Module

Transfer Rules

{PP,4894};;Score:0.0470PP::PP [NP POSTP] -> [PREP NP]((X2::Y1)(X1::Y2))

Translation Lexicon

Run Time Transfer System

Lattice Decoder

English Language Model

Word-to-Word Translation Probabilities

Word-aligned elicited data

Page 7: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 7

Learning Transfer-Rules for Languages with Limited Resources

• Rationale:– Bilingual native informant(s) can translate and align a

small pre-designed elicitation corpus, using elicitation tool– Elicitation corpus designed to be typologically and

structurally comprehensive and compositional– Transfer-rule engine and new learning approach support

acquisition of generalized transfer-rules from the data

Page 8: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 8

Transfer Rule Formalism

Type informationPart-of-speech/constituent

informationAlignments

x-side constraints

y-side constraints

xy-constraints, e.g. ((Y1 AGR) = (X1 AGR))

;SL: the old man, TL: ha-ish ha-zaqen

NP::NP [DET ADJ N] -> [DET N DET ADJ]((X1::Y1)(X1::Y3)(X2::Y4)(X3::Y2)

((X1 AGR) = *3-SING)((X1 DEF = *DEF)((X3 AGR) = *3-SING)((X3 COUNT) = +)

((Y1 DEF) = *DEF)((Y3 DEF) = *DEF)((Y2 AGR) = *3-SING)((Y2 GENDER) = (Y4 GENDER)))

Page 9: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 9

Rule Learning - Overview

• Goal: Acquire Syntactic Transfer Rules• Use available knowledge from the source

side (grammatical structure)• Three steps:

1. Flat Seed Generation: first guesses at transfer rules; flat syntactic structure

2. Compositionality: use previously learned rules to add hierarchical structure

3. Seeded Version Space Learning: refine rules by learning appropriate feature constraints

Page 10: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 10

Flat Seed Rule Generation

Learning Example: NP

Eng: the big apple

Heb: ha-tapuax ha-gadol

Generated Seed Rule:

NP::NP [ART ADJ N] [ART N ART ADJ]

((X1::Y1)

(X1::Y3)

(X2::Y4)

(X3::Y2))

Page 11: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 11

CompositionalityInitial Flat Rules: S::S [ART ADJ N V ART N] [ART N ART ADJ V P ART N]

((X1::Y1) (X1::Y3) (X2::Y4) (X3::Y2) (X4::Y5) (X5::Y7) (X6::Y8))

NP::NP [ART ADJ N] [ART N ART ADJ]

((X1::Y1) (X1::Y3) (X2::Y4) (X3::Y2))

NP::NP [ART N] [ART N]

((X1::Y1) (X2::Y2))

Generated Compositional Rule:

S::S [NP V NP] [NP V P NP]

((X1::Y1) (X2::Y2) (X3::Y4))

Page 12: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 12

Seeded Version Space LearningInput: Rules and their Example Sets

S::S [NP V NP] [NP V P NP] {ex1,ex12,ex17,ex26}

((X1::Y1) (X2::Y2) (X3::Y4))

NP::NP [ART ADJ N] [ART N ART ADJ] {ex2,ex3,ex13}

((X1::Y1) (X1::Y3) (X2::Y4) (X3::Y2))

NP::NP [ART N] [ART N] {ex4,ex5,ex6,ex8,ex10,ex11}

((X1::Y1) (X2::Y2))

Output: Rules with Feature Constraints:

S::S [NP V NP] [NP V P NP]

((X1::Y1) (X2::Y2) (X3::Y4)

(X1 NUM = X2 NUM)

(Y1 NUM = Y2 NUM)

(X1 NUM = Y1 NUM))

Page 13: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 13

AVENUE Prototypes

• General XFER framework under development for past two years

• Prototype systems so far:– German-to-English, Spanish-to-English– Hindi-to-English, Hebrew-to-English

• In progress or planned:– Mapudungun-to-Spanish– Quechua-to-Spanish– Arabic-to-English– Native-Brazilian language to Brazilian Portuguese

Page 14: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 14

Missing Science

• Monolingual learning tasks:– Learning morphology: morphemes and their meaning– Learning syntactic and semantic structures:

grammar induction• Bilingual Learning Tasks:

– Automatic acquisition of word and phrase translation lexicons

– Learning structural mappings (syntactic and semantic)

• Models that effectively combine learned symbolic knowledge with statistical information: new “decoders”

Page 15: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 15

Page 16: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 16

English-Chinese Example

Page 17: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 17

English-Hindi Example

Page 18: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 18

Spanish-Mapudungun Example

Page 19: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 19

English-Arabic Example

Page 20: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 20

Transfer Rule Formalism (II)

Value constraints

Agreement constraints

;SL: the old man, TL: ha-ish ha-zaqen

NP::NP [DET ADJ N] -> [DET N DET ADJ]((X1::Y1)(X1::Y3)(X2::Y4)(X3::Y2)

((X1 AGR) = *3-SING)((X1 DEF = *DEF)((X3 AGR) = *3-SING)((X3 COUNT) = +)

((Y1 DEF) = *DEF)((Y3 DEF) = *DEF)((Y2 AGR) = *3-SING)((Y2 GENDER) = (Y4 GENDER)))

Page 21: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 21

AVENUE PartnersLanguage Country Institutions

Mapudungun (in place)

Chile Universidad de la Frontera, Institute for Indigenous Studies, Ministry of Education

Quechua(discussion)

Peru Ministry of Education

Aymara(discussion)

Bolivia, Peru Ministry of Education

Page 22: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 22

The Transfer EngineAnalysis

Source text is parsed into its grammatical structure. Determines transfer application ordering.

Example:

他 看 书。 (he read book)

S

NP VP

N V NP

他 看 书

TransferA target language tree is created by reordering, insertion, and deletion.

S

NP VP

N V NP

he read DET N

a book

Article “a” is inserted into object NP. Source words translated with transfer lexicon.

GenerationTarget language constraints are checked and final translation produced.

E.g. “reads” is chosen over “read” to agree with “he”.

Final translation:

“He reads a book”

Page 23: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 23

Seeded VSL: Some Open Issues

• Three types of constraints:– X-side constrain applicability of rule– Y-side assist in generation– X-Y transfer features from SL to TL

• Which of the three types improves translation performance?– Use rules without features to populate lattice, decoder will select

the best translation…– Learn only X-Y constraints, based on list of universal projecting

features• Other notions of version-spaces of feature constraints:

– Current feature learning is specific to rules that have identical transfer components

– Important issue during transfer is to disambiguate among rules that have same SL side but different TL side – can we learn effective constraints for this?

Page 24: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 24

Examples of Learned Rules (Hindi-to-English)

{NP,14244}

;;Score:0.0429

NP::NP [N] -> [DET N]

(

(X1::Y2)

)

{NP,14434}

;;Score:0.0040

NP::NP [ADJ CONJ ADJ N] ->

[ADJ CONJ ADJ N]

(

(X1::Y1) (X2::Y2)

(X3::Y3) (X4::Y4)

)

{PP,4894};;Score:0.0470PP::PP [NP POSTP] -> [PREP NP]((X2::Y1)(X1::Y2))

Page 25: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 25

XFER MT for Hebrew-to-English• Two month intensive effort to apply our XFER approach

to the development of a Hebrew-to-English MT system• Challenges:

– No large parallel corpus– Limited coverage translation lexicon– Rich Morphology: incomplete analyzer available

• Accomplished:– Collected available resources, establish methodology for

processing Hebrew input– Translated and aligned Elicitation Corpus– Learned XFER rules– Developed (small) manual XFER grammar as a point of

comparison– System debugging and development– Evaluated performance on unseen test data using

automatic evaluation metrics

Page 26: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 26

Transfer Engine

English Language Model

Transfer Rules{NP1,3}NP1::NP1 [NP1 "H" ADJ] -> [ADJ NP1]((X3::Y1) (X1::Y2) ((X1 def) = +) ((X1 status) =c absolute) ((X1 num) = (X3 num)) ((X1 gen) = (X3 gen)) (X0 = X1))

Translation Lexicon

N::N |: ["$WR"] -> ["BULL"]((X1::Y1) ((X0 NUM) = s) ((Y0 lex) = "BULL"))

N::N |: ["$WRH"] -> ["LINE"]((X1::Y1) ((X0 NUM) = s) ((Y0 lex) = "LINE"))

Hebrew Input

בשורה הבאה

Decoder

English Output

in the next line

Translation Output Lattice

(0 1 "IN" @PREP)(1 1 "THE" @DET)(2 2 "LINE" @N)(1 2 "THE LINE" @NP)(0 2 "IN LINE" @PP)(0 4 "IN THE NEXT LINE" @PP)

Preprocessing

Morphology

Page 27: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 27

Morphology Example

• Input word: B$WRH

0 1 2 3 4 |--------B$WRH--------| |-----B-----|$WR|--H--| |--B--|-H--|--$WRH---|

Page 28: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 28

Morphology ExampleY0: ((SPANSTART 0) Y1: ((SPANSTART 0) Y2: ((SPANSTART 1) (SPANEND 4) (SPANEND 2) (SPANEND 3) (LEX B$WRH) (LEX B) (LEX $WR) (POS N) (POS PREP)) (POS N) (GEN F) (GEN M) (NUM S) (NUM S) (STATUS ABSOLUTE)) (STATUS ABSOLUTE))

Y3: ((SPANSTART 3) Y4: ((SPANSTART 0) Y5: ((SPANSTART 1) (SPANEND 4) (SPANEND 1) (SPANEND 2) (LEX $LH) (LEX B) (LEX H) (POS POSS)) (POS PREP)) (POS DET))

Y6: ((SPANSTART 2) Y7: ((SPANSTART 0) (SPANEND 4) (SPANEND 4) (LEX $WRH) (LEX B$WRH) (POS N) (POS LEX)) (GEN F) (NUM S) (STATUS ABSOLUTE))

Page 29: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 29

Sample Output (dev-data)

maxwell anurpung comes from ghana for israel four years ago and since worked in cleaning in hotels in eilat

a few weeks ago announced if management club hotel that for him to leave israel according to the government instructions and immigration police

in a letter in broken english which spread among the foreign workers thanks to them hotel for their hard work and announced that will purchase for hm flight tickets for their countries from their money

Page 30: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 30

Evaluation Results

• Test set of 62 sentences from Haaretz newspaper, 2 reference translations

System BLEU NIST P R METEOR

No Gram 0.0616 3.4109 0.4090 0.4427 0.3298

Learned 0.0774 3.5451 0.4189 0.4488 0.3478

Manual 0.1026 3.7789 0.4334 0.4474 0.3617

Page 31: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 31

Future Directions• Continued work on automatic rule learning (especially

Seeded Version Space Learning)– Use Hebrew and Hindi systems as test platforms for

experimenting with advanced learning research• Rule Refinement via interaction with bilingual speakers• Developing a well-founded model for assigning scores

(probabilities) to transfer rules• Redesigning and improving decoder to better fit the

specific characteristics of the XFER model• Improved leveraging from manual grammar resources• MEMT with improved

– Combination of output from different translation engines with different confidence scores

– strong decoding capabilities

Page 32: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 32

Flat Seed Generation

Create a transfer rule that is specific to the sentence pair, but abstracted to the POS level. No syntactic structure.

Element Source

SL POS sequence f-structure

TL POS sequence TL dictionary, aligned SL words

Type information corpus, same on SL and TL

Alignments informant

x-side constraints f-structure

y-side constraints TL dictionary, aligned SL words (list of projecting features)

Page 33: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 33

Compositionality - Overview

• Traverse the c-structure of the English sentence, add compositional structure for translatable chunks

• Adjust constituent sequences, alignments

• Remove unnecessary constraints, i.e. those that are contained in the lower-level rule

Page 34: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 34

Seeded Version Space Learning: Overview

• Goal: add appropriate feature constraints to the acquired rules• Methodology:

– Preserve general structural transfer– Learn specific feature constraints from example set

• Seed rules are grouped into clusters of similar transfer structure (type, constituent sequences, alignments)

• Each cluster forms a version space: a partially ordered hypothesis space with a specific and a general boundary

• The seed rules in a group form the specific boundary of a version space

• The general boundary is the (implicit) transfer rule with the same type, constituent sequences, and alignments, but no feature constraints

Page 35: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 35

Seeded Version Space Learning: Generalization

• The partial order of the version space:Definition: A transfer rule tr1 is strictly more general than another transfer rule tr2 if all f-structures that are satisfied by tr2 are also satisfied by tr1.

• Generalize rules by merging them:– Deletion of constraint– Raising two value constraints to an agreement

constraint, e.g. ((x1 num) = *pl), ((x3 num) = *pl) ((x1 num) = (x3 num))

Page 36: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 36

Seeded Version Space Learning

NP v det n NP VP …1. Group seed rules into version spaces as above.2. Make use of partial order of rules in version space. Partial order is defined

via the f-structures satisfying the constraints.3. Generalize in the space by repeated merging of rules:

1. Deletion of constraint2. Moving value constraints to agreement constraints, e.g.

((x1 num) = *pl), ((x3 num) = *pl) ((x1 num) = (x3 num)

4. Check translation power of generalized rules against sentence pairs

Page 37: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 37

Seeded Version Space Learning:The Search

• The Seeded Version Space algorithm itself is the repeated generalization of rules by merging

• A merge is successful if the set of sentences that can correctly be translated with the merged rule is a superset of the union of sets that can be translated with the unmerged rules, i.e. check power of rule

• Merge until no more successful merges

Page 38: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 38

Conclusions• Transfer rules (both manual and learned) offer

significant contributions that can complement existing data-driven approaches– Also in medium and large data settings?

• Initial steps to development of a statistically grounded transfer-based MT system with:– Rules that are scored based on a well-founded

probability model – Strong and effective decoding that incorporates the

most advanced techniques used in SMT decoding

• Working from the “opposite” end of research on incorporating models of syntax into “standard” SMT systems [Knight et al]

• Our direction makes sense in the limited data scenario

Page 39: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 39

AVENUE Architecture

User

Learning Module

ElicitationProcess

TransferRule

Learning

TransferRules

Run-Time Module

SLInput

SL Parser

TransferEngine

TLGenerator

TLOutputDecoder

MorphologyPre-proc

Page 40: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 40

Learning Transfer-Rules for Languages with Limited Resources

• Rationale:– Large bilingual corpora not available– Bilingual native informant(s) can translate and align a

small pre-designed elicitation corpus, using elicitation tool– Elicitation corpus designed to be typologically

comprehensive and compositional– Transfer-rule engine and new learning approach support

acquisition of generalized transfer-rules from the data

Page 41: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 41

The Elicitation Corpus

• Translated, aligned by bilingual informant• Corpus consists of linguistically diverse

constructions• Based on elicitation and documentation work

of field linguists (e.g. Comrie 1977, Bouquiaux 1992)

• Organized compositionally: elicit simple structures first, then use them as building blocks

• Goal: minimize size, maximize linguistic coverage

Page 42: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 42

The Transfer EngineAnalysis

Source text is parsed into its grammatical structure. Determines transfer application ordering.

Example:

他 看 书。 (he read book)

S

NP VP

N V NP

他 看 书

TransferA target language tree is created by reordering, insertion, and deletion.

S

NP VP

N V NP

he read DET N

a book

Article “a” is inserted into object NP. Source words translated with transfer lexicon.

GenerationTarget language constraints are checked and final translation produced.

E.g. “reads” is chosen over “read” to agree with “he”.

Final translation:

“He reads a book”

Page 43: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 43

Transfer Rule Formalism

Type informationPart-of-speech/constituent

informationAlignments

x-side constraints

y-side constraints

xy-constraints, e.g. ((Y1 AGR) = (X1 AGR))

;SL: the man, TL: der Mann

NP::NP [DET N] -> [DET N]((X1::Y1)(X2::Y2)

((X1 AGR) = *3-SING)((X1 DEF = *DEF)((X2 AGR) = *3-SING)((X2 COUNT) = +)

((Y1 AGR) = *3-SING)((Y1 DEF) = *DEF)((Y2 AGR) = *3-SING)((Y2 GENDER) = (Y1 GENDER)))

Page 44: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 44

Transfer Rule Formalism (II)

Value constraints

Agreement constraints

;SL: the man, TL: der MannNP::NP [DET N] -> [DET N]((X1::Y1)(X2::Y2)

((X1 AGR) = *3-SING)((X1 DEF = *DEF)((X2 AGR) = *3-SING)((X2 COUNT) = +)

((Y1 AGR) = *3-SING)((Y1 DEF) = *DEF)((Y2 AGR) = *3-SING)((Y2 GENDER) = (Y1 GENDER)))

Page 45: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 45

Rule Learning - Overview

• Goal: Acquire Syntactic Transfer Rules• Use available knowledge from the source

side (grammatical structure)• Three steps:

1. Flat Seed Generation: first guesses at transfer rules; flat syntactic structure

2. Compositionality: use previously learned rules to add hierarchical structure

3. Seeded Version Space Learning: refine rules by generalizing with validation (learn appropriate feature constraints)

Page 46: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 46

Examples of Learned Rules (I){NP,14244}

;;Score:0.0429

NP::NP [N] -> [DET N]

(

(X1::Y2)

)

{NP,14434}

;;Score:0.0040

NP::NP [ADJ CONJ ADJ N] ->

[ADJ CONJ ADJ N]

(

(X1::Y1) (X2::Y2)

(X3::Y3) (X4::Y4)

)

{PP,4894};;Score:0.0470PP::PP [NP POSTP] -> [PREP NP]((X2::Y1)(X1::Y2))

Page 47: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 47

A Limited Data Scenario for Hindi-to-English

• Put together a scenario with “miserly” data resources:– Elicited Data corpus: 17589 phrases– Cleaned portion (top 12%) of LDC dictionary: ~2725

Hindi words (23612 translation pairs)– Manually acquired resources during the SLE:

• 500 manual bigram translations• 72 manually written phrase transfer rules• 105 manually written postposition rules• 48 manually written time expression rules

• No additional parallel text!!

Page 48: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 48

Manual Grammar Development

• Covers mostly NPs, PPs and VPs (verb complexes)

• ~70 grammar rules, covering basic and recursive NPs and PPs, verb complexes of main tenses in Hindi (developed in two weeks)

Page 49: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 49

Manual Transfer Rules: Example;; PASSIVE OF SIMPLE PAST (NO AUX) WITH LIGHT VERB;; passive of 43 (7b){VP,28}VP::VP : [V V V] -> [Aux V]( (X1::Y2) ((x1 form) = root) ((x2 type) =c light) ((x2 form) = part) ((x2 aspect) = perf) ((x3 lexwx) = 'jAnA') ((x3 form) = part) ((x3 aspect) = perf) (x0 = x1) ((y1 lex) = be) ((y1 tense) = past) ((y1 agr num) = (x3 agr num)) ((y1 agr pers) = (x3 agr pers)) ((y2 form) = part))

Page 50: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 50

Manual Transfer Rules: Example

; NP1 ke NP2 -> NP2 of NP1; Ex: jIvana ke eka aXyAya; life of (one) chapter ; ==> a chapter of life;{NP,12}NP::NP : [PP NP1] -> [NP1 PP]( (X1::Y2) (X2::Y1); ((x2 lexwx) = 'kA'))

{NP,13}NP::NP : [NP1] -> [NP1]( (X1::Y1))

{PP,12}PP::PP : [NP Postp] -> [Prep NP]( (X1::Y2) (X2::Y1))

NP

PP NP1

NP P Adj N

N1 ke eka aXyAya

N

jIvana

NP

NP1 PP

Adj N P NP

one chapter of N1

N

life

Page 51: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 51

Adding a “Strong” Decoder

• XFER system produces a full lattice• Edges are scored using word-to-word

translation probabilities, trained from the limited bilingual data

• Decoder uses an English LM (70m words)• Decoder can also reorder words or phrases (up

to 4 positions ahead)• For XFER(strong) , ONLY edges from basic XFER

system are used!

Page 52: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 52

Testing Conditions

• Tested on section of JHU provided data: 258 sentences with four reference translations– SMT system (stand-alone)– EBMT system (stand-alone)– XFER system (naïve decoding)– XFER system with “strong” decoder

• No grammar rules (baseline)• Manually developed grammar rules• Automatically learned grammar rules

– XFER+SMT with strong decoder (MEMT)

Page 53: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 53

Results on JHU Test Set (very miserly training data)System BLEU M-BLEU NIST

EBMT 0.058 0.165 4.22

SMT 0.093 0.191 4.64

XFER (naïve) man grammar

0.055 0.177 4.46

XFER (strong)

no grammar0.109 0.224 5.29

XFER (strong) learned grammar

0.116 0.231 5.37

XFER (strong) man grammar

0.135 0.243 5.59

XFER+SMT 0.136 0.243 5.65

Page 54: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 54

Effect of Reordering in the Decoder

NIST vs. Reordering

4.8

4.9

5

5.1

5.2

5.3

5.4

5.5

5.6

5.7

0 1 2 3 4

reordering window

NIS

T s

core no grammar

learned grammar

manual grammar

MEMT: SFXER+ SMT

Page 55: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 55

Observations and Lessons (I)• XFER with strong decoder outperformed SMT even

without any grammar rules in the miserly data scenario– SMT Trained on elicited phrases that are very short– SMT has insufficient data to train more discriminative

translation probabilities– XFER takes advantage of Morphology

• Token coverage without morphology: 0.6989• Token coverage with morphology: 0.7892

• Manual grammar currently somewhat better than automatically learned grammar– Learned rules did not yet use version-space learning– Large room for improvement on learning rules – Importance of effective well-founded scoring of learned rules

Page 56: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 56

Observations and Lessons (II)

• MEMT (XFER and SMT) based on strong decoder produced best results in the miserly scenario.

• Reordering within the decoder provided very significant score improvements– Much room for more sophisticated grammar rules– Strong decoder can carry some of the reordering

“burden”

Page 57: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 57

Conclusions• Transfer rules (both manual and learned) offer

significant contributions that can complement existing data-driven approaches– Also in medium and large data settings?

• Initial steps to development of a statistically grounded transfer-based MT system with:– Rules that are scored based on a well-founded

probability model – Strong and effective decoding that incorporates the

most advanced techniques used in SMT decoding

• Working from the “opposite” end of research on incorporating models of syntax into “standard” SMT systems [Knight et al]

• Our direction makes sense in the limited data scenario

Page 58: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 58

Future Directions• Continued work on automatic rule learning

(especially Seeded Version Space Learning)• Improved leveraging from manual grammar

resources, interaction with bilingual speakers• Developing a well-founded model for assigning

scores (probabilities) to transfer rules• Improving the strong decoder to better fit the

specific characteristics of the XFER model• MEMT with improved

– Combination of output from different translation engines with different scorings

– strong decoding capabilities

Page 59: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 59

Rule Learning - Overview

• Goal: Acquire Syntactic Transfer Rules• Use available knowledge from the source

side (grammatical structure)• Three steps:

1. Flat Seed Generation: first guesses at transfer rules; no syntactic structure

2. Compositionality: use previously learned rules to add structure

3. Seeded Version Space Learning: refine rules by generalizing with validation

Page 60: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 60

Flat Seed Generation

Create a transfer rule that is specific to the sentence pair, but abstracted to the POS level. No syntactic structure.

Element Source

SL POS sequence f-structure

TL POS sequence TL dictionary, aligned SL words

Type information corpus, same on SL and TL

Alignments informant

x-side constraints f-structure

y-side constraints TL dictionary, aligned SL words (list of projecting features)

Page 61: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 61

Flat Seed Generation - Example

The highly qualified applicant did not accept the offer.Der äußerst qualifizierte Bewerber nahm das Angebot nicht an.

((1,1),(2,2),(3,3),(4,4),(6,8),(7,5),(7,9),(8,6),(9,7))

S::S [det adv adj n aux neg v det n] -> [det adv adj n v det n neg vpart](;;alignments:(x1:y1)(x2::y2)(x3::y3)(x4::y4)(x6::y8)(x7::y5)(x7::y9)(x8::y6)(x9::y7));;constraints:((x1 def) = *+) ((x4 agr) = *3-sing) ((x5 tense) = *past) …. ((y1 def) = *+) ((y3 case) = *nom) ((y4 agr) = *3-sing) …. )

Page 62: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 62

Compositionality - Overview

• Traverse the c-structure of the English sentence, add compositional structure for translatable chunks

• Adjust constituent sequences, alignments• Remove unnecessary constraints, i.e. those

that are contained in the lower-level rule• Adjust constraints: use f-structure of correct

translation vs. f-structure of incorrect translations to introduce context constraints

Page 63: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 63

Compositionality - Example

S::S [det adv adj n aux neg v det n] -> [det adv adj n v det n neg vpart](;;alignments:(x1:y1)(x2::y2)(x3::y3)(x4::y4)(x6::y8)(x7::y5)(x7::y9)(x8::y6)(x9::y7));;constraints:((x1 def) = *+) ((x4 agr) = *3-sing) ((x5 tense) = *past) …. ((y1 def) = *+) ((y3 case) = *nom) ((y4 agr) = *3-sing) …. )

S::S [NP aux neg v det n] -> [NP v det n neg vpart](;;alignments:(x1::y1)(x3::y5)(x4::y2)(x4::y6)(x5::y3)(x6::y4);;constraints:((x2 tense) = *past) …. ((y1 def) = *+) ((y1 case) = *nom) …. )

NP::NP [det AJDP n]-> [det ADJP n]

((x1::y1)…((y3 agr) = *3-sing)((x3 agr = *3-sing)

….)

Page 64: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 64

Seeded Version Space Learning: Overview

• Goal: further generalize the acquired rules• Methodology:

– Preserve general structural transfer– Consider relaxing specific feature constraints

• Seed rules are grouped into clusters of similar transfer structure (type, constituent sequences, alignments)

• Each cluster forms a version space: a partially ordered hypothesis space with a specific and a general boundary

• The seed rules in a group form the specific boundary of a version space

• The general boundary is the (implicit) transfer rule with the same type, constituent sequences, and alignments, but no feature constraints

Page 65: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 65

Seeded Version Space Learning

NP v det n NP VP …1. Group seed rules into version spaces as above.2. Make use of partial order of rules in version space. Partial order is defined

via the f-structures satisfying the constraints.3. Generalize in the space by repeated merging of rules:

1. Deletion of constraint2. Moving value constraints to agreement constraints, e.g.

((x1 num) = *pl), ((x3 num) = *pl) ((x1 num) = (x3 num)

4. Check translation power of generalized rules against sentence pairs

Page 66: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 66

Seeded Version Space Learning: Example

S::S [NP aux neg v det n] -> [NP v det n neg vpart](;;alignments:(x1::y1)(x3::y5)(x4::y2)(x4::y6)(x5::y3)(x6::y4);;constraints:((x2 tense) = *past) …. ((y1 def) = *+) ((y1 case) = *nom) ((y1 agr) = *3-sing) … )((y3 agr) = *3-sing) ((y4 agr) = *3-sing)… )

S::S [NP aux neg v det n] -> [NP v det n neg vpart](;;alignments:(x1::y1)(x3::y5)(x4::y2)(x4::y6)(x5::y3)(x6::y4);;constraints:((x2 tense) = *past) …((y1 def) = *+) ((y1 case) = *nom) ((y1 agr) = *3-plu) …((y3 agr) = *3-plu) ((y4 agr) = *3-plu)… )

S::S[NP aux neg v det n] -> [NP n det n neg vpart](;;alignments:(x1::y1)(x3::y5)(x4::y2)(x4::y6)(x5::y3)(x6::y4);;constraints:((x2 tense) = *past) …((y1 def) = *+) ((y1 case) = *nom)((y4 agr) = (y3 agr))… )

Page 67: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 67

Preliminary Evaluation

• English to German• Corpus of 141 ADJPs, simple NPs and

sentences• 10-fold cross-validation experiment• Goals:

– Do we learn useful transfer rules?– Does Compositionality improve

generalization?– Does VS-learning improve generalization?

Page 68: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 68

Summary of Results

• Average translation accuracy on cross-validation test set was 62%

• Without VS-learning: 43%• Without Compositionality: 57%• Average number of VSs: 24• Average number of sents per VS: 3.8• Average number of merges per VS: 1.6• Percent of compositional rules: 34%

Page 69: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 69

Conclusions

• New paradigm for learning transfer rules from pre-designed elicitation corpus

• Geared toward languages with very limited resources

• Preliminary experiments validate approach: compositionality and VS-learning improve generalization

Page 70: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 70

Future Work

1. Larger, more diverse elicitation corpus2. Additional languages (Mapudungun…)3. Less information on TL side4. Reverse translation direction5. Refine the various algorithms:

• Operators for VS generalization• Generalization VS search• Layers for compositionality

6. User interactive verification

Page 71: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 71

Seeded Version Space Learning: Generalization

• The partial order of the version space:Definition: A transfer rule tr1 is strictly more general than another transfer rule tr2 if all f-structures that are satisfied by tr2 are also satisfied by tr1.

• Generalize rules by merging them:– Deletion of constraint– Raising two value constraints to an agreement

constraint, e.g. ((x1 num) = *pl), ((x3 num) = *pl) ((x1 num) = (x3 num))

Page 72: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 72

Seeded Version Space Learning: Merging Two Rules

Merging algorithm proceeds in three steps. To merge tr1 and tr2 into trmerged:

1. Copy all constraints that are both in tr1 and tr2 into trmerged

2. Consider tr1 and tr2 separately. For the remaining constraints in tr1 and tr2 , perform all possible instances of raising value constraints to agreement constraints.

3. Repeat step 1.

Page 73: Enabling MT for Languages with Limited Resources Alon Lavie Language Technologies Institute Carnegie Mellon University

October 5, 2004 TMI 2004 Panel 73

Seeded Version Space Learning:The Search

• The Seeded Version Space algorithm itself is the repeated generalization of rules by merging

• A merge is successful if the set of sentences that can correctly be translated with the merged rule is a superset of the union of sets that can be translated with the unmerged rules, i.e. check power of rule

• Merge until no more successful merges