Lexical acquisition through particular adjectival endings for Croatian

Lexical acquisition through particular

adjectival endings for Croatian

Božo Bekavac, Krešimir ŠojatInstitute of Linguistics, Faculty of Philosophy, Zagreb

Motivation & Goals

• Recognition of unknown words necessary for many NLP applications

• No attempt for Croatian so far • Focus on recognition of adjectives based on

characteristic endings • Addition of recognized adjectives into

general lexicon • Creation of dynamic rule-based resource

Approach

• assumption adjectives unrecognized by the common lexicon tend to follow regular derivational patterns

• e.g. cyberski (cyber-), imunobioloških (immuno-biological), eurooptimističnog (eurooptimistic)

• Focus on adjectives, but applicable to other parts of speech

Resources used

• Croatian Morphological Lexicon (CML) - 621.000 types generated from ca 33.000 lemmas

• 30 M newspaper corpus consisting of 195.534 types

• (There will always be words not covered by general lexicons)

Adjectives in Croatian (1) Multext - East specification

1) type (qualificative, possessive)2) degree (positive, comparative, superlative)3) gender (masculine, feminine, neuter)4) number (singular, plural)5) case (nominative, genitive, dative, accusative, vocative, locative, instrumental)6) definiteness7) animate (relevant only for masculine-singular-

accusative)

Adjectives in Croatian (2)

• Adjectives: an open and productive class of words

• Morphologic features: derivation+inflection• Derivation:• suffix (e.g. Tomislav Tomislavov)• prefix + suffix (e.g. nad morem nadmorski)• compound + suffix (e.g. primorsko-goranski,

srednjoškolski)

Adjectives in Croatian (3)

• Inflection (e. g singular):dvojb - en

dvojb - ena

dvojb - enu

dvojb - en

dvojb - enu

dvojb - enim

Consequence

• Potential number for adjectival MSD interpretation is 256

• A great number of suffixes overlapping of suffixes (endings and ends) of different POS especially between Adjectives and Nouns

Internal homography (1)

• where same token represents different word-forms of the same lemma

• EXAMPLE: the word-form modalnom of the lemma modalan has five possible MSDs – Amsd, Amsl, Afsi, Ansd, Ansl

• All different MSDs with internal homography grouped under the same ending –alnom

Internal homography (2)

• modalan Afpmsan-n, Afpmsnn

• modalna Afpfsnn, Afpfsny, Afpfsvy, Afpmsan-y, Afpmsgn

• modalni Afpnpan, Afpnpay, Afpnpnn, Afpnpny, Afpnpvy, Afpnsgn, Afpfpan

• modalne Afpfpay, Afpfpnn, Afpfpny, Afpfpvy, Afpfsgn, Afpfsgy, Afpmpan, ...

External homography (1)

• where the same token represents different word-forms (i.e. MSD interpretations) of two or more lemmas

• EXAMPLE: kos– noun kos (Nmsn) of the lemma kos (blackbird)– adjective kos (Amsa; Amsn) of the lemma kos

(slant)

External homography (2) - endings

• Adjectival endings regularly homographic with those of other parts of speech were not taken into consideration at all

• Adjectival paradigms that are partially homographic only unambiguous endings used

External homography: endings/ends

Order of processing

CML (common lexicon)

RECOGNIZER(lexical transducer)

Temporary lexicon of unknown adjectives

Generation ofall word-forms

Lexical transducer –alan.grf

alan+.grfWed Jun 15 10:24:13 2005

<L> )(Root {$Root#$Suf,$Root#alan.A+ 453,452/0/442 af bk+alan}

(Suf

)

alanalnaalnealnialnihalnimalnimaalnoalnogalnogaalnojalnomalnomealnomualnu

mod

Variables

alne

Output

modalne,modalan.A+453,452/0/442 af bk

24 transducers i.e. different paradigms used

Lexical transducer –alan.grf applied on running text

ambijentalni,ambijentalan.A+453,452/0/442 af bk

bijenalna,bijenalan.A+ 453,452/0/442 af bk

cerebrospinalne,cerebrospinalan.A+453,452/0/442 af bk

doktrinalnom,doktrinalan.A+453,452/0/442 af bk

dvodimenzionalnima,dvodimenzionalan.A+453,452/0/442 af bk

dvokanalnom,dvokanalan.A+453,452/0/442 af bk ...

inflectional pattern code

Temporary final lexicon

• Results of lexical transducers stored in temporary lexicon

• Inflectional pattern code and lemma used for generation of all wfs of recognized A

• Such order of processing correctly recognizes wf dvojben as A and does not missclasify wfs with same ends (e. g. bazen)

• Results of generation stored in final lexicon

Final lexicon

aboridžinska,aboridžinski.A:qtfsn-:qtfsv-:qtrpa-:qtrpn-:qtrpv-

aboridžinske,aboridžinski.A:qtfpa-:qtfpn-:qtfpv-:qtfsg-:qtmpa-

aboridžinski,aboridžinski.A:qtmpn-:qtmpv-:qtmsay--:qtmsn-:qtmsv-

aboridžinskih,aboridžinski.A:qtfpg-:qtmpg-:qtrpg-aboridžinskim,aboridžinski.A:qtfpd-:qtfpi-:qtfpl-:qt

mpd-:qtmpi-:qtmpl-:qtmsi-:qtrpd-:qtrpi-:qtrpl-:qtrsi-...

Results (1)

Results (2)

• 13.933 new adjectival word-forms found by recognizer

• 5.035 word-forms belong to different lemmas

• 4.511 new lemmas added into the CML (after manual inspection) 393 type err!

• Precision: 97.01 %

Problems

• Beside inevetable type errors 131 wfs misclassified due to :

1. NE endings/ends homographic with adjectival endings (Joško, Aljaska)

2. Small amount of other POS still not present in the CML (ekosustav)

3. Foreign words and words of foreign origin (certificate)

Solution

• AD 1) is to preprocess the corpus with NERC system developed for Croatian (Bekavac, 2005)

• AD 2) the problem will be solved after the automatic disambiguation of word-forms when added into the CML

• AD 3) foreign words used in their original spelling (e.g. certificate) are not being added into the CML by default not big amount

Frequency of particular adjectival endings found in corpus

ski

čki

av

ški

alan

ičan

ljiv

ast

ovan

evan

Conclusion and future work

• Dynamic resource highly efficient for specific domains

• Applied order of processing overgeneration of word-forms is avoided

• FW to apply same metodology on other open word classess (Nouns and Verbs)