知識ベースを活用したエンティティリンキング

知識ベースを活用したエンティティリンキング

山田育矢 (Ikuya Yamada)

株式会社Studio Ousia / 慶應義塾大学 / 国立情報学研究所

15年7月9日木曜日

STUDIO OUSIA

Outline

‣‣

‣

2



STUDIO OUSIA

Entity linkingとは

‣

‣✦

✦

✦

4

Kyary Pamyu Pamyu is a Japanese model and singer. Her public image is associated with Japan's kawaisa culture

centered in the Harajuku, Tokyo.


STUDIO OUSIA

Entity linkingとは

‣

‣✦

✦

✦

4



Harajukuwikipedia/Harajuku wikipedia/Kawaii

KawaiiKyary Pamyu Pamyuwikipedia/Kyary_Pamyu_Pamyu


http://en.wikipedia.org/wiki/Harajuku


http://en.wikipedia.org/wiki/Kawaisa




STUDIO OUSIA

‣

5




STUDIO OUSIA

‣

5




STUDIO OUSIA

‣

‣

6




STUDIO OUSIA

‣

‣

6




STUDIO OUSIA

エンティティの特徴

‣

‣

‣

7


STUDIO OUSIA

エンティティの特徴: DBpedia Ontology Classes

‣

‣‣

8

http://mappings.dbpedia.org/server/ontology/classes/

Kyary Pamyu Pamyu: MusicalArtist, Artist, Person, Agent

iPhone: InformationAppliance, Device

Japan: Country, PopulatedPlace, Place




STUDIO OUSIA

エンティティの特徴: Wikipedia Link-based Measure[Milne and Witten ’08]

9

Image taken from Milne and Witten 2008. An Effective, Low-Cost Measure of Semantic Relatedness Obtained from Wikipedia Links


STUDIO OUSIA

何がエンティティ?‣

✦

✦

‣✦

‣

✦

✦

10


Evaluating the Helpfulness of Linked Entities to Readers [HT ’15]


STUDIO OUSIA

人に役立つエンティティ

‣

‣

✦

12




STUDIO OUSIA

人に役立つエンティティ

‣

‣

✦

12




STUDIO OUSIA

Wikipedia Manual of Style

‣ relevant and helpful

‣

13

http://en.wikipedia.org/wiki/Wikipedia:Manual_of_Style#Wikilinks




STUDIO OUSIA

アプローチ

‣

‣‣

14



TRUE FALSE/


STUDIO OUSIA

アプローチ

‣

‣‣

14



TRUE FALSE/


STUDIO OUSIA

アプローチ

‣

‣

15

Entity LinkingHelpfulness Evaluation

(Our Method)Input Text Entities

エンティティリンキングの後処理のステップとして実装


STUDIO OUSIA

List of machine-learning features

特徴量

‣‣‣‣‣‣

16

特徴量は６つのグループに分けられる


STUDIO OUSIA

Link Probability Features

‣

‣✦

✦

✦

17

Ca(m): A set of entities that contain mention as an anchorCt(m): A set of entities that contain mention

Link probabilityはWikipedia内で該当する文字列(メンション)がリンク文字列として出現する確率をあらわす


STUDIO OUSIA


18

Her public image is associated with Japan's kawaisa

culture centered in the Harajuku, Tokyo

Takeshita Street is a street lined with

fashion boutiques, cafes in Harajuku in

Tokyo, Japan.

Department Store and Museum is a department store

located in the Harajuku...

Takeshita Street Kyary Pamyu Pamyu Laforet

Link Plain text



http://en.wikipedia.org/wiki/Tokyo,_Japan













STUDIO OUSIA


18

Her public image is associated with Japan's kawaisa

culture centered in the Harajuku, Tokyo

Takeshita Street is a street lined with

fashion boutiques, cafes in Harajuku in

Tokyo, Japan.

Department Store and Museum is a department store

located in the Harajuku...

Takeshita Street Kyary Pamyu Pamyu Laforet

Link Plain text

LINK_PROB(Harajuku) = 2/3
















STUDIO OUSIA

Entity Features

‣‣‣‣‣

19


STUDIO OUSIA

Entity Class Features

20

Entity class featuresは、DBpedia, Freebaseから取得したエンティティのクラス情報をあらわす

Kyary Pamyu Pamyuwikipedia/Kyary_Pamyu_Pamyu

Harajukuwikipedia/Harajuku






STUDIO OUSIA


20



DBpedia:/ontology/PersonDBpedia:/ontology/ArtistFreebase:/people/personFreebase:/music/artist


…






STUDIO OUSIA


20



DBpedia:/ontology/PersonDBpedia:/ontology/ArtistFreebase:/people/personFreebase:/music/artist


…

DBpedia:/ontology/PlaceFreebase:/location/locationFreebase:/location/neighborhood…






STUDIO OUSIA

Topical Coherence Feature

‣

‣

‣

21

e: Entity; KB: Entities in KB; c(e): Entities having a link to e

e: Entity; E: Set of entities in the document


STUDIO OUSIA


22



0.2

0.4

REL(Kyary Pamyu Pamyu, kawaisa culture) = 0.4REL(Kyary Pamyu Pamyu, Harajuku) = 0.2


STUDIO OUSIA


22



0.2

0.4

REL(Kyary Pamyu Pamyu, kawaisa culture) = 0.4REL(Kyary Pamyu Pamyu, Harajuku) = 0.2

COHERENCE(Kyary Pamyu Pamyu, E) = (0.4 + 0.2) / 2 = 0.3


STUDIO OUSIA

その他の特徴量‣

✦

✦

✦

✦

✦

✦

✦

‣✦

✦

23


STUDIO OUSIA

Dataset

‣✦

✦

‣

‣‣

24

“Do you think converting the highlighted keyword into a link is sufficientlyhelpful to readers and could it improve the readers’ overall experience?”


STUDIO OUSIA

Dataset

‣✦

✦

‣‣

‣

25


https://github.com/studio-ousia/el-helpfulness-dataset/

https://github.com/studio-ousia/el-helpfulness-dataset/

STUDIO OUSIA 26

Annotation screen displayed to the annotators


STUDIO OUSIA

Machine-learning Algorithms

‣✦

✦

✦

✦

‣✦

✦

27

Precision Recall F1

C4.5 0.8281 0.8069 0.8173

SVMLINEAR 0.8581 0.7892 0.8221

SVMRBF 0.8676 0.7806 0.8217

AdaBoost 0.8484 0.8383 0.8433

Random Forest 0.8697 0.8419 0.8554


STUDIO OUSIA

Ablation Study: Overview

‣

‣

28

Feature Types Precision Recall F1

Link probability only 0.8228 0.8093 0.8158

LP+Entity 0.8511 0.8266 0.8385

LP+Entity class 0.8487 0.8341 0.8412

LP+Topical coherence 0.8281 0.8117 0.8197

LP+Textual 0.8268 0.8266 0.8266

LP+Mention Occurrence 0.8279 0.8117 0.8196

Effectiveness of features by their categories

Category name DescriptionLink probability (LP) Probability that mention is used as anchor

Entity (ENT) Several statistics of entity in WikipediaEntity class (EC) Entity classes in DBpedia and Freebase

Topical coherence (TC) How entity is related to topics of documentTextual (TXT) Several textual features

Mention occurrence (MO) How mention is appeared in documentSummary of Feature Categories


STUDIO OUSIA

Ablation Study: Overview

‣

‣‣

29

Feature Types Precision Recall F1

Link probability only 0.8228 0.8093 0.8158

LP+Entity 0.8511 0.8266 0.8385

LP+Entity class 0.8487 0.8341 0.8412

LP+Topical coherence 0.8281 0.8117 0.8197

LP+Textual 0.8268 0.8266 0.8266

LP+Mention Occurrence 0.8279 0.8117 0.8196

Effectiveness of features by their categories

Category name DescriptionLink probability (LP) Probability that mention is used as anchor

Entity (ENT) Several statistics of entity in WikipediaEntity class (EC) Entity classes in DBpedia and Freebase

Topical coherence (TC) How entity is related to topics of documentTextual (TXT) Several textual features

Mention occurrence (MO) How mention is appeared in documentSummary of Feature Categories


STUDIO OUSIA

1. DBPEDIA/ontology/Agent

2. FREEBASE/people/person

3. FREEBASE/business/employer

4. FREEBASE/organization/organization

5. SCHEMA ORG/Person

6. DBPEDIA/ontology/Person

7. FREEBASE/book/book subject

8. DBPEDIA/ontology/Disease

9. FREEBASE/business/business operation

10. SCHEMA ORG/Organization

11. DBPEDIA/ontology/Organization

12. FREEBASE/sports/proathlete

13. FREEBASE/location/location

14. DBPEDIA/ontology/Athlete

15. DBPEDIA/base/consumermedical/medical term

Ablation Study: Entity Class Features

‣

30

List of top 15 entity class features


STUDIO OUSIA
















‣

‣

31




STUDIO OUSIA
















‣

‣

‣

32




STUDIO OUSIA

まとめ

‣

‣

33


An End-to-End Entity Linking Approach for Tweets [#Microposts ’15]


STUDIO OUSIA

Background

‣

‣

35


STUDIO OUSIA

‣

36


STUDIO OUSIA

#Microposts NEEL Challenge

‣

‣

‣

37


STUDIO OUSIA

#Microposts NEEL Challenge 2015

‣

‣

‣

38


STUDIO OUSIA

#Microposts NEEL 2015コーパス

39

100000025580548097 31 35 http://dbpedia.org/resource/BRIC Thing100000025580548097 69 74 http://dbpedia.org/resource/Intel Organization100000025580548097 3 8 http://dbpedia.org/resource/Intel Organization100008638684475392 104 110 http://dbpedia.org/resource/Afghanistan Location100010138022330368 29 34 http://dbpedia.org/resource/Japan Location

‣‣


http://www.slideshare.net/giusepperizzo/neel2015-challenge-summary


http://dbpedia.org/resource/BRIC

http://dbpedia.org/resource/BRIC

http://dbpedia.org/resource/Intel




http://dbpedia.org/resource/Afghanistan

http://dbpedia.org/resource/Afghanistan

http://dbpedia.org/resource/Japan

http://dbpedia.org/resource/Japan


STUDIO OUSIA

アーキテクチャ

41

Entity Linking

NIL Mention Detection

Input Text Results

4つの教師付き学習器を使用

Type Prediction(KB Entity Mentions)

Type Prediction(NIL Mentions)

✦ エンティティとNILエンティティ(DBpediaに無いエンティティ)を検出し、そのエンティティの種類 (PERSON, LOCATION等)を分類する

✦ Entity Linkingとは別にNIL Mention Detectionモジュールを実装✦ エンティティとNILエンティティそれぞれに対してエンティティの種類の分類器を実装



STUDIO OUSIA

Entity Linking: メンション・エンティティ辞書

‣

‣✦

✦

✦

43

appleApple Inc.

Apple (fruit)


STUDIO OUSIA

Entity Linking: メンション候補の生成

‣

‣

44

Mention Candidate Generation

Mention Detection and Disambiguation


STUDIO OUSIA

Entity Linking: メンションの曖昧検索

‣

✦

✦

✦

‣

45




STUDIO OUSIA

Entity Linking: メンションの検出と曖昧性解消

‣‣‣

✦

✦

✦

✦

46




STUDIO OUSIA

Entity Linking: Wikipediaページビューからの特徴量

‣

‣

‣

47





STUDIO OUSIA

NIL Mention Detection

‣

‣✦

✦

✦

49



STUDIO OUSIA

Type Prediction: Entity Mentions

‣

‣✦

✦

51


STUDIO OUSIA

Type Prediction: NIL Entity Mentions

‣

‣✦

✦

52



STUDIO OUSIA

‣

54




STUDIO OUSIA

‣

54




STUDIO OUSIA

‣

55


STUDIO OUSIA

‣

55



STUDIO OUSIA

‣

‣

‣

57


STUDIO OUSIA 58

‣

‣

‣


STUDIO OUSIA 58

‣

‣

‣


STUDIO OUSIA

まとめ

‣

‣

‣

59


STUDIO OUSIA

ACL W-NUT 2015のShared Taskでも優勝しました!

‣‣

‣

60


http://noisy-text.github.io/

http://noisy-text.github.io/

THANK YOU!


Software

知識ベースを活用したエンティティリンキング