WikiNet:a very large-scale multi-lingual concept network

Vivi Nastase, Michael Strube

HITS gGmbH

Benjamin Börschinger, Cäcilia Zirn

University of Heidelberg

Anas Elghafari

University of Tübingen

2

Democratic Senator Barack Obama has been elected the first black president of the United States, prompting celebrations across the country. "It's been a long time coming, but tonight... change has come to America," the president-elect told a jubilant crowd at a victory rally in Chicago. His rival John McCain accepted defeat, and called on his supporters to lend the next president their goodwill.

3

Democratic Senator Barack Obama has been elected the first black president of the United States, prompting celebrations across the country. "It's been a long time coming, but tonight... change has come to America," the president-elect told a jubilant crowd at a victory rally in Chicago. His rival John McCain accepted defeat, and called on his supporters to lend the next president their goodwill.

4

Democratic Senator Barack Obama has been elected the first black president of the United States, prompting celebrations across the country. "It's been a long time coming, but tonight... change has come to America," the president-elect told a jubilant crowd at a victory rally in Chicago. His rival John McCain accepted defeat, and called on his supporters to lend the next president their goodwill.

5

Democratic Senator Barack Obama has been elected the first black president of the United States, prompting celebrations across the country. "It's been a long time coming, but tonight... change has come to America," the president-elect told a jubilant crowd at a victory rally in Chicago. His rival John McCain accepted defeat, and called on his supporters to lend the next president their goodwill.

6

Democratic Senator Barack Obama has been elected the first black president of the United States, prompting celebrations across the country. "It's been a long time coming, but tonight... change has come to America," the president-elect told a jubilant crowd at a victory rally in Chicago. His rival John McCain accepted defeat, and called on his supporters to lend the next president their goodwill.

7

Democratic Senator Barack Obama has been elected the first black president of the United States, prompting celebrations across the country. "It's been a long time coming, but tonight... change has come to America," the president-elect told a jubilant crowd at a victory rally in Chicago. His rival John McCain accepted defeat, and called on his supporters to lend the next president their goodwill.

8

Democratic Senator Barack Obama has been elected the first black president of the United States, prompting celebrations across the country. "It's been a long time coming, but tonight... change has come to America," the president-elect told a jubilant crowd at a victory rally in Chicago. His rival John McCain accepted defeat, and called on his supporters to lend the next president their goodwill.

9

Democratic Senator Barack Obama has been elected the first black president of the United States, prompting celebrations across the country. "It's been a long time coming, but tonight... change has come to America," the president-elect told a jubilant crowd at a victory rally in Chicago. His rival John McCain accepted defeat, and called on his supporters to lend the next president their goodwill.

10

Barack Obamahis

United StatesAmerica

presidentpresident-elect

11

Barack Obamahis

United StatesAmerica

presidentpresident-elect

12

Barack Obamahis United States

America

presidentpresident-elect

country

is asenator (2005-2008)

Chicago

part of

13

Making knowledge accessible to computers

14

The many facets of Wikipedia

15

16

17

18

The many facets of Wikipedia• Articles• Categories• Category-category links• Article-category links• Hyperlinks, redirects, disambiguation links• Anchor texts• Cross-language links• Infoboxes• Article structure• Edits and more

19

and now let's restructure (some of) this for machine consumption

20

The category and article network

21

– explicit relationsPrime Ministers of New Zealand, Movies directed by Woody Allen

– partly explicit relationsVillages in Brandenburg, Conflicts in 2000

– implicit relationsMixed martial arts television programs

– class attributesAlbums by artist, Books by genre

Wikipedia categories encode knowledge

22

Deconstructing categories

23

Deconstructing categories

24

Deconstructing Wikipedia categories

Apply patterns to parsed structure:– Queen (band) members,

Chancellors of Germany:Rel Cx, Cx Rel– Films directed by Woody Allen:Cy [VBN IN] Cx– Villages in Brandenburg:Cy [IN] Cx– Albums by artist:Cy by Cx

– Mixed martial arts television programs:Cx1 Cx2 … Cxn

25

Deconstructing categories→ explicit relation categories: Cy Rel

Queen (band) members

Freddy Mercury

Brian May

John Deacon

Roger Taylor

Queen (band)

Freddy Mercury

Brian May

John Deacon

Roger Taylor

members

26

Deconstructing categories→ explicit relation categories:

Cy [VBN IN] Cx

Movies Woody Allen

Annie Hall

Deconstructing HarryThe Scoop

…

isa directed by

Movies directed by Woody Allen

Annie Hall

Deconstructing Harry

The Scoop…

27

Deconstructing categories partly explicit relations Cy [IN] Cx

Villages in Brandenburg

Siethen Goerden

Villages

Siethen Goerden

Brandenburg

isa located-in

Geography

spatial

28

Deconstructing categories by categories: Cx by Cy

attribute

29

30

Category and articles network + deconstructed categories

31

How good is it?

32

Some issues

Relations that are too general – Military equipment of the Soviet Union → spatial

Unnamed relations – science fiction novels

33

Some issues

34

Category name ↔ infobox relation/attribute linkestablished for 42,060 categories

Propagated to 698,929 previously induced relations

35

36

After: Starting with category-article network Deconstructing categories Propagating infobox relations Mapping all these onto concepts

37

38

After: Starting with category-article network Deconstructing categories Propagating infobox relations Mapping all these onto concepts Adding cross-language links

39

40

41

After: Adding category-article network Deconstructing categories Propagating infobox relations Mapping all these onto concepts Adding cross-language links Adding some disambiguating pageswe obtain ...

42

WikiNet – a multi-lingual concept network

43

WikiNet – a multi-lingual concept network

A multi-lingual index- maps terms to ids

44

WikiNet – a multi-lingual concept network

A reversed index file to map ids to terms

45

WikiNet – a multi-lingual concept network

A list of named relations between ids

46

WikiNet – a multi-lingual concept network

A list of co-occurrence relations between ids

47

WikiNet

• Approx 3 million conceptsMultilingual accessParaphrases and variants

• Approx 36 ( + 162 ) million relations

48

WikiNet

• Language stats:– En: 3,347,712– Fr: 498,811– De: 459,697– It: 368,529– Tr: 83,239– Bg: 55,068...

49

What's next:

• Fill in cross-language gaps• Add more concepts – identify texts that contain

concepts we don't know about and extract the knowledge

• Add more relations Article – article linksNovel relations from texts

• Add organizationGet inspiration from Cyc!

50

... and also:

• Build a tool kit for use with WikiNet:Link texts to WikiNetVarious computations (relatedness, similarity, associations)

51

... and also:

• Put the resource and toolkit to use:

SummarizationMetonymy resolutionCoSyne – content synchronization of wiki pages

52

Thank you!

53

Thank you!

54

Connecting

55

Category links

56

Collaborative projects on the web:

Open Mind Common Sense (Push Singh) Mindpixel (Chris McKinstry)

How to break the knowledge acquisition bottleneck?