Knowledge Architecturein the Enterprise 2.0

Tom ReamyChief Knowledge Architect

KAPS Group

Knowledge Architecture Professional Services

http://www.kapsgroup.com

2

Agenda

Introduction

Web 2.0: Folksonomies

Search, Content Management, Text Analytics

Conclusion

3

Folksonomies – Wikipedia Definition

Folksonomy (also known as collaborative tagging, social classification, social indexing, and social tagging) is the practice and method of collaboratively creating and managing tags to annotate and categorize content. Folksonomy describes the bottom-up classification systems that emerge from social tagging.[1] In contrast to traditional subject indexing, metadata is generated not only by experts but also by creators and consumers of the content. Usually, freely chosen keywords are used instead of a controlled vocabulary.[2] Folksonomy (from folk + taxonomy) is a user generated taxonomy.

4

Web 2.0 – No need for Taxonomies etc.?

“Tags are great because you throw caution to the wind, forget about whittling down everything into a distinct set of categories and instead let folks loose categorizing their own stuff on their own terms." - Matt Haughey - MetaFilter

Tyranny of the majority - worst type of central authority More Madness of Crowds than Wisdom of Crowds “Things fall apart; the center cannot hold;

Mere anarchy is loosed upon the world,…The best lack all conviction, while the worstAre full of passionate conviction.” - The Second Coming – W.B. Yeats

5

Advantages of Folksonomies

Simple (no complex structure to learn)– No need to learn difficult formal classification system

Lower cost of categorization– Distributes cost of tagging over large population

Open ended – can respond quickly to changes Relevance – User’s own terms Support serendipitous form of browsing Easy to tag any object – photo, document, bookmark Better than no tags at all Getting people excited about metadata!

6

Folksonomies – Problems and Limits

Folksonomies don’t compare with taxonomies or ontologies Serendipity browsing is small part of search Limited areas of success – popular sites are popular Quality Content – finance, science, etc – not good candidates No mechanism for improving folksonomies Scale – Too Big (million hits) – Too Little (200 items) – Amazon

and LibraryThing Need intrinsic value of tagging – not tagging for better tags Bad Tags - idiosyncratic or too broad, errors, limited reach

– Most people can’t tag very well – learned skill

7

Del.icio.us Tags

Design blog software music tools reference art video programming webdesign web2.0 mac howto linux tutorial web free news photography shopping blogs css imported education travel javascript food games

Development inspiration politics flash apple tips java google osx business windows iphone science productivity books toread helath funny internet wordpress ajax ruby research humor fun technology search opensource

Photoshop media recipes cool work article marketing security mobile jobs rails lifehacks tutorials resources php social download diy ubuntu freeware portfolio photo movies writing graphics youtube audio online

8

Del.icio.us - Folksonomy Findability

Too many hits (where have we heard that before?)– Design – 1 Mil, software – 931,259, sex – 129,468

No plurals, stemming (singular preferred)– Folksonomy – 14,073, folksonomies – 3,843, both – 1,891– Blog-1.7M, blogs – 516,340, Weblog- 155,917, weblogs – 36,434,

blogging – 157,922, bloging – 697– Taxonomy – 9.683, taxonomies – 1,574

Personal tags – cool, fun, funny, etc– Good for social research, not finding documents or sites– How good for personal use? Funny is time dependent

9

Library Thing

Book people aren’t much better at tagging High level concepts – psychology (55,000), religion

(120,000), science (101,000) Issue – variety of terms – cognitive science – need at least

40 other tags to cover the actual field of cognitive science Strange tags – book (19,000) – it’s a book site? Combination of facets and topics

– Facets – Date (16th century, 1950’s, 2007) // Function (owned, not read) // Type (graphic novel, novel) // Genre (horror, mystery)

– Topics – majority like Del.icio.us

10

Library Thing – Book on Neuroscience

1) (Location: dining room)(1) biological(1) biology(8) box74(1) Brain(1) brain research(1) brains(1) cognitive neuroscience(1) cognitive science(1) consciousness(1) currently reading(1) HelixHealth(1) kognitionswissenschaft(1) medical(1) medicine(1) neuroscience(19) non-fiction(5) partread(1) Psychology(4) Science(10) textbook(10) theory(1)

Too General: Science, Psychology, biology, textbook Too specific: Location: dining room, box74 Facets: currently reading, partread

11

Better Folksonomies:

Will social networking make tags better? Not so far – example of Del.icio.us – same tags Quality and Popularity are very different things Most people don’t tag, don’t re-tag Study – folksonomies follow NISO guidelines – nouns, etc –

but do they actually work – see analysis Most tags deal with computers and are created by people

that love to do this stuff – not regular users and infrequent users – Beware true believers!

12

Knowledge Architecture : Social Networking KM 2.0 (or 3.0?)

– KM always concerned with social aspects of knowledge New relationship of center and users – more sophisticated

support, more freedom, more suggestions, more user input– - New roles – for users (taggers, part of variety of communities –

both distributed and central)– New roles for central – create feedback system, tweak the evolution

of the system, Develop initial candidates Communities of Practice – apply to tagging, ranking

– Community Maps – formal and informal – Map tags to communities – more useful suggestions– Use tags to uncover communities (see tech SNA)

13

Knowledge Architecture:Technology for Web 2.0 Enterprise Content Management

– Place to add metadata – of all kinds, not just keywords– Policy support – important, part of job performance– Add tag clouds to input page– More sophisticated displays

• Tag clouds mapped to community map• Tag clusters, taxonomy location

Semantic Software – Inxight, Teragram etc.– Suggest terms based on text, on tag clouds

Social Networking – add semantics– SNA – apply to people and tags

Wiki – more powerful than blogs, more work to set up & maintain

14

Knowledge Architecture: Putting it all togetherComplexity Theory and Folksonomies: Feedback

Ranking Methods– Explicit – people rank directly

• Categories, tags, taggers

• Good tags, best bets for terms or categories?

– Implicit – software evaluation, reverse relevance

Ranking Roles– Taggers – everyone (rewards, make it easy and fun)– Meta-taggers – everyone (but levels of meta-taggers)– Editors – tagging system, integration with taxonomy, resolve

disputes, Wikipedia model

15

Knowledge Architecture:Content Structures – Best of Both Worlds Start and end with a formal taxonomy / Ontology

– Findability vastly superior– Communication with others – share tags– Take advantage of conceptual relationships

Tagging experience – folksonomies plus – Users can type any word – system looks it up – plurals, synonyms,

preferred terms, spelling variations– Software suggestions – based on content of bookmark, document

and on popular user tags – natural level not top down– New terms flagged and routed to central team

Facets – for both things and documents (faceted taxonomy)– Software suggests facet values, user override – Cognitively simpler task than own value, complex hierarchy

16

Knowledge Architecture in the EnterpriseTechnology

Text Analytics

Content Management

Search – Browse – Faceted Navigation

17

Varieties of Taxonomy/ Text Analytics Software

Taxonomy Management Text Analytics

– Auto-Categorization, Entity Extraction– Sentiment Analysis

Software Platforms – Content Management, Search

Application Specific– Business Intelligence

Vendors of Taxonomy/ Text Analytics Software

Attensity Business Objects –

Inxight Clarabridge ClearForest Data Harmony / Access

Innovations

Lexalytics Multi-Tes Nstein SchemaLogic Synaptica Teragram Wikionomy Wordmap Lots More

18

19

Why Taxonomy Software?

If you have to ask, you can’t afford it Spreadsheets

– Good for calculations, days of taxonomy development over– (almost)

Ease of use – more productive– Increase speed of taxonomy development– Better Quality – synonyms, related terms, etc.

Distributed development – lower cost, user input (good and bad)

20

Text Analytics Software – Features

Entity Extraction– Multiple types, custom classes

Auto-categorization – Taxonomy Structure – Training sets – Bayesian, Vector space– Terms – literal strings, stemming, dictionary of related terms– Rules – simple – position in text (Title, body, url)– Boolean– Full search syntax – AND, OR, NOT– Advanced – NEAR (#), PARAGRAPH, SENTENCE

Advanced Features– Facts / ontologies /Semantic Web – RDF +– Sentiment Analysis

21

Current State of Content Management

Content Management– Strong on management, weak on content– Content is a black box – simply moved around

What is missing is the meaning dimension– In-depth and articulated understanding of content

Perceived Solution – Delphi Survey – Taxonomy– 90% plan on taxonomy strategy in 24 months– 76% taxonomy is important

Text Analytics integrated into CM - essential

22

Taxonomic Content Management: Work Flow with Meaning Preliminary Foundation Work

– Design the ontology– Develop taxonomies– Design metadata standards– Collaborative development of controlled vocabularies

Authors, SME’s – check document in:– Have a summary either written by human or software– List of metadata suggestions, entities – people, places, etc.– Provisional categorization– Decision: publish or submit for review, central team or community of

experts.– Request for additional keywords or categorization issues

23

Taxonomic Content Management: Work Flow with Meaning

Central Team– Review documents – easier, faster– Use summaries, metadata, entities to provide context – Review infrastructure requests – new keywords, categories

Integrated Work Flow– Strengths of local and central– Variety of roles, flexible (few dedicated roles needed).– Collaborative categorization and keywords by SME, software,

and central team• SME’s can function as central team

24

Semantics and Search: An Integrated Approach:Elements Multiple Knowledge Structures

– Facet – orthogonal dimension of metadata– Taxonomy - Subject matter / aboutness– Ontology – Relationships / Facts– Subject – Verb - Object

Software - Text analytics, auto-categorization, entity extraction

People – tagging, evaluating tags, fine tune rules and taxonomy

People – Users, social tagging, suggestions

25

Faceted Navigation: Strengths and Weaknesses

Strengths:– More intuitive – easy to guess what is behind each door

• 20 questions – we know and use– Dynamic selection of categories

• Allow multiple perspectives– Trick Users into “using” Advanced Search

• wine where color = red, price = x-y, etc.. Weaknesses:

– Difficulty of expressing complex relationships • Simplicity of internal organization

– Loss of Browse Context• Difficult to grasp scope and relationships

– Limited Domain Applicability – type and size• Entities not concepts, documents, web sites

26

27

28

29

30

31

Integrated Design – Facets & SemanticsDesign Issues - General What is the right combination of elements?

– Faceted navigation, metadata, browse, search, categorized search results, file plan

What is the right balance of elements?– Dominant dimension or equal facets

Full Facets – Multiple intersecting filters– 1 or 2 filters (source / type) – No

When to combine search, topics, and facets?– Search first and then filter by topics / facet– Browse/facet front end with a search box

32

Integrated Design – Facets & SemanticsDesign Issues - General Good Information Architecture

– Space wars – summary or full facet display– Simplicity vs. research power– Source and Type are basics– Standard Facets – People, Companies, Place, Industry– Interactive interface – sliders, date ranges

Semantics still hardest – summaries, related, rank Taxonomy – just another facet?

– Keywords vs. simple taxonomy Tag Clouds / Clusters – how useful? Feedback – numbers of stories vs. top stories

33

Conclusion

Web 2.0 is not the answer. The answer is: A knowledge architecture that integrates web 2.0 with:

– Knowledge structures like taxonomies, facets, and basic level categories

– Text analytics – categorization, entity extraction– Content Management with meaning– KA team that works with distributed groups in a variety of

ways– A smart search that builds on a Knowledge Architecture that

includes taxonomies, faceted metadata, best bets and search logs

34

Resources

Information Today – Enterprise Search Sourcebook – 2008 Knowledge Architecture Approach to Search

– Tom Reamy – Enterprise Search Sourcebook – 2009 CMS Watch – Content Management

– www.cmswatch.com Text Analytics Software

– Information Today – www.infotoday.com Web Sites

– Taxonomy Community of Practice: http://finance.groups.yahoo.com/group/TaxoCoP/