OK So Enterprise Search is "Janky" - Now What?

Copyright © 2016 Earley Information Science1

OK so Enterprise Search is "Janky" – Now What?April 20, 2016

Copyright © 2016 Earley Information Science

Seth Earley, EISDino Eliopulos, EISEd Dale, EYJeff Fried, BA Insight


Today’s Agenda

• Welcome & Housekeeping• Dino Eliopolus, Managing Director, Earley Information Science

(@DEliopulos)

• Session duration & questions

• Session recording & materials

• Take the polls & the survey!

• The Panelist Point of View• Seth Earley, CEO, Earley Information Science (@SethEarley)

• Ed Dale, Ernst & Young (@EdDale)

• Jeff Fried, CTO, BA Insight (@JeffFried)

• Expert Panel Discussion• Questions & Answers• Join the conversation: #earleyroundtable

Copyright © 2016 Earley Information Science3 Copyright © 2016 Earley Information Science

OK so Enterprise Search is "Janky" – Now What?


• Experienced leader and innovator in industry and high-end professional IT consulting with deep specialization in user experience and highly complex business applications.

• Has over 2 decades of experience in applying Machine Learning, Data Mining and other AI techniques to deliver rich content-driven solutions for Retail, CRM, hi-tech manufacturing, healthcare / insurance and financial services.

• Has depth in many industries including Financial Services, Retail / CPG, Telecommunications, Travel and Entertainment, Healthcare, Pharmaceuticals, Hi-Tech Manufacturing and Energy.

• Expertise in all aspect of IT Professional Services including strategy, planning, forecasting, budgeting, measurement, sales, talent acquisition / management and retention, career stewardship, program management and service delivery.

• Highly collaborative and results-oriented management style delivers outstanding outcomes for his clients, his employers and his teams.

Dino Eliopulos - Biography

Dino EliopulosManaging DirectorEarley Information

Science


• For years, organizations have tried to harness the power of search technologies to give employees access to the content and resources they need to do their jobs.

• During that same time, search technology has evolved to include keyword, tagging, natural language and semantic search.

• But the experience that users have with search has not seemed to improve at the rate that search technologies have evolved.

• Organizations continue to introduce new technologies, processes and data into their enterprise to make search better, but still the number one complaint that users have about their intranets is that they can’t find anything.

The Problem with Enterprise Search

"I can't find anything on our portal."Intranet Connections: “10 Complaints about your Intranet Portal”

“One of our staff’s main complaints (there were many) about our old intranet was the search. It was slow and the results didn’t contain what you were looking for.“Interact-Intranet: “What a difference an intranet makes”


• Virtual Assistants– New model that leverages learning and prediction and structured

interactions (E.g., Cortana, Siri, ABIe, Google Now)

• Discovery– Proactive collection of information to create personalized information

feeds for users (E.g., Spotify, Pandora)

• Graph Search– Integrating multiple sources of information together about a user to

improve relevance and personalize results (e.g., Facebook, Sharepoint 2016 / GQL)

Recent Trends in Search that will “help”


Seth Earley - Biography

Seth EarleyCEO and Founder

Earley Information Science

Over 20 years experience

Current work

Co-authorEditor

MemberFormer Co-Chair

FounderFormer adjunct professor

Guest speakerAIIM Master Trainer

Course Developer & Master Instructor

Data science and technology, content and knowledge management systems, background in sciences (chemistry)

Enterprise IA and Semantic Search

Information Organization and Access

US Strategic Command briefing on knowledge networks

Northeastern University

Boston Knowledge Management Forum

Long history of industry education and research in emerging fields

Academy of Motion Picture Arts and Sciences, Science and Technology Council Metadata Project Committee

Editorial Journal of Applied Marketing Analytics

Data Analytics Department IEEE IT Professional Magazine

Practical Knowledge Management from IBM Press

Cognitive computing, knowledge and data management systems, taxonomy, ontology and metadata governance strategies


• The definition of “search” has changed.– It’s not a white box. It’s an experience.– Search is about aggregation, access, and capabilities.– Search algorithms may be improving, but they can’t infer intent or human context (i.e., searcher’s role or

perspective).

• Search can’t be “bolted on” to a project or application.– Search optimization requires integrated design methods:

• user• task and process• content• inputs and outputs• use cases and scenarios

Search Functionality in Context (Search-Based Applications)


ORGANIZING PRINCIPLES &

INFORMATION ARCHITECTURE

SEARCH TOOLS &

TECHNOLOGY

PROCESS, CULTURE &

GOVERNANCE

Findability Ecosystem

Search effectiveness is reliant on a combination of factors

Why do people have trouble locating information?

The answer required us to look at the complete findability ecosystem.


1-UNPREDICTABLE 2-AWARE 3-COMPETENT 4-SYNCHRONIZED 5-CHOREOGRAPHED

METADATA PROCESSES

Chaotic tagging practices with no taxonomy Custom metadata, use of lists

Enterprise taxonomy replicated in site columns & managed

metadata

Document sets, term store, & attributes used effectively to

group, aggregate, sort, & filter assets

Contextualized/ personalized metadata with auto-population

of values & quality audits

IA / USER EXPERIENCE

Haphazard creation sites with inconsistent content models

and poor site experience

More consistent stable departmental sites with little

cross site structure

Site collection consistency, reference metadata with

content relevant to specific groups and audiences

Cross collection consistency, enforced metadata standards

context dependent user experience

Integration of structured and unstructured information from multiple systems dynamically

presented to support user tasks

SEARCH INTEGRATION Random document generator Some tuning of search with

content tagging

Scopeable search with consistent but uncontrolled

tagging

Faceted search with taxonomy-governed tagging

Search-based application with associative relationships (related search), tuned

algorithms, facets, and metrics driven use cases

USER PROFICIENCY AND CONTENT

PRACTICES

Poor or minimal usage, lack of awareness of capabilities or

content practices

Early adopters and power users using out-of-box features, little

control of content

Departmental collaboration with basic content control

Cross-team project-oriented collaboration with information

lifecycle management

Automated workflows and reporting for compliance with enterprise content standards

GOVERNANCEInformation sprawl, lack of

vision, no intentional decision making

Awareness of challenges, activity is monitored but not

constrained

Assigned responsibilities, oversight and communications

infrastructure in place

Intentional decision making, resource allocation, change

controls in effect

Agenda-driven business leadership and stakeholder

engagement to effect continuous process

improvement

Findability Model: Search, IA & Content Management


Bottom-Up Development

Both top-down and bottom up approaches to developing IA are needed

OBSERVE SUMMARIZE CONCEIVE DEVELOP IDENTIFY AUDIENCES

DEFINE TASKS

BUILD USE CASES

IDENTIFY CONTENT ORGANIZE

CONTENT ANALYSIS TAXONOMY METADATA CONTENT

MODELSMENTAL MODELS

SITE MAPS & NAVIGATION WIREFRAMES

Top-down Information Architecture

Bottom-up Information Architecture


How to interpret use cases (example):

• Actor: persona development, actor taxonomy/model• Action: search domains, scenarios• Objective: scenarios and test cases, search design• Content: content types, schemas, autoclassification• Metadata: taxonomies, content models

Detailed Use Cases Library

ACTOR ACTION OBJECTIVE CONTENT USED METADATA

Consultant Find approaches for use in a project

Mobilize an engagement Methodology IndustryProject type Topic


Search as an Application

AUDIENCE ANALYSISCONTENT ANALYSIS

CONTENT AUTHORING

& PUBLISHING PROCESSES

CONTENT TYPES, METADATA SCHEMAS& TAXONOMY DESIGN

WORKFLOW & SYSTEM INTEGRATION

SOLUTIONARCHITECTURE DESIGN

SEARCH BASED APPLICATIONS(CONTENT IN CONTEXT)

WORKSTREAMS

CONTENT MODELING • Content Type Definitions• Metadata Schema Design• Managed Metadata Service

Design• Taxonomy Framework and

Development

Process & Integration –• Workflow Design• Automated vs. Manual Process

Analysis• Online vs. Offline Functional

Capability• Data Integration and

Synchronization

Solution Architecture Design –• Site Collection Architecture• Site Maps and Logical Content

Organization

Search Based Applications –• Navigation• Wireframes• User Interface Design

Audience & Content Analysis –• Content Audits and Inventories• Personas, User and Group

Matrices• User Scenarios and Use Cases

Content Authoring & Publishing –• Content Creation and Curation• Information Lifecycle

Management Design• Publication Process Modeling


Content Continuum: Structure

Less Structure More Structure

• Problem solving• Collaboration• Opportunity work• Creative authoring

KNOWLEDGE CREATION KNOWLEDGE REUSESpans Structured and

Unstructured Processes

CLAS

S O

F AP

PLIC

ATIO

N

BlogsRecords

ManagementDocument

Management

Process Management

Wikis Collaborative Spaces

Instant Messaging

Email Management

Web Content Management

Learning Management

Digital Asset Management

PORTALSpan of Control (SharePoint)MYSITES

• Accessing information• Answering questions• Scheduled work• Content management

CHAOTIC PROCESSES:

CONTROLLED PROCESSES:


Content Continuum: Value

Less Value More Value

UNFILTERED CONTENT VETTED & APPROVEDSpans Casual and Formal

Tagging & Organizing Principles

STRUCTURED TAGGING(TAXONOMY)Span Of Metadata Intentionality

SOCIAL TAGGING(FOLKSONOMY)

LOW-COST CONTENTLESS ACCESSIBLE

HIGH-COST CONTENTMORE ACCESSIBLE

TYPE

OF

CON

TEN

T

Best PracticesBenchmarks

Approved MethodologiesMessage Text Discussion

Postings

External NewsInterim

Deliverables

Templates

Example Deliverables

Content Repositories


Content Continuum: Task context

Localized Application Generalized Application

• More focused use• Service line/ function

scope• “Employee desk” • “In the weeds” technical

problem solving

NARROW AUDIENCE BROAD AUDIENCESpans Narrow and Broad

Audiences and Application

GLOBALSpan of ConsumptionLOCAL

• General use• Organizational scope• “Headquarters lobby”• Higher-level messaging

and common processes

APPLICATION SPECIFIC:

ENTERPRISE WIDE:

TASK

OBJ

ECTI

VE Communicate policy

Common process

documentation

Firm-wide messagingTechnical assets SME asset

curation

Solve specific

problems Capability

development

Executive level vision

Reuse broadly applied assets

BU Level Assets


Content Continuum: Summary

Application Construct Less Structure More Structure

Nature of Process Chaotic Processes Controlled Processes

Knowledge Management Knowledge Creation Knowledge Reuse

Purpose/Application Problem Solving/Collaboration Accessing Information/Answering Questions

Span of Control My Sites Enterprise Publishing

Class of Tool Collaboration/Communication Workflow/Document Management

Information Construct Unfiltered Filtered

Cost Lower Cost Higher Cost

Value Lower Value Higher Value

Editing/Vetting Informal Formal

Tagging Folksonomy Taxonomy

Ease of Access Low High

Type of Content Messaging/Interim Deliverables Best Practices/Reference Materials

Task Construct Engagement Level Policy Level

Audience Narrow Broad

Application Local General

Context Function al areas Organization

Level of detail Technical / In depth Higher-level & common processes

Metaphor Employee desk Headquarters lobby


Information and Access are Heterogeneous

Search/Tagging/Taxonomy Integration Framework

Data Sources

Access Mechanisms

BI IntegrationAuto categorization/Clustering

Entity Extraction

Faceted Search

Semantic Search

Business Intelligence

Customer Relationship Mgt

Document repositories

Custom databases and applications

Intranets/web pages

Product Lifecycle ManagementDigital Asset Management

Data WarehousesMessaging

ERP Systems

Ontology Navigation


Search as Recommendation Engine

Some segmentation concepts based on work of Vladimir Dimitroff

Simple Attribute Models• Few variables• Unambiguous• Objective

Less ComplexBased on empirical understandingEasier to model

More ComplexBased on probabilities Learning algorithms

Sophisticated Attribute Models• Multiple variables• Potential ambiguity • Subjective – requires

knowledge of domain and experience with behaviors

Latent Attribute Models• Many variables• Patterns emergent• Dependent on probabilities

“Black boxes”“Secret sauce” “Latent Dirichlet Allocation”“LDA”

Subjective Attribute Models• Based on judgment of

modeler• Greater ambiguity • More difficult to validate

Matching algorithmsQuery across data


Poll Question #1Where are you in the maturity model?


A. CrawlB. WalkC. RunD. Sub-orbital flightE. Light-speed

Where are you in the maturity model?


Jeff Fried - Biography

Jeff FriedCTO, BA Insight

Jeff.fried@bainsight@jefffried

Longtime Search Nerd• CTO, BA Insight

• Senior PM, Microsoft

• VP, FAST

• SVP, LingoMotors

Passionate About• Search

• SharePoint

• Search-driven applications

• Information Strategy

BlogDoMoreWithSearch.com

Technet Column“A View from the Crawlspace”

[email protected]

Search is a “Wicked Problem”

Wicked Problems are Problems Worth Solving

A wicked problem is a problem that is difficult or impossible to solve because of incomplete, contradictory, and changing requirements that are often difficult to recognize.

25

26

Technology Expectations

End-User Expectations

Consumerization of IT & Intuitive User Experience

Commoditization of Search Engines


Poll Question #2What are the biggest challenges you are facing related to search in your organization?


A. Finding and implementing the right search technologyB. Executing the right content processesC. Managing and monitoring governanceD. Marshaling the correct resources / staff /skills

What are the biggest challenges you are facing related to search in your organization?


• 16 years with Ernst & Young• Leader of the Search Services Team, Global

Markets – EY Knowledge, which is responsible for the EY Home Page intranet search and SharePoint search– Sets strategy for Enterprise Search which is to

leverage the new SharePoint 2013 environment as the Enterprise Search environment, while providing support for the existing environment

– Manages team activities in support of search performance analysis, tuning and other issues as well as content findability and content gap identification

Ed Dale - Biography

Ed DaleSearch Services Manager

Ernst & Young

Search is workEarley Executive Roundtable – April 2016Ed Dale

Page 33

The context for knowledge: our organization

EY is an organization of member firms operating in 150 countries.► We collaborate globally to offer audit, tax, transaction and advisory services.► Each service line has a wide, diverse range of business units and offerings.► Our organization is constantly growing and evolving.

We compete in a market where insights are the product:knowledge is and will be a key differentiator.

Page 34

The context for knowledge: our people

Our 212,000 people are our greatest asset. ► Their collective intelligence drives a client experience that is connected, responsive and

insightful.► They could be working from any site in any location.► We have a large population of Millennials accustomed to being self-sufficient through the

internet and connected by social networks.

We must be able to connect people to each other and to the best of EY’s knowledge anytime, anywhere.

Page 35

My perspective

► One consistent theme► The process to improve enterprise search is:

► Identify the correct content► Measure how well search returns that content► Tune the search engine to return that content better► Measure the change► Repeat

► Technology makes the work easier, but does not replace it► Interesting trends in search technology are ones that make the work easier

► Graph search – adds information to relevancy► About the person► About their actions

► Data lake simulation


Poll Question #3How do you measure the effectiveness and quality of your search solution?


A. We are not measuringB. We are reviewing feedback from usersC. We are using analytics to manually monitor

and inform continuous improvementD. We are dynamically targeting and measuring

personalized relevancy based on multiple sources of evidence

How do you measure the effectiveness and quality of your search solution?


Panel Discussion


Roundtable Discussion

Ed DaleSearch Services

Manager Ernst & Young

Dino EliopulosManaging Director Earley Information

Science

Seth EarleyCEO

Earley Information Science

Jeff FriedCTO

BA Insight


Center of Excellence Model for Enterprise Search

1 Evaluate Current State

Envision Future State

Determine Gaps

Prioritize Projects

Create Roadmaps

Assessment Search and Findability Strategy

Education and Knowledge Transfer

Research & Discovery (phase two)

2 3 4 5

6 7 8 9 10

11 12

Research & Discovery (phase one)

InteractionDesign

Requirements Definition

Content Analysis

& Modeling

Process Analysis &

Improvement

User Analysis

& Modeling

User TypesAudience ProfilesPersonasPersonalization

User ScenariosTask AnalysisUse Case Definitions

“who”Content ProfilesMetadata SchemasTaxonomyInformation Lifecycle

“why” “what”TechnologySolution ArchitectureNavigational ModelsWireframes

“how”Test &

Validate

Future State

Search Program Governance Strategic Advisory ProgramSt

rate

gy &

Visi

onDe

sign

& D

evel

opM

aint

ain

Enha

nce

and

Evol

ve


Suggested ResourcesGroups

Enterprise Search Professionals on LinkedInhttps://www.linkedin.com/groups/161594

Enterprise Search on LinkedInhttps://www.linkedin.com/groups/1812889

Enterprise Search Products & Services on LinkedInhttps://www.linkedin.com/groups/2638369

BooksEnterprise Search by Martin Whitehttp://www.amazon.com/Enterprise-Search-Enhancing-Business-Performance/dp/1491915536/ref=sr_1_1?s=books&ie=UTF8&qid=1461164050&sr=1-1&keywords=enterprise+search

Relevant Search by Doug Turnbull & John Berrymanhttps://www.manning.com/books/relevant-search

Search User Interfaces by Marti Hearsthttp://www.amazon.com/Search-User-Interfaces-Marti-Hearst/dp/0521113792/ref=sr_1_1?s=books&ie=UTF8&qid=1461164235&sr=1-1&keywords=search+marti

Search Patterns by Peter Morville & Jeffery Callenderhttp://www.amazon.com/Search-Patterns-Discovery-Peter-Morville/dp/0596802277/ref=sr_1_2?s=books&ie=UTF8&qid=1461164235&sr=1-2&keywords=search+marti

Other Resources

IDC Case Study on Knowledge Sharing and Reusehttp://www.earley.com/sites/default/files/IDC_Case-study_Applied-Materials_2014-06-04.pdf

Six Critical Success Factors for SharePoint Enterprise Content Management (ECM) Implementations (white paper)http://info.earley.com/6-critical-success-factors-sharepoint-ecm-implementation-whitepaper

High Impact Solutions for Solving Content Chaos (recorded webcast)http://www.earley.com/training-webinars/high-impact-solutions-solving-content-chaos

Making Intelligent Virtual Assistants a Realityhttp://info.earley.com/make-intelligent-virtual-assistant-reality-whitepaper

https://www.linkedin.com/groups/161594



http://www.amazon.com/Enterprise-Search-Enhancing-Business-Performance/dp/1491915536/ref=sr_1_1?s=books&ie=UTF8&qid=1461164050&sr=1-1&keywords=enterprise+search

https://www.manning.com/books/relevant-search

http://www.amazon.com/Search-User-Interfaces-Marti-Hearst/dp/0521113792/ref=sr_1_1?s=books&ie=UTF8&qid=1461164235&sr=1-1&keywords=search+marti

http://www.amazon.com/Search-Patterns-Discovery-Peter-Morville/dp/0596802277/ref=sr_1_2?s=books&ie=UTF8&qid=1461164235&sr=1-2&keywords=search+marti

http://www.earley.com/sites/default/files/IDC_Case-study_Applied-Materials_2014-06-04.pdf

http://info.earley.com/6-critical-success-factors-sharepoint-ecm-implementation-whitepaper

http://www.earley.com/training-webinars/high-impact-solutions-solving-content-chaos

http://info.earley.com/make-intelligent-virtual-assistant-reality-whitepaper


Earley Information Science (EIS)

Information Architects for the Digital Age

Founded – 1994 Headquarters – Boston, MA

www.earley.com

For more info contact:

[email protected]

[email protected]

Thanks to our Sponsors

Next Roundtable topicMay 25th - Predictive Analytics, AI and the Promise of Personalization