Everything you always wanted to know about SharePoint search relevance …

Joris PoelmansSharePoint ArchitectRealDolmen

About meWork at RealDolmen

Principal consultant and product manager

SharePoint Server MVP since 2005

Active in Belux Information Worker User Group – www.biwug.be

Blog: http://jopx.blogspot.com

Twitter: @jopxtwits

3

AgendaChallenges with search

Configuration options to drive relevanceQuery rulesThesaurusDynamic reordering rulesAuthoritiesRanking models

5

Challenges with search – data complexity

What we give to search engines

What most search engines see1 2 3 4awaycomecontrastivecustomizeforfs14howoptimizingourrelevance (3)

shouldsimplesomess14techniquesview (2)whatwhenwithyou

Author: victor poznanskiTitle: conf relevancy sharepoint Date: 9.4.9Type: pptAuthority: 6Anchor: SPC presentation …

6

Challenges with search – understanding intent

7

What is a good search result?

All relevant documents (precision)

Complete answer (recall)

Sweet spotBalance between recall, precision and relevance

8

Microsoft goals for search in 2013

Find Answer ExtendFind what you’re looking for with intelligent results tailored to you

Get answers and take action with an experience that’s always a step ahead

Build smarter/no-code applications that can scale for any need or business requirement

9

Configuration in the Conceptual Relevance Flow

For all queries:

Authorities: Level 1: http://networksRanking model: {incorporate user ratings}

Query:BT networkquarterly report

Search Web Part

Query Processing Engine

Document Collection

Thesaurus: BT British TelecomBest bets: BT Network …/BTN.htm

(WORDS BT, British Telecom) AND(WORDS network, networks, networked) AND (WORDS quarterly, quarterlies) AND(WORDS report, reports, reported)

Mixed Results for:• BT Network best bet• BT network quarterly

report• BT network

ContentType=reports

Dynamic Reordering Rules: Quarterly Report {prefer docs from http://reports}

Query Rule: {Terms} Quarterly Report {Terms} ContentType=“reports”

10

Configuration options for tuning relevanceQuery dependent

• Thesaurus• Best bets/promoted

results• Dynamic reordering

rules• Query rules

Influence Dynamic Ranking

Query independent

• Authorities• Ranking models

Influence Static Ranking

AgendaChallenges with search

Configuration options to drive relevanceQuery rulesThesaurusDynamic reordering rulesAuthoritiesRanking models

Query rules drive relevanceQuery Conditions

What queries should be handled?

Query Actions

What happens when a rule matches?

PublishingOptions

Is the rule active and for how long?

Query rules: conditions and actions PHRASE-BASED MATCHES• Exact match, beginning or end• Ad-hoc or term store dictionary• Match a regex (advanced)

HISTORICAL (LOG-BASED) MATCHES• Is this query more likely aimed at

the following source…?• Do people mostly click on result

of the following type…?

CURATED (BEST BET)• Show a promoted result

REFORMULATE THE USER QUERY• Show a block of results• Replace the core results

with a different query

ActionsQuery Conditions

14

Query rules and result blocks

Query rulesDemo

AgendaChallenges with search

Configuration options to drive relevanceQuery rulesThesaurusDynamic reordering rulesAuthoritiesRanking models

16

Using a thesaurus to improve relevancyWhat is it?• Expands a query with synonyms• Specify synonyms for single or multiple words• Possible to specify language for a synonym to apply• Input from business is essential• Only one thesaurus possible per farm

17

Using a thesaurus to improve relevancyHow to use it?1. Create a .CSV filea.Key = term to trigger a synonym forb.Synonym = term that should be triggered by the keyc.Language = language that should be applied to the synonym

2. Import using SharePoint 2013 management shell

Import…http://technet.microsoft.com/en-us/library/jj219579.aspx

18

Using a thesaurus to improve relevancyThings to know• No export possible for a thesaurus, change requires a

redeploy• Entries are not case sensitive• Language is not mandatory• Synonyms are not given the same weight as the initial term

– it is possible to change the weight - http://bit.ly/1bcSSm4

19

Demo scenario – Importing thesaurus and modifying the weight

Using a thesaurusDemo

AgendaChallenges with search

Configuration options to drive relevanceQuery rulesThesaurusDynamic reordering rulesAuthoritiesRanking models

21

Dynamic reordering rules (1/3)What is it?Dynamic Rules let you reorder results according to a set of criteria• Just like query rules apply to user queries, dynamic rules apply to results

They are configured as part of a query• They translate into a special query syntax called XRANK

Result actions let you promote or demote results that match• In many cases, you can use this instead of Best Bets.

22

Dynamic reordering rules (2/3)How to configure it?

Result Conditions

Result Actions

Promote up or demote down

Manual condition looks like a query

23

Dynamic reordering rules (3/3)What is my score?

A promotion-to-top places any result at the top of the results

A demotion-to-bottom places any result at the bottom of the results

A promotion of 10 moves a mediocre relevant result to the top

A demotion of 10 places a mediocre revelant result under the most relevant results.

24

Boosting results

Dynamic reordering

Demo

AgendaChallenges with search

Configuration options to drive relevanceQuery rulesThesaurusDynamic reordering rulesAuthoritiesRanking models

26

Authorative sites

27

Authorities: Connected

1

0

1

1

2

4

3

2

∞

4

Setting an authority affects all sites connected through hyperlinks

Sites are weighted

by distance to the authority

28

Authorities: Connected

1

0

1

1

2

1

1

2

∞

0

Setting an authority can have unexpected side- effects

29

Authorities: Connected

1

0

1

1

2

1

-

2

∞

0

Non-authorities do not affect connected sites

AgendaChallenges with search

Configuration options to drive relevanceQuery rulesThesaurusDynamic reordering rulesAuthoritiesRanking models

31

How Can You Change Ranking?

ΣRank Score

ωpowerpoint

ωauthor

ωactivity

Title=“XBOX Monthly report”

Type=pptActivity=2M clicksAuthor=Ian RankemRating = 5

Title=“XBOX Monthly report”

Type=pptActivity=2M clicksAuthor=Ian RankemRating = 5

Query: Ian Monthly

ωrating

Transform (normalization)

Document(managed properties)

32

BM25F ranking model• SharePoint 2013 calculates relevance rank score using BM25F

(fielded version of BM25)

• Default model is a two-stage ranking model (second stage using a neural network)

• Install August 2013 CU to have an additional two-stage linear model and use this as start point

See Customizing ranking models to improve relevance in SharePoint 2013 - http://bit.ly/1cxf6A1Microsoft Research white paper - http://trec.nist.gov/pubs/trec14/papers/microsoft-cambridge.enterprise.pdf

33

Examing item rankingExplain rank page _layouts/15/explainrank.aspx requires two parameters:• q – which contains the query• d – which specifies the path of the item for which

you want to see rankdetailDownload search display template from http://bit.ly/IUaPiZ Mavention search ranking app - http://blog.mastykarz.nl/understanding-item-ranking-sharepoint-2013-search-mavention-explain-rank-app-sharepoint/

Important: If you get a 403 error add yourself on the Search Service Application with full control

34

Explainrank.aspx and Mavention item ranking app

Ranking modelsDemo

35

Configuring ranking modelsUse PowerShell to administer ranking models• Get-SPEnterpriseSearchRankingModel• New-SPEnterpriseSearchRankingModel• Remove-SPEnterpriseSearchRankingModel

Different steps:• Export existing model• Create managed properties you need to use in your model• Add managed property into your model with a specific weight and

normalization• Import model back again• Not possible anymore to use DefaultRankingModelID webpart property –

still looking for workaround

• Use querystring parameter rm=[GUID for ranking model] to test your

model

36

Using a custom ranking model

Ranking modelsDemo

37

Summary• SharePoint 2013 allows you to

easily capture intent and provide relevant results

• Use custom ranking models when you have considered or tried simpler options (Authorities, thesaurus, query rules, dynamic rules, ....)

• Search projects are iterative and are executed in multiple phases - use analytics reports, user feedback web parts, etc ...

• Extra tooling expected – check out https://channel9.msdn.com/Events/SharePoint-Conference/2012/SPC145

Thank You