User Search Behaviors: Insights from Easy Search Transaction Log Analyses

William Mischo, Mary Schlembach, Jason Heldreth, Avinash KumarGrainger Engineering Library Information CenterUniversity of Illinois at Urbana-Champaign

LIS Methods Round TableApril14, 2015

Easy Search Custom Transaction Log Analysis

• Rich literature on TLA• We record user actions, system suggestions, search

reformulations, and clickthroughs in custom logs• 2010-2011 study of 1.4 million searches and 1.5 million

clickthroughs • 2014 study of 1 million searches, 1.1 million clickthroughs

over 10 month period May 2013 to March 2014• Initially supported by NSF and IMLS• Background: OPAC TLAs have provided ambiguous results;

web search engine behaviors different

Examples • Competent Jerks, Lovable Fools, and the Formation of Social Networks• The virtual supermarket: An innovative research tool to study consumer

food purchasing• Mothers and fathers : a study of the development and negotiation of

parental behaviour• Matouschek, Kellis, Serrano, Fersht Nature• Kinsella and Phillips 2005• Hemenway, D. (2010). Why We don't Spend Enough on Public Health. New

England Journal of Medicine, 362(18), 1657• university, need for money• J. Quant. Spectrosc. Radiat. Transfer.

2014 Transaction Log Analysis • Breakdown of searches from the main Gateway:– Easy Search and associated tabs: 902,420 76.5% • the gateway Easy Search default tab: (55.1%)

– advanced search: 56,170 4.8%– departmental library page searches: 158,750 13.5%– myeasysearch: 18,159 1.5%– search suggestion searches: 42,708 3.6%

• Total: 1178207

4

2014 Transaction Log Analysis • Gateway searches total: 902,420– Easy Search tab: 649,320– Books search tab: 106,375– Article tab: 76,375– Journal Title tab: 70,350

• 2007: 49.4% of all searches were known-item (KI)• 2011: 51.2% of searches in 54% of sessions were KI• 2014: 57.7% of searches were KI

5

2014 Transaction Log Analysis • Sample of 974,137 user-entered searches:– average words per query is 5.11 words– This is up from earlier studies: • 3.58 words per query in 2008; 3.76 words per query in

2009; 4.33 words per query in 2011• 93,642 one word, 158,054 greater than 8 words• 49.6% were 3 words or less, 29.4% are 6 words or more• longest search was 299 words

6

2014 Transaction Log Analysis • 20.98% of all searches in 29.13% of all sessions

used a search assistance suggestion or custom added search result link

• 57.2% of the 489,272 search sessions were single query sessions

• 6% of the sessions contain 6 or more queries, 1,157 sessions contain more than 20 queries, and 96 search sessions contain more than 50 queries

7

Search Assistance Features

• Spelling suggestions• Redo as author search prompt• Direct links to frequent searches and Libguides• Links to matching e-journal title(s)• Links to citation linker for full-text retrieval• Limit results to exact phrase, title word(s), or title

phrase• Dark background searches

Additional Search Assistance Features• Provide list of related terms• System performs additional phrase and title searches

within selected resources• Displays e-journal and database list results• Pass-through command language search• Ask-a-Librarian live link• Additional target searches are performed based on

user query• Links to e-book titles displayed on OPAC results line

Search Characteristics 2010

Characteristics of User Searches Percentage of all

Searches

Boolean Operators (“and”, “or”, “not”) entered13.03%

AND operator searches (the vast majority of ANDs appear as conjunctions in known titles)

12.76%

AND operator in all caps 0.41%OR operator-- lower and upper case entered

0.22%

NOT operator-- lower and upper case 0.13%Commas entered 6.03%Parentheses entered 0.88%Quotes (single and double) entered 3.95%Prepositions entered 27.26%“+” sign entered 0.42%Truncation symbol(*) entered 0.12%

Clickthroughs by Category 2010

• Clickthroughs into e-book content targets were 7.87% of all target clicks in 11.36% of all sessions

User Clickthroughs by Category Percentages

Article databases 47.65%

All books 33.01% (OPACs 26.01%, E-Books 7%)

Journal/Database titles 17.41%

Web search engines 0.72%

Newspapers and news sites 0.87%

Reference titles 0.35%

Primo Clickthrough Study May-June 2014 • Primo clicks 15,068 times between 05-01-2014 and

06-21-2014. Total of 147,326 clickthroughs during that time

• Looked at a sample of 478 of the 15,068 searches• 245 known-item searches (51.8%) and 228 topical

searches (48.2%)• Of the 245 known-item searches, 159 were judged

successful - they brought back the expected results in the Primo first page or as the top item

12

Primo Clickthrough Study (2)• Of the 159 ‘successful’ searches, all of them were also

successful in Scopus, Ebsco, CrossRef, WorldCat Discovery, Web of Knowledge, VuFind, IShare, Google, or Google Scholar searches

• Removing the Google results left 9 searches that were successful in Primo but not via Scopus, Ebsco, Web of Science, CrossRef, VuFind, IShare, or WorldCat Discovery. Removing CrossRef and WorldCat Discovery resulted in 35 searches successful in Primo but not successful in Scopus, Ebsco, Web of Science, VuFind, or IShare

13

Primo Clickthrough Study (3)• Found that CrossRef and WorldCat Discovery

good at retrieving successful search results for full citation searches

• 48 known-item searches that were unsuccessful in Primo but successful in one of the other Easy Search targets - Scopus, Ebsco, Web of Science, CrossRef, VuFind, IShare, WorldCat Discovery, or Google/GS

14

Primo Clickthrough Study (4)• Evaluated sample against Arizona State’s

Summon. In one case, a known-item search that was unsuccessful in Primo yielded a first page match in the AZU Summon. However, there were 50 known-item searches that were successful in our Primo that were unsuccessful (item not on top or first page) in the AZU Summon

15