View
26
Download
2
Category
Tags:
Preview:
DESCRIPTION
Presentation for Bay Area Search Meetup. Dynamic Ranked Retrieval. Surf Canyon Technology. Improves relevancy of organic results (up to 40% on top of Google) and revenue generated by sponsored links (over 4% ). - PowerPoint PPT Presentation
Citation preview
Presentationfor
Bay Area Search Meetup
Surf Canyon Technology
Improves relevancy of organic results (up to 40% on top of Google)and revenue generated by sponsored links (over 4%)
Dynamic Ranked Retrieval
• Harnesses real-time user behavior data to dynamically alter SERP “on the fly”
• Sits atop search engine as additional intelligent, post-processing layer
• Operates automatically in background without explicit user input• Provides “encouragement” and “entertainment” to search
experience• Two patents issued. Third pending.
2
Opportunity for Improvement
• Half of potential Web sales are lost because visitors simply can't find what they want. – Gartner
• According to Outsell, in a study indicating rising frustration with internet search, “in many instances the culprit is search failure due to irrelevant results.” – MediaPost, “Irrelevant Results Threaten Search”
• 79% of professionals feel that their queries are not always understood and only 10% find what they are looking for on the first attempt. – Convera
About 75% of the actions someone takes when they receive a search result page would indicate they didn't get the correct information, so they
either click back or retype their query – Mike Nichols, GM of search at Microsoft
People search for 11 minutes on average before finding what they're looking for, and half abandon searches without getting that far. –
Microsoft
The most popular feature in search is the back-click – Stefan Weitz, Director at Bing
3
Media Coverage“If you like the idea of more personalized Web searches... you might like Surf Canyon... Surf Canyon worked well for me with multiple search engines, retrieving data from result pages I likely wouldn't have opened... your days of futile Web searching are numbered.” – The Mossberg Solution
“After having spent time analyzing Surf Canyon’s product, we believe that, because of its ability to quickly and easily enable users to locate relevant information that might otherwise have been impossible to find, real-time personalization represents a significant next step in the evolution of search.” – Gene Munster, Managing Director at
“I’ve been using the plug-in for several weeks now and have quickly grown to love it. It’s one of my favorite types of technology - I downloaded it and forgot about it until it made my life easier.” – Carla Thompson, senior analyst at
“… what I like best about Surf Canyon is the interface. It doesn’t take you to another Web page. The recommended results just appear underneath the appropriate link. It feels more like an application than a cumbersome website where you have to click through multiple pages to find what you want. Google could take a lesson in interface design from Surf Canyon here with all of its Ajax goodness.”– Erick Schonfeld, co-editor at
“Surf Canyon is one of the best search tools I’ve seen for some time. You may quote me on that.” – Mark Gibbs, columnist at
4
Industry Recognition
Research paper entitled “Demonstration of Improved Search Result Relevancy Using Real-time Implicit Relevancy Feedback” was published by SIGIR in December 2009.
“Surf Canyon’s evaluation is done on an operational system with real users, and it gives a lot of insight into the benefits of real-time personalization. This is a great paper and one of the most interesting, original and relevant IR papers I have read in a while!”
– Thorsten Joachims, Associate Professor of Computer Science at Cornell University
5
Instantaneous Relevancy
Initial relevanciesas determined byunderlying searchengine
Searcher
clicks
result #5
Recalculatedrelevancies asdetermined usinginstantaneoususer intent model
#5 becomes most
relevant
(instantaneous
relevance)Some
relevancies
increase
Others
decrease
6
Illustration I: Ambiguity in Keyword Search
Searcher is
looking for
reviews and
clicks this link
7
Illustration II: Recommendations and Targeted Advertisements
Results
immediately
re-ranked
from
subsequent
pages
Sponsored links
take advantage of
improved targeting
8
Illustration III: Cumulative Instantaneous User Model
Subsequent pages
completely re-
ranked based on
instantaneous user
intent model
9
Demonstration
10
Adoption Statistics
Since launching Surf Canyon in February ’08, adoption has been strong and retention has been exceptional. Over 6 billion queries processed to date.
November ’09 – Surpassed 1 million queries a dayMay ’11 – Surpassed 10 million queries a day
% of all result clicks that are recommendations:• Google 8% ± 0.1%
• Yahoo! 8% ± 0.6%
• Bing 9% ± 0.8%
% of all result clicks after the 1st that are recommendations:• Google 18% ± 0.2%
• Yahoo! 23% ± 0.4%
• Bing 22% ± 1.8%
11
Relevancy Evaluation
P(IR) = 3.2 – (0.0025 ± 0.00101) * IRInstantaneous relevancy has virtually no dependence on initial rank
12
Relevancy Evaluation
Recommendations: ~80% improvement in click frequency with instantaneous user intent model. Consistent for all display positions
Recommended Link Test
13
Finding Stuff FasterData from millions of real user queries demonstrate the extent to which users
are now able to access relevant information.
Surf Canyon improves relevancy by up to 40% in blind page 2 test.
Result – Unambiguous absolute user preference for dynamic re-ranking using implicit relevance feedback
14
Results Vary by CategoryE-commerce and Reformulations are particularly amenable to real-time
implicit personalization.
15
Results Vary by Query LengthDifficult and heavy-recall queries, which correlate well with query length,
provide superior opportunities for real-time implicit personalization, although there is a positive benefit across all query lengths.
16
Results Vary by Result Set Homogeneity
Queries that offer a relatively homogeneous result set present fewer obvious opportunities for reformulation and, as a result, benefit more from real-time
implicit personalization.
17
Revenue Enhancement
When using instantaneous user intent model to personalize Sponsored Links:4.0% ± 4.5% (95% confidence interval, 7% two-tail t-test) revenue enhancement
Ads are targeted
using
instantaneous
user intent model
generated during
previous queries
Test conducted with lack of CPC information, no granular data and no opportunity to optimize the ad selection or display
This was the first and only test conducted with a 1st tier ad feed
Conclusion: Significant opportunity for even better results through collaboration
18
Business StatusMark Cramer, CEO – 20 years of in-depth technology industry knowledge and experience
at Hewlett-Packard, Keynote Systems, NexTag and NetGeo, where he was CEO. EE from MIT and MBA from Harvard.
Mike Wertheim, Chief Architect – 22 years of experience in software development including 9 years as an engineer at Sybase and 3 years as Chief Architect at Linkify. BS in Computer Engineering from CMU.
TimelineApr-06 IncorporationAug-06 Filed utility patent “Dynamic Search Engine Results Employing User Behavior”
May-07 Filed utility patent “Real Time Implicit User Modeling for Personalized Search”
Feb-08 Launched browser add-on
Apr-08 Closed $600k in seed fundingJan-09 Favorably reviewed in the Mossberg Solution column of the WSJ
Jun-09 Filed utility patent “Adaptive UI for Real-Time Relevance Feedback”Jul-09 Presented research at SIGIRAug-09 Launched search engine
Nov-09 Surpassed 1 million queries per day
May-11 Surpassed 10 million queries per dayJan-12 Two patents, “Dynamic Search…” and “Adaptive UI…,” issued
19
Thank you
Mark Cramermcramer@surfcanyon.com
@surfcanyon
Recommended