Real-time Search 101 - SMX Toronto 2010 - Rob Garner - iCrossing

SMX Toronto

April 2010Rob GarnerStrategy Director

iCrossing

Real-time Search 101

What‟s the big deal about real-time?

• It‟s an entirely different animal in the search world– It‟s about both search, and discovery

– Just like image search, local search, news search, etc., real-time

search is an important segment of web search

• To be successful, real-time needs these two unique

aspects1) It must have a crawler-based algorithm

2) It must have a human-driven social layer

2

Trust and authority are also paramount to its success

There is more to real-time than just Tweets

• Consider the following elements of “in-the-moment” content:

– Forums

– Blog posts

– Comments on blogs

– Webpages

– Feeds

– Microformatted data

3

Real-Time Search Is Much Bigger Than Just Twitter , Search Insider

http://www.mediapost.com/publications/index.cfm?fa=Articles.showArticle&art_aid=113613

Twitter can‟t be beat in real-time streams, for news

“as-it-happens”

• More content and news

focused

• Highly relevant to PR and

brand monitoring

stakeholders

• News travels within seconds

of occurrence in many

cases

– Ex. Michael Jackson Death

– Earthquakes and other natural

disasters

– Localized emergencies and

crimes (building on fire)

4

But nobody beats Google at any other level of recency

• Combines both search

and social

• Robust crawl surfaces

both web chatter and

web assets within

seconds and minutes of

occurrence

5

The new „R‟ word in search: Recency

6Copyright iCrossing - Proprietary and Confidential

Recency in Real-Time Search: It's Much Broader Than Just 'Right Now' , Search Insider

http://www.mediapost.com/publications/index.cfm?fa=Articles.showArticle&art_aid=114558

Recency impacts a major segment of all search queries

• OneRiot study claims that up to 40% of all queries

have some sort of real-time intent. This includes:– Entertainment

– News and politics

– Current events

– Etc.

• OneRiot only allows content publishers to advertise

7

Copyright iCrossing - Proprietary and Confidential

The spectrum of recency and historical search

8

Search through history of all documents

Search history of last year last month right now

1) Old way

2) New way

Some keywords are better suited either to real-

time or shorter time frames, than historical search

• “Hotel deals”

• “Things to do tomorrow in Manhattan….”

• “Cowboys vs. Packers TV channel”

• “Coupons”

• Auction-related

• Keywords with seasonality

• Etc.

But can real-time predict…the future?

10

Source: Flickr“The future,

Conan?”

Examples of search recency

Ex. Search over “all time”

12

Ex. Search for current information, last 30

days – “hotel coupon”

13

Search for “hotel coupon”: Each result yielded a current and valid coupon

Ex. Search over the last few days

14

“SXSW” “SMX Munich”

Ex. Search Right Now

15

Social Relevancy: How engines are looking at Twitter

Consider the parallel aspects of websites, and Twitter accounts

• Domain authority : username authority

• Duplicate content : re-tweeplicate content

• Blogging freshness : content freshness : microblogging freshness

• Links : number of followers

• Quality of links : Quality of followers

• Inbound links vs. outbound links : ratio of followers in contrast to the number of

people followed

• Linking to bad : good neighborhoods

• Themes of Twitter users

16

What An Algorithmic Approach To Twitter's Social-Search Layer Might Look Like, Search Insider

http://www.mediapost.com/publications/index.cfm?fa=Articles.showArticle&art_aid=109358

A Big Week For Real-Time And Social Search, Search Insider

http://www.mediapost.com/publications/index.cfm?fa=Articles.showArticle&art_aid=116320

Real-time keyword planning: Google Trends

• Seasonality of many keyword topics is highly predictable, and

marketers can capitalize on spikes in search and social interest

by planning ahead

17

http://www.google.com/trends

Real-time keyword planning: Google Hot Trends

• Content publishers who have the ability to write and publish

fluidly can jump into the search and social mix by observing

rapidly trending topics in Google Hot trends (and also Twitter,

and OneRiot)

18

http://www.google.com/trends/hottrends

More examples of real-time search

One Riot

• Aggregates citations from

multiple social sources,

including Twitter

• Propagates URLs based on

social popularity from

trusted social nodes

• Offers paid advertising to

content providers only

http://www.oneriot.com

20

Bing‟s Twitter Engine

• A mix of tweets, and

popularly tweeted URLs

• Arguably has more of a

social approach to real-time

than Google

http://www.bing.com/twitter

21

Wowd

• Shows most popular pages that

users are viewing “right now”,

and over history

• Rich SERP allows rankings to

move up and down the page in

front of your eyes

• View popular pages, or watch

real-time results at the keyword

level

• Results created socially by its

users who are sharing their real-

time viewing habits in a cloud

http://www.wowd.com

22

Wowd: A Real-Time Search Engine That Will Make You Go 'Wow'

http://www.mediapost.com/publications/index.cfm?fa=Articles.showArticle&art_aid=115444

Google‟s “Latest”

• Currently no ads on “latest” real

time results

• Ads kick in with a recency of 24

hours or later

• Universal results kick in with a

recency of 24 hours or greater

• Can be sorted by either

relevance, or by date

• They have experimented with

forum content

• Filter by “all,” videos, updates,

news, blogs, books, and

discussion (forums and other

sites)

23

Google‟s “Latest” - More

• Streams as results become

available, as fast as the query

is popular

• De-duplicates tweets

• Relies on web crawl and spam

filtering

• Some spam still does get by

• Not as robust as Twitter for

breaking events, but

dominates in every other

aspect

24

Google Forum “Discussions” search

25

Google Alerts

26

Twitter Search

• Results shown only in

reverse-chronological order

• Does not de-duplicate tweets

• Spam is often a problem for

popular queries

• Does not store history over

two weeks

• One of the best ways on the

web to track and gauge real-

time events

27

Twitter Real-time discovery stream

• One of the best (if not *the*

best) sources for real-time

discovery

• Your Twitter stream is only as

good as the quality of people

and sources you follow

• Does not allow for back

searches of tweets in your

stream

28

THANK YOU

SMX Toronto

April 2010

Rob GarnerStrategy Director

iCrossing.com

Rob.Garner@icrossing.com

214.676.2089

@robgarner

www.facebook.com/garner

GreatFinds.iCrossing.com

Linkedin.com/in/robgarner

Media Post Search Insider Blog