RETRIEVING RELEVANT AND INTERESTING TWEETS DURING LIVE TELEVISION BROADCASTSRianne Kaptein, Yi Zhu, Gijs Koot, Judith Redi and Omar Niamut | TNO & TU Delft

LIVE EVENT: VOLVO OCEAN RACE

2 | Retrieving relevant and interesting tweets during live television broadcasts

RELEVANT & INTERESTING TWEETS

3 | Retrieving relevant and interesting tweets during live television broadcasts

RESEARCH QUESTION

o How can we design an event profiler that generates a set of query terms to

retrieve relevant and interesting tweets during an event?

o Compared to following the event title / hashtag

o Not only consider relevancy, but also interestingness

ADAPTIVE EVENT PROFILER

Event Title

from TV guide

(Initial) Search

Keywords

Search keywords

to addRule based preprocessing, e.g. remove words like “live” and dates

Set of Twitter Search Results

Search / Stream search keywords using the Twitter API

Show tweets in display /Write tweets to database/

Relevance Feedback

Repeat regularly during event to pick up new keywords

ADAPTIVITY

To cope with the dynamically changing nature of Twitter we will use a moving

window for data collection

Each n minutes we analyse the last n minutes of activity on Twitter

To avoid topic drift, each iteration will be based on the initial search keywords,

all previous search keywords will be considered again

KEYWORD TYPES

Main keyword: Starting point, event title or hashtag

Relevant keywords: Tweets containing a relevant keyword are considered

relevant, and are shown to the user in the interface

Candidate keywords: Potentially relevant keywords, are followed during time

period t to see if they can be promoted to a relevant keyword in the next

timeperiod t+1

7 | Retrieving relevant and interesting tweets during live television broadcasts

ADAPTIVE EVENT PROFILER STEPS (PART 1)

1. In period t follow the main keywords and the relevant keywords to retrieve

a set of tweets.

2. From the set of tweets containing the main keywords, extract the top n

most frequent terms, hashtags and usernames. These terms are the new

candidate keywords.

3. In period t + 1 follow the main keywords, the relevant keywords and the top

n candidate keywords to retrieve a set of tweets.

4. For each keyword generate a Maximum Likelihood Estimation language

model based on the tweets that contain the keyword

5. For each candidate and relevant keywords, assess the similarity of its

language model and that of the main keywords using Kullback-Leibler

divergence.

ADAPTIVE EVENT PROFILER STEPS (PART 2)

6. Select the k keywords (either candidate or relevant) most similar to the

main keyword, and designate them as relevant keywords for the upcoming

period t + 2.

7. In period t + 2 follow the main keyword and the k relevant keywords to

select relevant tweets.

8. Go to 1.

EXPERIMENT

Event: Sochi Winter Olympics 2014

Broadcasted live on Dutch National Television

Sports event: Speed skating finals, popular sport in The Netherlands

10 | Retrieving relevant and interesting tweets during live television broadcasts

EXPERIMENT:SET-UP

2 Sessions with 10 participants

Participants watched 25 minute live broadcast.

3 keyword levels (high, middle, low)

Click to like a tweet or not

Post-experiment questionnaire to evaluate the relevancy and interestingness

of a subset of the tweets displayed

EXPERIMENT: USER INTERFACE

EXPERIMENT: RESULTS CLICKS

5.959 clicks

1.729 different tweets

Average of 300 clicks per user

41/44 tweets evaluated in questionnaire

30 tot 35% of the tweets were liked

13 | Retrieving relevant and interesting tweets during live television broadcasts

# tweets # positive fraction

Tweets containing main keyword

721 0,42

Tweets not containing main keyword

1008 0,32

EXPERIMENT: RESULTS CLICKS

Considerable individual differences

EXPERIMENT: RESULTS QUESTIONNAIRE

15 | Retrieving relevant and interesting tweets during live television broadcasts

EXPERIMENT: RESULTS QUESTIONNAIRE

Higly relevant terms:

#sochi2014

#10km

Schaatsen

Kleibeuker

Usernames skaters

Irrelevant terms:

Unrelated usernames

Relevant terms:

zilver

vrouwen

16 | Retrieving relevant and interesting tweets during live television broadcasts

EXPERIMENT:RESULTS QUESTIONNAIRE

IRRELEVANT TWEETS

Found keyword is used in a different context

Word can mean something in another language

Keyword is too general

18 | Retrieving relevant and interesting tweets during live television broadcasts

UNINTERESTING TWEETS

Simple statements like: “Watching ….”,

“Looking forward to ..”

Commercial tweets

Direct messages part of a (private) conversation

19 | Retrieving relevant and interesting tweets during live television broadcasts

INTERESTING TWEETS

Tweets containing a picture or movie

Tweets from participants and visitors of

the event

Inside information

Fun facts

20 | Retrieving relevant and interesting tweets during live television broadcasts

CONCLUSIONS

• Event profiler retrieves significantly more likeable tweets than following a

single keyword

• Without introducing too much noise.

• Relevant tweets are not necessarily interesting, but interesting tweets are

usually relevant.

THANK YOU FOR YOUR ATTENTION

QUESTIONS?