Eyes on the World: Putting Your Photos in Context

Eyes on the World:Putting Your Photos in Context

Mor Naaman

Yahoo! Research Berkeley

http://zonetag.research.yahoo.com 2

Attraction Map of Paris

Stanley Milgram, 1976. Psychological Maps of Paris

http://zonetag.research.yahoo.com 3

Attraction Map of London

Jaffe et al, 2006.

http://zonetag.research.yahoo.com 4

Outline

• Wisdom of the tags – mining information from geo-tagged photos

• ZoneTag – your photos in context

• Why we tag – a user study of tagging

http://zonetag.research.yahoo.com 5

Alternative Outline

How?

What?

Why?

http://zonetag.research.yahoo.com 6

Information Overload?

Flickr “geotagged”

http://zonetag.research.yahoo.com 7

What can we derive?

• Dataset:

(photo_id, user_id, latitude, longitude)

(photo_id, tag)

Given all photos from a geographic region,- find a “representative” summary set- generate a meaningful visualization

http://zonetag.research.yahoo.com 8

Issues to Tackle

• Noisy data

• Photographer biases– In locations– In Tags

• Wrong data

Whatever, color, city, spectrum, santa barbara, california, usa, Lookatme, Herbert Bayer Chromatic Gate

http://zonetag.research.yahoo.com 9

Intuition

More “activity” in a certain location indicates importance of that location

Tag that are unique to a certain location can suggest importance of that location

http://zonetag.research.yahoo.com 10

(Very) Simple Example

http://zonetag.research.yahoo.com 11

Summary of San Francisco

Golden Gate Bridge TransAmerica

AT&T Baseball Park

Golden Gate Twin Peaks Golden Gate

Bay BridgeOcean Beach Chinatown

http://zonetag.research.yahoo.com 12

Tag Maps

http://zonetag.research.yahoo.com 13

Tag Maps

http://zonetag.research.yahoo.com 14

Outline

• Wisdom of the tags – mining information from geo-tagged photos

• ZoneTag – your photos in context

• Why we tag – a user study of tagging

http://zonetag.research.yahoo.com 15

Outline

How?

What?

Why?

http://zonetag.research.yahoo.com 16

Another Look at the Tag Map

http://zonetag.research.yahoo.com 17

ZoneTag(?)

How would we:Create/store?Find?Share?Discover?

“Everything in the world exists to end up in a photograph” -- Susan Sontag

http://zonetag.research.yahoo.com 18

• Programmable

• Context-aware

• Network-connected

• Quality… it’s getting there

Why Cameraphones?

• Numbers, numbers…– 500,000,000

(Source: Future Image Inc.)

http://zonetag.research.yahoo.com 19

Current Mobile Experience

• Difficult to share (or even save!)

• Hard to find– No context– No semantic

information

http://zonetag.research.yahoo.com 20

ZoneTag Experience

• 2-click upload (same key!)

• Photo uploaded with location and time metadata

http://zonetag.research.yahoo.com 21

ZoneTag Experience

• Tagging made easy– Tag/annotate your photos from the phone

http://zonetag.research.yahoo.com 22

Where do locations come from?

• Bluetooth GPS (when available)

• User-contributed cell tower mapping

http://zonetag.research.yahoo.com 23

Where do tags come from?

• Tags I used in this context (`home’)

• Tags my friends used in this context (`Research Meeting’)

• Tags other people used in this context (`PARC’, ‘Mor Naaman’, `This talk sucks’)

http://zonetag.research.yahoo.com 24

Where do tags come from?

• Stuff around you: – Yahoo! Local – Upcoming.org (`Mor Naaman @ PARC’, `PARC

George E. Pake Auditorium’)

• Stuff from you (any RSS 2.0 feed):– Calendar (G, Upcoming.org, Kiko,…)– Favorite hangouts

http://zonetag.research.yahoo.com 25

Suggested Tags

“When I went to upload, there were already all these exotic tags like

"Bill Graham Civic

Auditorium" and "Bob dylan

live" available...it

sure was convenient to just select and go…”

http://zonetag.research.yahoo.com 26

Where do tags go?

• Back to the original RSS items– Upcoming.org

• Action Tags– Trigger a call to a web service

• With parameters

– Command line - from your phone

Rotate:right Email:dad Group:zonetag Scanr:document

http://zonetag.research.yahoo.com 27

I’m Too Lazy (you’re not alone)

• Tagging is the means, not the goal

• Benefits even if you never tagged a single image

(Bradley Horowitz, elatable.com)

(?)

(?)

(?)

http://zonetag.research.yahoo.com 28

Things We Learned

• “Best” feature: 2-click upload– Even users that often tag

• For some, last chance to annotate

• Mobile tagging - not just experts, not all experts…

http://zonetag.research.yahoo.com 29

Things We Learned

• Tagging behavior: – All over the place

– More than Shozu– Connected to privacy

The relative number of users for different ranges of tag usage (n=172)

0%

5%

10%

15%

20%

25%

30%

<0.5 <1 <1.5 <2 <2.5 <3 <3.5 <4 <4.5 <5 5+

http://zonetag.research.yahoo.com 30

Outline

• Wisdom of the tags – mining information from geo-tagged photos

• ZoneTag – your photos in context

• Why we tag – a user study of tagging

http://zonetag.research.yahoo.com 31

Outline

How?

What?

Why?

http://zonetag.research.yahoo.com 32

Why Tag?

• What are tags?

http://zonetag.research.yahoo.com 33

Why Tag?

Bay Bridge, Fog, [San Francisco, 94105]

http://zonetag.research.yahoo.com 34

Why Tag?

RedSox, Fenway, GreenMonster, [Boston, 02215], …

http://zonetag.research.yahoo.com 35

Why Tag?

Liz, sunbathing, sunny, sunglasses, concrete, [Palo Alto, 94306], …

http://zonetag.research.yahoo.com 36

Tag Affordances on Flickr

• Displayed next to photo

• Can be used to search:– Your own photos– Others’ photos– Public photos

http://zonetag.research.yahoo.com 37

Tags on ZoneTag

• By default, tags are kept from one photo to next

• Tags used in a location will be suggested in that location– To you– To others

• Immediate (i.e., during event)

• Place name tags added automatically

http://zonetag.research.yahoo.com 38

User Study

• 13 ZoneTag users (23-45, 9m, 4f)

• All “taggers” (no use to ask non-taggers why they tag)

• Structured interviews

http://zonetag.research.yahoo.com 39

Expected…

• Organization/retrieval

• Creator/synthesizer

• “Tagging for community”

http://zonetag.research.yahoo.com 40

Found…

• Motivation taxonomy• No “synthesizers”• Inspired by community

• Organization/retrieval• Creator/Synthesizer• “Tagging for community”X

http://zonetag.research.yahoo.com 41

Motivation Taxonomy

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

“If I tagged ahead of time I can go back and get all my pictures of [my children]

…”

“…I then think “well, maybe I should tag

this” so I can find it again

later”“I’m

obsessive-compulsive”

http://zonetag.research.yahoo.com 42

Motivation Taxonomy

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

“I want at least one hook of association in there that can help me reconstruct…”

http://zonetag.research.yahoo.com 43

Motivation Taxonomy

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

“I tag photos with what I think might

be interesting to other

people, stuff I think

people will like”

“I know that tagging can connect my photos to activities, and get more interest”

“I want to look at all

[my neighborhood’s] tags. … That’s

definitely a reason I’m

putting these tags in ”

“...at night, by the fires all around the market, ... at each word that one man says -- such as 'wolf', 'sister', 'hidden

treasure', 'battle', 'scabies', 'lovers' -- the others tell, each

one, his tale of wolves, sisters, treasures, scabies, lovers,

battles.” -- Italo Calvino,

Invisible Cities

http://zonetag.research.yahoo.com 44

Motivation Taxonomy

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

“…so that my friends can see what I am

up to” I can tell my mom [with the tag] “look, we

went to…”

“I left reviews of

places – like at the

airport, when my flight was delayed, I

tagged “Aloha Air sucks.”

http://zonetag.research.yahoo.com 45

Breakdown

Organization Communication

Self P2,P6,P10, P12, P3, P8, P11

P1, P6, P2

Social (friends/family)

P8, P6

P1, P4, P7, P11, P6, P8

(public) P3, P9, P13, P2, P4, P6, P7, P8, P11, P12

http://zonetag.research.yahoo.com 46

Suggested Tags

• Useful for entry, when they “work”– Critical mass needed– Better filtering

• Inspire and give example

• Overtagging may be issue

“Tag suggestions were huge for

me; they really cut down on typing”

“I thought: this is how I want it to work all the

time!”

“This person was in my phone for a

month - who is she???”

“I try to use as many

suggested tags that apply. … I also use it for auto-

completion – I type “s” to get San

Francisco”

http://zonetag.research.yahoo.com 47

Open Questions

• Those who don’t…

• Better filtering, tag suggestions

• Longer usage trends

• Effect of community on usage

• …

http://zonetag.research.yahoo.com 48

Final Notes

• All Flickr/ZT photos, Creative Commons or with author permission

• http://www.flickr.com/photos/frozenquack/106111967/• http://www.flickr.com/photos/bryce_edwards/272002447/• http://www.flickr.com/photos/plasticbag/188818779/• http://www.flickr.com/photos/groovymother/131591824/• http://www.flickr.com/photos/deaneckles/220482624/

http://zonetag.research.yahoo.com 49

ZoneTag For All

• Everything we can do, you can do (better?)– Cell ID translation– Suggested Tags– More…

• Use ZT-technology with your app!

• Email me

http://zonetag.research.yahoo.com 50

Thanks

With: Shane Ahern, Simon King, Rahul Nair, Nathan Good, Marc Davis, Morgan Ames, Dean Eckles, Alex Jaffe, Tamir Tassa

Download, sign up, get the full scoop:http://zonetag.research.yahoo.com

Become an intern, get involved:Email me.

Mor Naamanmor@yahoo-inc.com