moscow_developer_day

Brad FitzpatrickGoogle, Inc.2007-12-19

2

Opening up the Social Graph or…

“Interop: Past, Present, & Idyllic Future”

Google Code DayMoscow, 2007-12-18

Brad Fitzpatrick Google, Inc.2007-12-19

3

About the Speaker

Creator of …

• LiveJournal

– Blogging + social networking

– Granular privacy settings on everything

– RSS/Atom (publish + consume)

• Feed aggregator

– Open, documented APIs

• OpenID

– Decentralized identity protocol,

– No single point of failure (no company’s failure can stop it)

• Lots of backend infrastructure for high-performance & high-availability sites

– Memcached, MogileFS, Perlbal, djabberd (XMPP server), …

4

About the Speaker

Things that excite me…

• Decentralization

• Interop, Cooperation

• Open Protocols

• Social Networking

5

Things I want to discuss…

The Social Graph

• How are we all connected to each other?

Social Applications

• … doing useful things with the Social Graph

Interop

• One Social Graph, or thousands of disconnected ones? Both?

• How to write social applications? Where to run them?

Privacy

• Private identifiers, private relationships, personas, …

Where does the data live?

The bright future

• Glimmers of hope on the horizon

6

The Social Graph

How we’re all connected to each other

• There is only one real social graph in the world

• Each website currently just has their own instances of subsets of it

Directed graph

• Some sites just like to model it as symmetric (undirected) for various reasons (virality, less user explanation, less code complexity?)

“Giant Global Graph”, Tim Berners-Lee

• http://dig.csail.mit.edu/breadcrumbs/node/215

• GGG, from the inventor of WWW

• Links people + relationships, not documents (like WWW does)

Alice TomIs fascinated by

Is annoyed by

7

Social Applications

Message

Poke

Throw sheep

….

Dopplr.com:

• Define your travel dates & destinations,

• Then tells you which friends will be in same cities during your travels.

• It has to know your friends to do this

– This is why it’s a social application … it depends on the GGG.

– Should it ask you to tell it again all your friends? Or get it from elsewhere? Where?

8

Privacy, Personas, …

Personas

• Example:

– Work (Professional)

– Public Internet (don’t reveal much)

– Personal (for friends only)

• Sync accounts only within the same persona

• Don’t let one persona reveal another persona, unless desired.

– OK: public internet --> work (depending on user)

– Not OK: work --> public internet (again, depending on user)

9

Privacy, Personas, …

What can be private?

• Identifiers can be private

– When identifier is also contact information (email, IM, phone)

– So sharing your email address book as your public friends list… your won’t have many friends much longer. :-)

– You can reveal your own contact information:

• … you’re a plumber. 3am phone calls okay… extra money!

• And relationships, even without contact information, can be private.

– Perhaps you shouldn’t be friends with your wife & your mistress on a public social networking site. (at least with same persona :-))

10

Social Network Proliferation

My personal experience

• LiveJournal

– Only one I really used and kept up-to-date.

– All my friends were there.

– Life was simple.

• Friendster, MySpace, Facebook, Orkut, Hi5, LinkedIn, Tribe, Meetup, Twitter, Jaiku, Dodgeball, Pownce, Digg, …

– Fatigue. Too hard to keep up-to-date. Stress.

Where are my friends now?

• All over the place. Who knows. So many social networking sites. I have no accurate repository of relationships.

11

The Problems

Proliferation of non-interoperable social networking sites & social applications.

• Islands of friends

• Can’t add friends across islands

– Can’t even synchronize your friends on each island, should you want to

• Can’t “throw sheep across islands”

Hard to create social applications

• Roll your own? Use a platform? If so, which one?

• Where do you get the social graph?

– Ask users? Get it from elsewhere? Where?

12

The Problems

Users are frustrated.

• Sick of redefining their friends on each site.

• Sick of invite spam.

• Worried, confused about privacy & persona management.

Developers are frustrated.

• Hard-to-get, incomplete social graph data, even when users want it to be available.

– Ask users for it? That frustrates them.

– Tie yourself to a platform? Which? Why?

13

The Ideal

In the ideal, future(?) world,

• Social applications are easy to write

• The social graph is easy to access

– … but subject to users’ access controls,

– … and users are in control of their data, able to keep it globally updated.

• This all works with open protocols and no centralized provider

– Users can even run their own graph providers (similar to what OpenID enables today, for identity), or delegate to his/her trusted graph/“Address Book” provider(s).

14

We’re not there yet.

15

How do we get there?

Lot of little steps?

• Building blocks seem to be all maturing.

• Slowly converge on the ideal.

One big step by everybody at the same time?

• I doubt it.

16

Comparison to other social systems

Email

• Only brief period of time without interop, mostly in pre-networking days.

• Global identifiers: brad@______.___ (says which “brad”)

• Works beautifully with the exception of lack of sender’s identity

– SPF/DomainKeys aren’t widely used

– Spam would be a lot easier to fight if you knew who people were

17

Comparison to other social systems

Instant Messaging

• Historically a proprietary mess of non-interoperating walled gardens

– AOL can’t talk to Yahoo can’t talk to MSN can’t talk to ICQ

• Some business mergers & ad-hoc interop deals

– AOL and ICQ. MSN and Yahoo.

• Hacks to ease the pain of being on each service give illusion of interop:

– Gaim, Trillian, Miranda, AIM-in-GMail/GTalk

– Not real federation, still no global identifier. (what to put on business card?)

• Promise for the future: Jabber (XMPP). True federation.

– Earthlink, Google Talk, LiveJournal, SAPO.pt, ….

– Free federation (both libre & gratis) federation. No business deals. Anybody can play, just like email (except XMPP has authentication of peers)

– Global identifiers (brad@___.__)

18

Comparison to other social systems (table)

Email IM XMPP IM Social Networking

Interop Yes Kinda (biz deals, spotty federation)

Yes No

Peer authentication

Kinda (DK, SPF, …)

Yes Yes Not really (OpenID, kinda)

Global Identifiers

Yes Kinda. Mix of styles.

Yes Kinda (URLs?)

Hacks to ease multi-provider pain

Not needed

Yes (Trillian, Gaim, etc..)

Not needed

Few, uncommon, often banned.

# of providers tons handful relatively few

rapidly increasing number of them

19

How do we improve the situation?

Lot of sub-problems.

• Not one solution.

• Lots of little steps.

Let’s discuss some of them….

20

OpenSocial

Assumptions:

• There will be more than one social networking site.

– (there are already dozens)

• While many sites have niche-/market-/region-specific social apps, many social apps are common, useful on all sites.

• It should be easy to write portable social applications that plug into any site (container)

OpenSocial

• Write your social app in HTML+JavaScript, languages you already know, and deploy it in any OpenSocial container (site).

• No dependence on Google. It’s just a spec. Open Source reference implementation of container code provided, or make your own.

21

opensocial.* APIs

People

• information about individual people and their relationships to each other

Activities

• ability to post and view updates on what people are doing

Persistence

• a simple key-value data store to allow server-free stateful applications

22

opensocial.people.*

Where does the OpenSocial container get its view of the social graph from?

• That’s up to the container.

– Its own graph.

– Somebody else’s graph?

– The public subset of the “Giant Global Graph”, as crawled from public XFN and FOAF markup?

– ACL’d private user data from somewhere?

– Merged view of some/all of the above?

• Lot of interesting things happening in this space, all very new…

23

Interesting Things Lately…

Diso-project.org

• Distributed social networking:

• XFN, hCard, OpenID, OAuth, etc…

Plaxo.com, FriendFeed.com, Mugshot.org

• Aggregate from multiple accounts

SixApart’s http://updates.elsewhere.im/

• Realtime XML stream of public relationship changes

• Anybody can submit with OAuth

OSocial.net

• “a fake social network, a meta open social network mixing profiles from different social networks.”, uses OpenSocial, FOAF, XFN

24

Throwing Sheep Between Islands

Not many deep discussions yet, sadly.

• Just high-level “wouldn’t it be nice if….” thoughts.

XMPP has been proposed.

• Probably a good candidate

• Bootstrapping problem: need at least 1-2 largish providers supporting it, and/or many small providers?

Probably a ways off.

25

What can the developer community do now?

Throw sheep between islands?

• XMPP? Harder. Boot strapping problem. Ways off.

Sync friends between islands, then throw sheep at them on each island you visit?

• More viable short-term “solution”

– When users want to sync their accounts (each account may be a separate persona)

• But where to get the data? …

Publish it!

• Making it easier for users to get their data out makes them more confident to put data in & use your site.

26

Exporting public relationship data

FOAF (foaf-project.org)

• “Friend Of A Friend”

• XML RDF files describing profiles, friends, interests, schools, and SHA1_hex(“mailto:” + email_address). Extensible.

• Declared in HTML with:

– <link rel="meta" type="application/rdf+xml" title="FOAF" href=”…”>

• Supported by LiveJournal, Tribe.net, et al

XFN (microformats.org)

• Much lighter weight:

– <a href=“http://bob.com/” rel=“friend”>Bob</a>.

• Supported by tons of sites, more every day.

27

Who’s who? A persona’s many accounts.

Users can link their accounts together.

With XFN:

• <a href=“http://othersite.com/brad/” rel=“me”>My other site</a>.

With FOAF:

• foaf:mbox_sha1sum

• foaf:homePage, etc

28

A directed graph of “I claim that’s me!” edges

brad.livejournal.com myspace.com/bradfitz

bradfitz.com

me!

me!me!

29

Can’t trust one-way “me” claims!

30

Attacker


bradfitz.com

me!

me!me!

attacker.com

me!

31

From start node, only follow “me” edges forward…


bradfitz.com

me!

me!me!

attacker.com

me!

david.livejournal.com

sergey.com

friend

friend

32

Syncing

Example of something people have been talking about doing with this massive directed graph…

Idea:

• Cluster accounts in same persona (rel=“me” links, etc)

• Find “friend” links

• Find when two users are friends on site A, but not on site B

– Did they just not know about each other on site B?

33

Syncing


bradfitz.com

me!

me!me!

david.livejournal.com

myspace.com/david

friend missing friend

me!

me!

34

In Summary

Things currently not great,

• Still a lot of problems

But I’m optimistic:

• OpenID, OAuth, Yadis, XRDS

• OpenSocial

• FOAF, microformats (XFN, hCard)

• XMPP

• All the pieces are slowly coming together for some beautiful solutions

• Community seems eager to work together to glue all the parts together, fleshing out deficiencies

Brad FitzpatrickGoogle, [email protected]

Travel

moscow_developer_day