Challenges in building and using aLifetime Personal Information

Storebased on MyLifeBits

Gordon BellGordon BellAccelerating Change Accelerating Change ── 6 November 2004 6 November 2004

The 1 TB Life

1TB gives you 65+ years of:1TB gives you 65+ years of: 100 email messages a day (5KB each)100 email messages a day (5KB each) 100 web pages day (50KB each)100 web pages day (50KB each) 5 scanned pages a day (100KB each)5 scanned pages a day (100KB each) 1 book every 10 days (1 MB each)1 book every 10 days (1 MB each) 10 photos per day (400 KB JPEG each)10 photos per day (400 KB JPEG each) 8 hours per day of sound - e.g. telephone,8 hours per day of sound - e.g. telephone,

voice annotations, and meeting recordings (8 Kb/s)voice annotations, and meeting recordings (8 Kb/s) 1 new music CD every 10 days (45 min each at 128 Kb/s)1 new music CD every 10 days (45 min each at 128 Kb/s)

It will take you 5 years to fill up your 80 GB driveIt will take you 5 years to fill up your 80 GB drive Want video? Buy more cheap drives (1 TB/year lets Want video? Buy more cheap drives (1 TB/year lets

you record 4 hours/day of 1.5 Mb/s video)you record 4 hours/day of 1.5 Mb/s video)

Everything goes in a database

You need all the features of a databaseYou need all the features of a database(Consistency, Indexing, Pivoting, Queries, Speed/scalability, Backup, (Consistency, Indexing, Pivoting, Queries, Speed/scalability, Backup, replication)replication)

If you don’t use one, you will find yourself creating one!If you don’t use one, you will find yourself creating one! Files as blobs, also sync with file system for legacy appsFiles as blobs, also sync with file system for legacy apps

SQLSQL

MyLifeBits Software

MyLifeBits store

database

Voice Voice annotation annotation tooltool

Text Text annotation annotation tooltool

Telephone Telephone capture toolcapture tool

TV capture TV capture tooltool

TV EPG TV EPG download download tooltool

Radio Radio capture capture & EPG& EPG

PocketPC PocketPC transfer transfer tooltool

PocketRadio PocketRadio playerplayer

Import filesImport files

MyLifeBits MyLifeBits ShellShell

files

Legacy Legacy applicationsapplications

Browser Browser tooltool

InternetInternet

IM captureIM capture

MAPI MAPI interfaceinterface

Legacy Legacy email clientemail client

GPS import & GPS import & Map displayMap display

SenseCamSenseCam

Screen saverScreen saver

MemexAs We May Think, Vannevar Bush, 1945

““A memex is a device in which an individual stores all A memex is a device in which an individual stores all his books, records, and communications, and which his books, records, and communications, and which is mechanized so that it may be consulted with is mechanized so that it may be consulted with exceeding speed and flexibility”exceeding speed and flexibility”

Full-text search, text & audio annotations, and Full-text search, text & audio annotations, and hyperlinkshyperlinks

I am data

The guinea pig

Gordon Bell is digitizing his lifeGordon Bell is digitizing his life Has now scanned virtually all:Has now scanned virtually all:

Books written (and read when possible)Books written (and read when possible) Personal documents (correspondence including memos and email, Personal documents (correspondence including memos and email,

bills, legal documents, papers written, …)bills, legal documents, papers written, …) PhotosPhotos Posters, paintings, photo of things (artifacts, …medals, plaques)Posters, paintings, photo of things (artifacts, …medals, plaques) Home movies and videosHome movies and videos CD collectionCD collection And, of course, all PC filesAnd, of course, all PC files

Now recording: phone, radio, TV (movies), web pages… Now recording: phone, radio, TV (movies), web pages… conversations and meetings to comeconversations and meetings to come

Paperless throughout 2002. 12” scanned, 12’ discardedPaperless throughout 2002. 12” scanned, 12’ discarded.. Only 44 GB, incl. 10 wma, 14 SQL!!! Video: o(100) + 500 movOnly 44 GB, incl. 10 wma, 14 SQL!!! Video: o(100) + 500 mov

Capture and encoding

I mean everything

50+ year old newspaper clippings

400 year old books

O(100s) tapes from videotape “black hole”

Personal LifeLog Applications

Conservator

Baby Book

Companion Caretaker

Babysitter

AdvisorMentorTutor

Autobiography

Photo Album

PersonalAssistant

Diary/Journal

Biography

Medical Manager

ExecutorObituary

Others Self

Assistant for Elderly

Application controlled by:

Oth

ers

Sel

fA

pp

lica

tio

n u

sed

by:

Personal Proxy

Parole Officer

Pers Flight Recorder

Meeting Prep

Captain’s Log

Trustee

Financial Manager

Personal Search is notProfessional or Web search

System sees every entry & accessSystem sees every entry & accessEverything, not just a professional life Everything, not just a professional life Limited to SIS, not an infinite amount, Limited to SIS, not an infinite amount,

covers a profession & personal lifecovers a profession & personal life

Web as seen by search engines

MyLifeBits

Knowledge breadth e.g. Dewey classification

Depth e.g. information item types & coverage

Professional user

Why bother? ..some reasons Technologist: “we can” an opportunity e.g. 1 TB disksTechnologist: “we can” an opportunity e.g. 1 TB disks For all of us with new media: a need e.g. jpg. Mp3For all of us with new media: a need e.g. jpg. Mp3 Environmentalist: eliminates “atoms” (paper, CDs…)Environmentalist: eliminates “atoms” (paper, CDs…) For business--memory enhancement & faster search:For business--memory enhancement & faster search:

Let content analysis and data mining discover trends Let content analysis and data mining discover trends and correlations in our lives…that even we don’t know.and correlations in our lives…that even we don’t know.

Business: It costs more to delete than it costs to storeBusiness: It costs more to delete than it costs to store Preservationist: decays or disappears unless its savedPreservationist: decays or disappears unless its saved For the human pack rat: “I may need it some day.” For the human pack rat: “I may need it some day.” For posterity and nostalgia: “Maybe others will want it.”For posterity and nostalgia: “Maybe others will want it.” Stories and ambience: basis for creating contentStories and ambience: basis for creating content For the aging & failed memory: surrogate memoryFor the aging & failed memory: surrogate memory

So you’ve got it – now what do you do with it?

““A record if it is to be useful … must be A record if it is to be useful … must be continuously extended, it must be stored, and continuously extended, it must be stored, and above all it must be consulted” above all it must be consulted”

““The difficulty seems to be, not so much that The difficulty seems to be, not so much that we publish unduly … but rather that we publish unduly … but rather that publication has been extended far beyond our publication has been extended far beyond our present ability to make real use of the record” present ability to make real use of the record”

- Vannevar Bush- Vannevar Bush

Using my life bits: beyond folders

#1: Folders#1: Folders

One item. One place.One item. One place.

It worked for 1000s of years.It worked for 1000s of years.

My docs and archive

S

Self

EE

X- Employer

EmployerEmployer

X-EmployerProjectProject

ProjectProject

Employer

Library/file cab

Library/file cab

Library/file cab

Library/file cab

Library/file cab

Library/file cab

Active Employer

Library/file cab

Library/file cabLibrary/file cab

<1995 Library/file cabLibrary/file cab

Project

BusinessInvests, family $s, & Legal

Personal, including Medical

Library/file cab

Freedom from hierarchyc:\my documents\talks\MyLifeBits.pptc:\my documents\talks\MyLifeBits.ppt

ID=location=organization=display stringID=location=organization=display stringDon’t make me invent unique namesDon’t make me invent unique namesDon’t make me file everythingDon’t make me file everythingOr let me pick multiple foldersOr let me pick multiple folders

Using my life bits: easily adding valuable content

#2: Text annotations#2: Text annotations

Making bits more valuable and retrievable. Making bits more valuable and retrievable.

“Its just bits until it is annotated”

Getting the user to tell a story is the ultimate in media value

A story is a “layout” in time and spaceA story is a “layout” in time and space Most valuable content (by selection, and by being well annotated)Most valuable content (by selection, and by being well annotated) Stories must include links to any media they use (for future navigation/search – Stories must include links to any media they use (for future navigation/search –

“transclusion”).“transclusion”). Cf: MovieMaker; Creative Memories PhotoAlbumsCf: MovieMaker; Creative Memories PhotoAlbums

Dapeng was an Dapeng was an intern at BARC intern at BARC for the summer for the summer of 2000of 2000

We took him to We took him to lunch at our lunch at our favorite Dim Sum favorite Dim Sum place to say place to say farewellfarewell

At table L-R: Dapeng, Gordon, Tom, Jim, Don, At table L-R: Dapeng, Gordon, Tom, Jim, Don, Vicky, Patrick, JimVicky, Patrick, Jim

Annotation like this…

VoiceAnnotation

Annotation when you feel like it, how you feel like it

Screensaver is the killer app!Screensaver is the killer app!

Using my life bits:the value of time & time posts

#3: “I remember when…”#3: “I remember when…”

The 1The 1stst or 2 or 2ndnd most important retrieval handle. most important retrieval handle.

MyLifeBits time overlap

MyLifeBits on-the-fly time clustering

MSR Next Media Team

M Stewart Lifeline v2Mark Stewart’s Lifeline

Copyright Mark Stewart, 2004

190019101920193019401950196019701980199020002010

Chester Bell

Lola Bell

Gordon Bell

Sharon (Smith)

Kirksville, MO

M.I.T.

U. of N.S.W.

Gwen Druyor Bell

Brigham (son)

Fiona Bell

Bridget Bell

Laura (daughter)

Kolbe Schultz

Stryker Schultz

Sheridan Forbes

M.I.T. Speech Lab

Digital (DEC)

CMU

Encore

NSF

Ardent

Bell Ltd.

Microsoft Res.

Computer Museum

F: father

F: mother

F: self

F: Sister

Education

Education

Education

F: spouse

F: son

F: grandChild

F: grandChild

F: daughter

F: grandchild

F: grandchild

F: Significant Other

W/Education

Work

Work

Work

Work

Work

Work

Work

Organization

Using my life bits:Where, an essential attribute

#4: I remember where#4: I remember where

Just essentialJust essential..

Using my life bits: pivoting on data to aid recall

#5: Relationships (links)#5: Relationships (links)

Using something near ‘it”, to find “it”.Using something near ‘it”, to find “it”.

MyLifeBits Entities & Links

AnnotatesAnnotates

Caller in Phone CallCaller in Phone Call

Photo of EventPhoto of Event

TranscludesTranscludes

PhotoFinder - Schneiderman and Kang

Using my life bits:never enough meta-data …

but, can you afford it?b

#6: more meta-data (properties)#6: more meta-data (properties)

I remember something about the contentI remember something about the content

(understanding a person’s work)(understanding a person’s work)

Lederberg Finder page

Dublin core of a given item

Using my life bits:classification of everything

#7: classification#7: classification

Is any gain from non-automated classification Is any gain from non-automated classification worth the cost and pain?worth the cost and pain?

Is traditional classification required?

……at OCLC there was unanimous agreement at OCLC there was unanimous agreement among faculty and participants thatamong faculty and participants that

“access to electronic resources “access to electronic resources requires controlled vocabulary and requires controlled vocabulary and classification”classification”

OCLC Institute, “Knowledge Access Management: Tools OCLC Institute, “Knowledge Access Management: Tools and Concepts for Next Generation Catalogers”, 17-19 and Concepts for Next Generation Catalogers”, 17-19 November 1997, Dublin, Ohio.November 1997, Dublin, Ohio.

““I have watched as hundreds of I have watched as hundreds of millions of dollars have been millions of dollars have been invested to re-invent the wheel - invested to re-invent the wheel - often badly.”often badly.”

-Marcia Bates-Marcia Bates

www.alberteinstein.info

Professional Life:

Organizations

Administrivia

Projects

Library

Lederberg papers official reports

Number of document segments

Lederberg Artifact types Abstracts Agendas not Announcements m; Application forms Articles m Autobiographies m Bibliographies m Biographies m Brochures m Certificates m Correspondence m Diaries m Drafts (documents) Drawings m Electronic images m Essays m Eulogies Excerpts Grant proposals Interviews m Invitations

Laboratory notebooks m Laboratory notes Lecture notes Lectures m Legal documents m Legislative records Lists Manifestoes Memoirs m Minutes Monographs m Narratives Newsletters Newspaper columns m Notebooks m Notes Obituaries Official reports Oral histories m Petitions Photographic prints m

Press releases mProcedures Proceedings mPrograms mProposals mQuestionnaires Reminiscences Reports mResolutions Resumes Reviews mSchool records Speeches mSummaries Tables (documents) Technical reports mTranscripts mTypescripts Video recordings m

Species: Animals: Chordata: Vertebrata: bony fish

Computer structures: digital computer: minicomputer (refined: Digital Equipment Corp.)

Computer structures taxonomy: computers

Classification wish list Download classifications rather than build themDownload classifications rather than build them Definitions & synonyms should help find what I wantDefinitions & synonyms should help find what I want Today it is too expensive to manually classify my Today it is too expensive to manually classify my

scanned paper. E.g. “right time” meta-data is critical!scanned paper. E.g. “right time” meta-data is critical! Next year we hope “the system” can classify papers Next year we hope “the system” can classify papers

and other documents e.g. billsand other documents e.g. bills In 10 years we expect all documents to appear In 10 years we expect all documents to appear

electronically & classified electronically & classified with a little help from mewith a little help from me

Using my life bits:Ontologies…

useful? or fool’s errand?#8: “ontology”???#8: “ontology”???

““Succumbing to the ‘ontology’ fallacy”Succumbing to the ‘ontology’ fallacy”-Bates-Bates

MyLifeBits: Some Lives(t) PersonalPersonal

Parents, children, grandkidsParents, children, grandkids CGB himselfCGB himself GKBGKB SSFSSF Close friendsClose friends

GB $s; Legal entitiesGB $s; Legal entities Personal incl. several legal Personal incl. several legal

structuresstructures Properties: autos, real estate,Properties: autos, real estate, Investments & contractsInvestments & contracts

Past prof. companies/organiz’nsPast prof. companies/organiz’ns DECDEC Carnegie-Mellon U.Carnegie-Mellon U. DEC, NSF, Encore, Ardent, DEC, NSF, Encore, Ardent,

Me Inc., Bell-MasonMe Inc., Bell-Mason

Bell-Mason DirectorBell-Mason Director Diamond & Vanguard Brds.Diamond & Vanguard Brds. Startups & boardsStartups & boards

CGB@ MicrosoftCGB@ Microsoft MLBMLB ClustersClusters TelepresenceTelepresence WWW presenceWWW presence

Computer History MuseumComputer History Museum BOD memberBOD member Fund-raisingFund-raising CyberMuseumCyberMuseum

190019101920193019401950196019701980199020002010

Chester Bell

Lola Bell

Gordon Bell

Sharon (Smith)

Kirksville, MO

M.I.T.

U. of N.S.W.

Gwen Druyor Bell

Brigham (son)

Fiona Bell

Bridget Bell

Laura (daughter)

Kolbe Schultz

Stryker Schultz

Sheridan Forbes

M.I.T. Speech Lab

Digital (DEC)

CMU

Encore

NSF

Ardent

Bell Ltd.

Microsoft Res.

Computer Museum

F: father

F: mother

F: self

F: Sister

Education

Education

Education

F: spouse

F: son

F: grandChild

F: grandChild

F: daughter

F: grandchild

F: grandchild

F: Significant Other

W/Education

Work

Work

Work

Work

Work

Work

Work

Organization

Using my life bits:Providing insight, including…Where did I spend my time? What has been by output?

#9: logging & reports#9: logging & reports

Interface to xls

TV Usage

Using my life bits:Recording everything!

#10: CARPE#10: CARPEContinuous archival recording of Continuous archival recording of

personal experiencespersonal experiences

The A/V/real time data Future: new capture modes/devices

SenseCam

Deja View

Body Media

Quindi

Sensecam &Interactive jewellery

Open Problems

The Agenda for the Tbyte(s), Lifetime, PC:The killer app after office and mail.searching

1.1. Guarantee that data will live forever! “dear appy” problemGuarantee that data will live forever! “dear appy” problem2.2. Cheap, easy, and data-rich (e.g. time, place) capture:Cheap, easy, and data-rich (e.g. time, place) capture:

GPS and time everywhereGPS and time everywherePaper capture has to be as easy as discarding (scanner/shredder)Paper capture has to be as easy as discarding (scanner/shredder)Personal meeting capture...Personal meeting capture...E-book…e-magazines & journals need to have critical mass! E-book…e-magazines & journals need to have critical mass! Telephony and audio capture with indexingTelephony and audio capture with indexingMedia Center compatible for entertainment (photos, video, TV, radio)Media Center compatible for entertainment (photos, video, TV, radio)

3.3. Content analysis (critical for photo & video!); doable for text. Needs doing!Content analysis (critical for photo & video!); doable for text. Needs doing!4.4. Information control: privacy, security, expunge/deniability,… Information control: privacy, security, expunge/deniability,… 5.5. Having to be schizophrenic or have a lobotomy when leaving a “life”Having to be schizophrenic or have a lobotomy when leaving a “life”6.6. One One dbase for everything (articles, books, conversations, ... financial dbase for everything (articles, books, conversations, ... financial

transactions) …vs. long-term use of hierarchical files. transactions) …vs. long-term use of hierarchical files. Is dbase intuitive?Is dbase intuitive?7.7. Annotations/meta-information add every-increasing valueAnnotations/meta-information add every-increasing value

Easy annotation for aiding search and Easy annotation for aiding search and it becomes the contentit becomes the content8.8. Other “killer apps”: Alzheimer, immortality, surrogate memory?Other “killer apps”: Alzheimer, immortality, surrogate memory?9.9. GUI’s to improve use (e.g. time to learn, use, retention)GUI’s to improve use (e.g. time to learn, use, retention)

www.MyLifeBits.com

The “dear appy” problemDear Appy, Dear Appy,

How committed are you?How committed are you?Please come back to me.Please come back to me.Forever yours truly, Forever yours truly, Lost and forgotten dataLost and forgotten data

Who’s responsible?Who’s responsible?Media or 8 track cassette, 8” floppyMedia or 8 track cassette, 8” floppyEvolving platform, file, and databaseEvolving platform, file, and databaseEvolving, incompatible standards & formats for Evolving, incompatible standards & formats for legacy data that disregard ancestorslegacy data that disregard ancestorsEvolving and/or disappearing appsEvolving and/or disappearing apps   

A Storocratic Oath

1.1. Do no harm to datesDo no harm to dates(File creation, Photo taken)(File creation, Photo taken)

2.2. Do no harm to device created & Do no harm to device created & other meta-data.other meta-data.

• Camera data & location data are sacred.Camera data & location data are sacred.

3.3. Support & aid the creation of critical meta-Support & aid the creation of critical meta-data. data.

• When/how the user feels like itWhen/how the user feels like it• Auto-magically!Auto-magically!

4.4. Maintain user confidentialityMaintain user confidentiality

The killer app?? Input, File, Classify, and Find…Input, File, Classify, and Find…OperationalOperational

““Stuff I’ve Seen” (e.g. msg, name, paper, fact, Stuff I’ve Seen” (e.g. msg, name, paper, fact, birthday, phone call, photobirthday, phone call, photo

Observe every action…Observe every action…Time & motion (routing, communicating, Time & motion (routing, communicating,

scheduling … thinking)scheduling … thinking)Archival one’s self Archival one’s self

Finder aka Table of Contents aka Site Map Finder aka Table of Contents aka Site Map Story telling. Story telling.

Screen saver & personal ambienceScreen saver & personal ambience