Upload
cathal-gurrin
View
677
Download
0
Embed Size (px)
Citation preview
LIFELOGGING A NEW ERA OF PERSONAL DATA
Dr. Cathal Gurrin (@cathal) lifelogger - researcher - educator
Dublin City University & Insight Centre for Data AnalyticsBiohackers Summit 2015
24th September 2015
Using mobile/wearable devices and information loggers to automatically record everything you see, hear, learn and experience. Creates a complete and accurate record of an individual - a Lifelog.
Challenge is to extract value from this new data.
Lifelogging
In the era of lifelogging, you will be able to summon up any memory or life experience…
It will change the way we work and learn, improve our health, change relationships…
It will change what it means to be human, and it is happening now. In fact, it is inevitable…
Sense and analyse factors of interest through numbers to gain
knowledge
Using knowledge for self-improvement
through experimentation Digitise as much as you can of
life experience… for many reasons, mostly unknown…
LifeloggingQuantifiedSelf
Biohacking
Positioning my Research
Lifelogging cangenerate thousands of
images per day, hours of audio/video
and tens of thousandsof sensor readings,
biometrics,EEG, communications,
interactions…
The challenge is to automatically analyse this data and make it useful for the individual.
Quantified Self
Enhanced Knowledge
Power to Change
Performance Enhancement
Data for Empowerment
New Insights
Population-wide studies
Healthcare Enhancement
Enhancing Human Memory
Upgraded Recall
Assistive Technologies
Enhanced Memory
New Interactions
Rich Sharing
Data Partners/Carers
Social Enhancements
Why? To provide knowledge to empower…
Quantified-Self Memory
Memory Enhancement
RECALL/RETRIEVAL
REFLECTION REMINISCENCE
Quantified-Self Analytics with Limitless
REFLECTION
A Search Engine for Life Experience. Never Forget.
RECALL/RETRIEVAL
Reliving Past Memories for Personal Uses or Sharing.
REMINISCENCE
The aim is to develop prototype memory upgrading software. An assistive technology that experiences what you experience and is always on and does
not need any user input, except queries.
Sensory(Memory(
Short-term(Memory(
Long-term(Memory(
Musical?( Explicit((conscious)(
Declara<ve((events(and(facts)(
Episodic((events(and(experiences)(
Seman<c((concepts(&(facts)(
Implicit((unconscious)(
Procedural(That means looking at human memory, and how it works…
Episodic Memory, Query Mechanisms, etc…
Automatically annotate, enrich, link and store for future search, retrieval and
access.
IndexPervasive access to
support Reminiscing,
Reflection and Retrieval of Experiences
InteractAutomatically sense using a small set of
wearable and informational
sensors.
SenseAutomatically
generate meaningful units of retrieval by modeling human
memory.
Segment
Four Core Components are Required to build a Lifelogging Platform
There are a lot of research challenges here, at every step.
But they are all needed to develop a lifelogging platform technology.
Autographer Panasonic 4K Google Glass
Moves App Google Fit BASIS Watch Strava
RescueTime LoggerMan Camlapse
MyTracks OpenPaths
CameraPhoneInstagramMedia Lifelogging
Activity
Information Access
SMS Backup
CallRecorderPro
VoiceRecorder
WebServices
SwarmLocation
23&Me EEG DietOthers
Health
Last.fm
NarrativeClip
Secure Personal Lifelog
It is not conventional photos, just data, 2,000 - 5,000 per day! Too many for an individual to analyse
My Lifelog in Numbers
70+ Papers and 12 first generation prototypes
10 Years of location log, with millions of
GPS points
80 Million: heartbeats,
with GSR and activity
1 Year of computer interactions
(mouse, keyboard)
9 Years
of lifelog, since 2006
16.5 Million wearable camera images
About 1TB per year
Segmentation of raw data into units such as events or moments.
These can be enriched automatically with metadata,
increasing their value.
Events are analogous to our episodic
memory
Like all multimedia data, we began by browsing, but there is too much data,
much repetition. We need search (Googlisation).
2.5 year study into locating important items: Increase from 25% to 75% success in 1/10th the time when
searching not browsing.
Searching is based on data analytics and machine/deep learning to ‘understand’ the sensor data.Segmentation
Find the unit of retrieval for many use-cases… there is no one correct unit
EnrichmentAutomatically turn raw
sensor data into meaningful information
Search EngineTo index the data
InterfacesSupporting Applications
Aiden Doherty, DCU, office setting, conversation, indoor,
discussing CHI paper.
“On Sept 23rd, I was in DCU discussing the CHI paper with Aiden at his desk”
The challenge is to automatically extract knowledgefrom the lifelog data to supportrecall/retrieval, reminiscence
and reflection.
Raw$Sensors$
What$doing$
What$Environment$
Movement$• Ac8vity$• Energy$
Where$Who$is$there$
When$$
Why$
“Shopping for a coat lastTuesday in Helsinki”
Enrich Semantics by Applying Data Analytics
Kahneman et al. A survey method for characterizing daily life experience: The day reconstruction method. Science, 306(5702):1776–1780, 2004.
1
2
3
4
Intimate Relations
5
6
Socialising
Relaxing
Pray/Worship/Meditate
Eating
Exercising
7 Watching TV
8 Shopping
9
10
11
12
Preparing Food
13
14
On the Phone
Napping
Taking Care of Children
Computer/Internet
Housework
15 Working
16 Commuting
Recognising Life Activities
Summarisation REFLECTION
Search EngineRECALL/RETRIEVAL
BrowsingREMINISCENCE
Considering the Use-Cases and
Developing Applications
Amended from: Abigail J. Sellen and Steve Whittaker. 2010. Beyond total capture: a constructive critique of lifelogging. Commun. ACM 53, 5 (May 2010)
How does this relate to BioHacking?There is a lot of R&D still to do… no consideration of UI
No adequate lifelogging device or software yet
Privacy (lifelogging looks out, QS looks in)Trust for sharing and storage
Get data now, you can not get data retrospectivelyNew tools and software will extract value later
Some Final Thoughts
Privacy Awareness - Automated Negative Face Blurringwith real-time Policy-driven Access Restrictions
Privacy