Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
July 25, 2010 SensorKDD 2010 1
Activity Recognition Using Activity Recognition Using Cell Phone AccelerometersCell Phone Accelerometers
Jennifer Kwapisz, Gary Weiss, Samuel MooreJennifer Kwapisz, Gary Weiss, Samuel MooreDepartment of Computer & Info. ScienceDepartment of Computer & Info. Science
Fordham UniversityFordham University
July 25, 2010 2SensorKDD 2010
We are Interested in WISDMWe are Interested in WISDM
WISDM: WISDM: WIrelessWIreless Sensor Data MiningSensor Data MiningPowerful portable wireless devices are becoming Powerful portable wireless devices are becoming common and are filled with sensors common and are filled with sensors Smart phones: Android phones, iPhoneSmart phones: Android phones, iPhoneMusic players: iPod TouchMusic players: iPod Touch
Sensors on smart phones include:Sensors on smart phones include:Microphone, camera, light sensor, proximity sensor, Microphone, camera, light sensor, proximity sensor, temperature sensor, GPS, compass, temperature sensor, GPS, compass, accelerometeraccelerometer
July 25, 2010 3SensorKDD 2010
AccelerometerAccelerometer--Based Activity Based Activity RecognitionRecognition
The ProblemThe Problem: use accelerometer data to : use accelerometer data to determine a userdetermine a user’’s activitys activityActivities include:Activities include:
Walking and joggingWalking and joggingSitting and standingSitting and standingAscending and descending stairsAscending and descending stairsMore activities to be added in future workMore activities to be added in future work
July 25, 2010 4SensorKDD 2010
Applications of Activity RecognitionApplications of Activity Recognition
Health ApplicationsHealth ApplicationsGenerate activity profile to monitor overall type and Generate activity profile to monitor overall type and quantity of activityquantity of activityParents can use it to monitor their childrenParents can use it to monitor their childrenCan be used to monitor the elderlyCan be used to monitor the elderly
Make the device contextMake the device context--sensitivesensitiveCell phone sends all calls to voice mail when joggingCell phone sends all calls to voice mail when joggingAdjust music based on the activityAdjust music based on the activity
Broadcast (Broadcast (FacebookFacebook) your every activity) your every activity
July 25, 2010 5SensorKDD 2010
Our WISDM PlatformOur WISDM Platform
Platform based on Android cell phonesPlatform based on Android cell phonesAndroid is GoogleAndroid is Google’’s open source mobile computing OSs open source mobile computing OSEasy to program, free, will have a large market shareEasy to program, free, will have a large market share
Unlike most other work on activity recognition:Unlike most other work on activity recognition:No specialized equipmentNo specialized equipmentSingle device naturally placed on body (in pocket)Single device naturally placed on body (in pocket)
July 25, 2010 6SensorKDD 2010
Our WISDM PlatformOur WISDM Platform
Current research was conducted offCurrent research was conducted off--linelineData was collected and later analyzed offData was collected and later analyzed off--lineline
In future our platform will operate in realIn future our platform will operate in real--timetimeIn June we released realIn June we released real--time sensor data time sensor data collection app to Android marketplacecollection app to Android marketplace
Currently collects accelerometer and GPS dataCurrently collects accelerometer and GPS data
July 25, 2010 7SensorKDD 2010
AccelerometersAccelerometers
Included in most smart phones & other devicesIncluded in most smart phones & other devicesAll Android phones, iPhones, iPod Touches, etc.All Android phones, iPhones, iPod Touches, etc.TriTri--axial accelerometers that measure 3 dimensionsaxial accelerometers that measure 3 dimensions
Initially included for screen rotation and Initially included for screen rotation and advanced game playadvanced game play
July 25, 2010 8SensorKDD 2010
Examples of Raw DataExamples of Raw Data
Next few slides show data for one user over a Next few slides show data for one user over a few seconds for various activitiesfew seconds for various activitiesCell phone is in userCell phone is in user’’s pockets pocketEarthEarth’’s gravity is registered as accelerations gravity is registered as accelerationAcceleration values relative to axes of the Acceleration values relative to axes of the device, not Earthdevice, not Earth
In theory we can correct this given that we can In theory we can correct this given that we can determine orientation of the devicedetermine orientation of the device
July 25, 2010 9SensorKDD 2010
StandingStanding
July 25, 2010 10SensorKDD 2010
SittingSitting
July 25, 2010 11SensorKDD 2010
WalkingWalking
July 25, 2010 12SensorKDD 2010
JoggingJogging
July 25, 2010 13SensorKDD 2010
Descending StairsDescending Stairs
July 25, 2010 14SensorKDD 2010
Ascending StairsAscending Stairs
July 25, 2010 15SensorKDD 2010
Data Collection ProcedureData Collection Procedure
UserUser’’s move through a specific courses move through a specific coursePerform various activities for specific timesPerform various activities for specific timesData collected using Android phonesData collected using Android phonesActivities labeled using our Android appActivities labeled using our Android app
Data collection procedure approved by Data collection procedure approved by Fordham Institutional Review Board (IRB)Fordham Institutional Review Board (IRB)Collected data from 29 usersCollected data from 29 users
July 25, 2010 16SensorKDD 2010
Data PreprocessingData Preprocessing
Need to convert time series data into examplesNeed to convert time series data into examplesUse a 10 second example duration (i.e., window)Use a 10 second example duration (i.e., window)
3 acceleration values every 50 ms (600 total values)3 acceleration values every 50 ms (600 total values)Generate 43 total featuresGenerate 43 total features
Ave. acceleration each axis (3)Ave. acceleration each axis (3)Standard deviation each axis (3)Standard deviation each axis (3)Binned/histogram distribution for each axis (30)Binned/histogram distribution for each axis (30)Time between peaks (3)Time between peaks (3)Ave. resultant acceleration (1)Ave. resultant acceleration (1)
July 25, 2010 17SensorKDD 2010
Final Data SetFinal Data SetID Walk Jog Up Down Sit Stand Total1 74 15 13 25 17 7 151 2 48 15 30 20 0 0 113 3 62 58 25 23 13 9 190 4 65 57 25 22 6 8 183 5 65 54 25 25 77 27 273 6 62 54 16 19 11 8 170 7 61 55 13 11 9 4 153 8 57 54 12 13 0 0 136 9 31 59 27 23 13 10 163 10 62 52 20 12 16 9 171 11 64 55 13 12 8 9 161 12 36 63 0 0 8 6 113 13 60 62 24 15 0 0 161 14 62 0 7 8 15 10 102 15 61 32 18 18 9 8 146 16 65 61 24 20 0 8 178 17 70 0 15 15 7 7 114 18 66 59 20 20 0 0 165 19 69 66 41 15 0 0 191 20 31 62 16 15 4 3 131 21 54 62 15 16 12 9 168 22 33 61 25 10 0 0 129 23 30 5 8 10 7 0 60 24 62 0 23 21 8 15 129 25 67 64 21 16 8 7 183 26 85 52 0 0 14 17 168 27 84 70 24 21 11 13 223 28 32 19 26 22 8 15 122 29 65 55 19 18 8 14 179
Sum 1683 1321 545 465 289 223 4526% 37.2 29.2 12.0 10.2 6.4 5.0 100
July 25, 2010 18SensorKDD 2010
Data Mining StepData Mining Step
Utilized three WEKA learning methods Utilized three WEKA learning methods Decision Tree (J48)Decision Tree (J48)Logistic RegressionLogistic RegressionNeural NetworkNeural Network
Results reported using 10Results reported using 10--fold cross validationfold cross validation
July 25, 2010 19SensorKDD 2010
Summary ResultsSummary Results
July 25, 2010 20SensorKDD 2010
J48 Confusion MatrixJ48 Confusion Matrix
20817214Stand
32703204Sit
21258921399Down
221073232388Up
111216127516Jog
028272141513WalkActual
Class
StandSitDownUpJogWalk
Predicted Class
July 25, 2010 21SensorKDD 2010
ConclusionsConclusions
Able to identify activities with good accuracyAble to identify activities with good accuracyHard to differentiate between ascending and Hard to differentiate between ascending and descending stairs. To limited degree also looks like descending stairs. To limited degree also looks like walking.walking.Can accomplish this with a cell phone placed Can accomplish this with a cell phone placed naturally in pocketnaturally in pocketAccomplished with simple features and standard Accomplished with simple features and standard data mining methodsdata mining methods
July 25, 2010 22SensorKDD 2010
Related WorkRelated Work
At least a dozen papers on activity recognition using At least a dozen papers on activity recognition using multiple sensors, mainly accelerometersmultiple sensors, mainly accelerometers
Typically studies only 10Typically studies only 10--20 users20 usersActivity recognition also done via computer visionActivity recognition also done via computer visionActigraphyActigraphy uses devices to study movementuses devices to study movement
Used by psychologists to study sleep disorders, ADDUsed by psychologists to study sleep disorders, ADDA few recent efforts use cell phonesA few recent efforts use cell phones
Yang (2009) used Nokia N95 and 4 usersYang (2009) used Nokia N95 and 4 usersBrezmesBrezmes (2009) used Nokia N95 with real(2009) used Nokia N95 with real--time recognitiontime recognition
One model per user (requires labeled data from each user)One model per user (requires labeled data from each user)
July 25, 2010 23SensorKDD 2010
Future WorkFuture Work
Add more activities and usersAdd more activities and usersAdd more sophisticated featuresAdd more sophisticated featuresTry timeTry time--series based learning methodsseries based learning methodsGenerate results in real timeGenerate results in real timeDeploy higher level applications: activity profilerDeploy higher level applications: activity profiler
July 25, 2010 24SensorKDD 2010
Other WISDM ResearchOther WISDM Research
Cell PhoneCell Phone--Based Biometric identificationBased Biometric identification11
Same accelerometer data and same generated features but Same accelerometer data and same generated features but added 7 users (36 in total)added 7 users (36 in total)If we group all of the test examples from one cell phone and If we group all of the test examples from one cell phone and apply majority voting, achieve 100% accuracyapply majority voting, achieve 100% accuracyCan be used for security or automatic personalizationCan be used for security or automatic personalization
Interested in GPS Interested in GPS spatiospatio--temporal data miningtemporal data mining
1 Kwapisz, Weiss, and Moore, Cell-Phone Based Biometric Identification, Proceedings of the IEEE 4th International Conference on Biometrics: Theory, Applications, and Systems (BTAS-10), September 2010.
July 25, 2010 SensorKDD 2010 25
Thank YouThank You
Questions?Questions?