A feasibility study on smartphone accelerometer-based recognition of household activities and influence of smartphone position VincenzoDellaMea1,OmarQuattrin1,MariaParpinel2

1) Dept.ofMathematicsandComputerScience,UniversityofUdine,Italy

2) Dept.ofMedicalandBiologicalSciences,UniversityofUdine,Italy

This is an original manuscript of an article published by Taylor & Francis in Informatics for Health and Social Care on 22 Dec 2016, available online: https://www.tandfonline.com/doi/full/10.1080/17538157.2016.1255214 VincenzoDellaMea,OmarQuattrin&MariaParpinel(2017)Afeasibilitystudyonsmartphoneaccelerometer-basedrecognitionofhouseholdactivitiesandinfluenceofsmartphoneposition,InformaticsforHealthandSocialCare,42:4,321-334,DOI:10.1080/17538157.2016.1255214CORRESPONDINGAUTHOR:

VincenzoDellaMeaDept.ofMathematicsandComputerScience,UniversityofUdineViadelleScienze206,33100UDINE,ItalyPhone:+39-0432-558461Fax:+39-0432-558499Email:Vincenzo.dellamea@uniud.it

KEYWORDS

Activityrecognition;physicalactivity;smartphone;accelerometer;householdactivities;mobile

applications

RUNNINGHEAD

Smartphone-basedhouseholdactivitiesrecognition

ABSTRACT

BACKGROUND:Obesityandphysicalinactivityarethemostimportantriskfactorsforchronic

diseases.Thepresentstudyaimedat:i)developingandtestingamethodforclassifyinghousehold

activitiesbasedonasmartphoneaccelerometer;ii)evaluatingtheinfluenceofsmartphone

position;andiii)evaluatingtheacceptabilityofwearingasmartphoneforactivityrecognition.

METHODS:AnAndroidapplicationwasdevelopedtorecordaccelerometerdataandcalculate

descriptivefeatureson5-secondtimeblocks,thenclassifiedwith9algorithms.Household

activitieswere:sitting,workingatthecomputer,walking,ironing,sweepingthefloor,goingdown

stairswithashoppingbag,walkingwhilecarryingalargebox,andclimbingstairswithashopping

bag.Tenvolunteerscarriedouttheactivitiesforthreetimes,eachonewithasmartphoneina

differentposition(pocket,arm,andwrist).Userswerethenaskedtoansweraquestionnaire.

RESULTS:1440timeblockswerecollected.Threealgorithmsdemonstratedanaccuracygreater

than80%forallsmartphonepositions.Whileforsomesubjectsthesmartphonewas

uncomfortable,itseemsthatitdidnotreallyconditiontheactivities.

CONCLUSIONS:Smartphonescanbeusedtorecognisehouseholdactivities.Afurther

developmentistomeasuremetabolicequivalenttasksstartingfromaccelerometerdataonly.

1 Introduction In the last 50 years, the changing of lifestyle has produced an increase in over-nutrition and

sedentaryhabits.Epidemiologicalstudieshavedemonstratedthatobesityandphysicalinactivity

arethemostimportantriskfactorsforchronicdiseases[1,2].In2004,5.5%and4.8%ofdeaths

globallywerecausedbyphysicalinactivityandoverweightandobesity,respectively[3].Recently,

themetabolicsyndrome(MS)hasbeendefinedasamedicalconditioncharacterizedbyexcessof

adiposity (especially abdominal), glucose intolerance, high levels of haematic lipids, and high

bloodpressure.InItaly,theprevalenceofMSrangesfrom3%insubjectsbetween20and29years

to 25% in subjects over 70 years [4]. Two major international associations (the International

DiabetesAssociation and theAmericanHeartAssociation) agree that thedevelopmentofMS is

primarilyduetoobesityandlimitedornophysicalactivityandthatMSisaclusterofmetabolic

riskfactorsforchronicdiseases[5].BehavioursassociatedwithMSriskfactorsaredietaryhabits

andphysicalactivity,whichcanbeevaluatedwithspecificquestionnaires to indirectlyestimate

energyintakeandexpenditure[6,7].Inthepresentpaperwedealonlywithphysicalactivityand

relatedenergyexpenditure.

The gold-standard methods to calculate energy expenditure are doubly labelled water

methodologyanddirect/indirect calorimetry,but theyare feasibleonly in the researchsettings

because they are expensive and require specific equipment [7]. Thus, a shortcut to evaluate an

individualphysicalactivitylevelistousequestionnaires,interviewsandactivitydiaries[8].

The most common and validated instrument to collect information on physical activity is the

International Physical Activity Questionnaire (IPAQ) [9,10], which is aimed at recollecting the

activities of the last week tomeasure their intensity throughmetabolic equivalent task (MET)

minutes.OneMETisconsideredastherestingmetabolicrateobtainedduringquietsitting;energy

expendituresofotheractivitiesareexpressedasmultiplesoftherestinglevelMETandtheyrange

from 0.9 (when sleeping) to 18 (when running at 17.54 km/h). The IPAQ method allows to

calculate two scores related to the physical activity carried out in a typicalweek: a categorical

score (low,moderate,high)anda continuous score (METminutesperweek), corresponding to

theMETsspentsummingupallactivitiestimespertheirMETvalue.

A first classification of physical activities accompanied by their METs was published in the

CompendiumofPhysicalActivitiesin1993[11].TheCompendiumwassubsequentlyupdated,the

lastupdatebeingpublishedin2011[12].ThecompleteCompendiumisavailableonline[13],and

iscurrentlyusedtostandardizeactivitiesinmanyepidemiologicalstudies.

Scores calculatedusing IPAQrepresentonlya standardizedestimate,because theCompendium

was not developed to determine the precise energy cost of activities within an individual.

However, Byrne et al. [14] andKozey et al. [15] have developedmethods to consider personal

variationsinsex,bodymass,height,andageforcorrectingMETvalues.

While questionnaires are typically administered to users by professionals, they are indeed a

suitabletoolforpatientempowerment[16],i.e.,activeinvolvementoftheperson/patientintheir

own care, including directly recording their own health data, also backed by evidence on

significanceofpatientprovideddata[17].

More recently, the availability of smartphones and tablets has led to the so-called m-Health

(mobilehealth),basedonphones,shorttextmessaging,mobilewebaccess,uptothemostrecent

smartphone applications [18]. In particular, the effectiveness ofmobile systemswhen used for

collectingpatientdiarieshasbeendemonstrated[19].Allthesetoolsmaybeseenasanewwayto

strengthen the patient-physician relationship [20], in particular in the case of chronic diseases,

whichdeveloponthelongtermandhaveconsequencesonageing[21].

Infact,manystudieshavebeenpublishedintheliteratureontheuseofwearableaccelerometers

forrecognizingactivitiesbeingdonebythesmartphoneowner.

Mostoftheresearchavailableuntilnowhasbeencarriedoutusingmorethanoneaccelerometer

in different body parts. In one of the seminal studies, Foerster et al. [22] exploited 5

accelerometersandshowedthattwowereneededtorecognizethebasicsituations(sitting,lying,

standing, and moving) and more were needed for more specific movement subtypes. Five

accelerometers, butof abiaxial type,werealsousedbyBaoet al. [23],whodemonstrated that

thoseplacedonthethighandthedominantwristallowedtodistinguishamonganumberofbasic

activities.Asimilarstudyhasbeenrecentlyreproducedwithadifferentclassifier[24],confirming

previousresults.

In the last few years, a growing number of papers have focused on the use of a single

accelerometer.Khanetal.[25]usedonetriaxialaccelerometeronthesubject’schestandobtained

goodrecognitionaccuracy,even foractivitiessuchasclimbingandgoingdownstairs.Leeetal.

[26]obtainedsimilarresultsplacinganaccelerometerontheleftwaist.Longetal.,inadditionto

walkingandrunning,consideredalsobicycling,drivingandgenericsports[27].Drivingwaseasily

recognizable,whiletheothersweremorecomplexduetotheaccelerometerposition(ontheside)

andthevarietyofmovements.Ravietal.workedalsoonvacuumcleaningandteethwashing,with

atriaxialaccelerometeronthesubject’spelvis[28].Ofcourse,activitiesrelatedtomouthorhands

weredifficulttorecognize.

Somestudieshaveusedothersensorsinadditiontotheaccelerometer.InthestudybyParkkaet

al.[29],20differentsensorsplacedalongthebodyandincludingtwoaccelerometerswereused

tosuccessfullyrecognizebasicactivities.

Otherexperimentshaveembeddedvarioussensorsinonesinglemultimodaldevice.Maureretal.

demonstrated that such a device was able to recognize basic activities independently of the

position,althoughclimbingandgoingdownstairsremainedmorecomplextodiscriminate[30].

Lester et al [31] used 7 sensors in a single device and were able to recognize also stairs and

elevatormovements,duetopressureandaudiosensors.

However, in recent years, a new technology –smartphones- has appeared that has changed the

perspectives on the concrete applicability of sensor-based (in particular accelerometer-based)

activity recognition. In fact, smartphone accelerometers are being used more and more in

experiments aimed at recognizing activities, due to the wide use of smartphones, the growing

availabilityofembeddedsensors,andthecomputationalpowerprovided.

In one of the first experiments, Miluzzo et al. [32] used some internal smartphone sensors

(accelerometer,microphone, and GPS) to recognize basic activities, and reported difficulties in

distinguishingstandingandsittingwhenthephonewasplaced in the trouserspocketoronthe

belt.

Kwapiszetal.[33]usedasmartphoneaccelerometertoautomaticallyrecognizesixdailyactivities

(walking, jogging, sitting, standing, climbing and going down stairs). In their study stair

movementswerethemostdifficulttoclassify,whiletheotheractivitieswereeasilyclassifiedwith

goodprecision.Wuet al. exploited also a gyroscope, but still stairmovementsweredifficult to

recognize[34].Inarecentstudy,Dernbachetal.[35]introducedamongtheactivitiesalsosome

typicalhouseholdactivities:cleaning,cooking,plantwatering,washinghands,andtakingadrug.

Theseactivitiesweredifficult torecognizebecausemadeupofaseriesofdifferentmovements,

sequentialorparallel,andthusnoteasilyrepresentableasasingleaction.

Finally, some studies have evaluated the performance of smartphone accelerometers and have

confirmedthattheiraccuracyissimilartothatofprofessionaltriaxialaccelerometers[36,37].

Toourknowledge,householdactivitieshavebeen studiedonlybymeansof awearable system

consistingofSmartShoesensorandawristaccelerometer[38].Householdactivitieshaveindeed

been under scrutiny for their energy expenditure [39] because, according to international

recommendationsonphysicalactivity levels [1,2], theycontribute to the30minutesperdayof

moderate-intensity activity required to confer health benefits, but their impact is difficult to

estimate.

Tosummarize,previousstudiesshowthatusingmore thanonesensor,either indifferentbody

partsor ina singledevice, allows to identifywithgoodaccuracyavarietyofactivities, someof

which are more difficult to recognize than others. Fortunately, basic activities can be easily

recognizedalsowithasingletriaxialaccelerometer,even ifembeddedinasmartphonebecause

equivalent to professional devices. Thelatter evidence has helped to switch from less practical

setups, where the subject had to wear unusual sensors and devices, to some more natural

exploitation of nowadays common smartphones. The issue of where to position the sensor or

smartphone remains partially open.While some studies have identified a position-independent

model for non-smartphone sensors (e.g., [30]),most of the times, some activities are harder to

recognize because the sensor position is somewhat biasing the kind of activities that could be

recognized.

However, since the most investigated activities include walking, running, bicycling, etc., a

completecharacterizationofthephysicalactivitycarriedoutbyapersonwhoworksbutdoesno

sportisyettobeachieved.Furthermore,toourknowledge,thereisnotyetanevaluationofthe

influence of the smartphone position on activity classification, which should also be evaluated

fromauseracceptabilitypointofview.

Thus, the present study aimed at: i) developing and testing amethod for classifying household

activitiesbasedonsmartphoneaccelerometeroutput;ii)evaluatingtheinfluenceofsmartphone

positiononactivityrecognition;andiii)preliminarilyevaluatingtheacceptabilityofcontinuously

wearingasmartphoneforautomatedrecognitionofactivities.

2 Methods Theprocessofrecognisingactivitiescanbeorganisedinthreesteps:

- collectionofaccelerometerdataduringactivities;

- computationofdescriptivefeaturesfromrawdata;

- classificationofactivitiesbasedupontheirdescriptivefeatures.

Foreachofthesesteps,actionsweretakeninorderto:

- individuaterelevantfeaturestobecomputed(seeSection2.1);

- support data collection and feature computation through a mobile application (Section

2.1);

- select one or more classification algorithms with good performance from a pool of

candidatesobtainedfromtherelevantliterature(Section2.2).

Forthelatterpoint,wedesignedanevaluationmethodologybasedontheexecutionofanumber

of activities (Section 2.3) by subjects (Section 2.4) carrying a smartphone in three different

positions.During theactivities,datawerecollectedandsummarizedusingamobileapplication.

After the activities, the datawere classified bymeans of selected classification algorithms. The

performanceofthealgorithmsforthethreesmartphonepositionswasfinallycomparedwiththe

methodologydescribedinSection2.5.

Acceptabilityofwearingthesmartphonewasevaluatedthroughashortusersurvey(Section2.4).

Finally, we attempted to understand whether the selected algorithms were computationally

suitableforlow-resourcedevicesbyempiricallymeasuringtheirspeed(Section2.5).

We conducted the study in agreementwith the declaration of Helsinki and collected informed

consent from participating subjects. However, since itwas an observational study that did not

involve drug testing and was carried out on healthy people, according to our Institution

regulations,itdidnotneedethicalapprovalbyourEthicsCommittee.

2.1 Accelerometer data collection AprototypeAndroidapplicationwasdesignedanddevelopedtorecordaccelerometerdataonthe

threeaxesandcalculatedescriptivefeatures.Theapplicationsavestwofilesforfurtherusage:one

containingallrawdatacollected,andonecontainingdescriptivefeaturescalculatedasdepictedin

thefollowing.

Recorded data were segmented in 5-second blocks according to a methodology proposed by

Kwapiszetal.[33]andadoptedinmanyotherstudies.Eachblockwasthendescribedbymeansof

22featuresthathadbeenindividuatedamongthoseproposedinotherstudies:

- meanandstandarddeviationoneachofthethreeaxis[33],foratotalof6features;

- minimumandmaximumoneachofthethreeaxis[35],foratotalof6features;

- zero-crossingrateandabsoluteaveragedeviationoneachofthethreeaxis[30],foratotal

of6features;

- correlationoneachpairofaxis(XY,XZ,YZ)[28],foratotalof3features;

- resultingaverageacceleration[33].

2.2 Classification Toclassifytheactivities,theWekasoftware[40]wasused,asinothersimilarstudies,becauseof

the large amount of different classification algorithms it provides. The software was run on a

desktop computer (2.66GHz Intel® Coretm i7 620m processor, 4 GB RAM), thus classification

occurred offline and not directly during the activities. Algorithms were compared using the

Experimenter module of Weka on 10 runs of 10 cross-fold validation for each algorithm and

smartphoneposition.

Wekaprovidesforaverylargenumberofclassificationalgorithms.Inourexperiment,wedecided

torelyatfirstonthealgorithmsproposedinotherstudies.

Inparticular,asinthecaseofdescriptivefeatures,wereliedagainonthepaperbyKwapiszetal.

[33]. In thatpaper, theauthorsused J48,LogisticRegression, andMultilayerPerceptron.Wealso

addedthealgorithmsusedbyDernbachetal.[35]:NaiveBayes,BayesNet,DecisionTable,andKstar.

Finally, we completed the set of algorithms with another two algorithms, LogitBoost and

RandomForest,whichwechosetocoverWekacategoriesnotconsideredbyotherAuthors.

2.3 Activities Activitieswereselectedinordertorepresentdifferenthouseholdactivitylevelsaccordingtotheir

MET:sitting (1MET), sittingandworkingat thecomputer (1.8MET),walking (2MET), ironing

(2.3MET),sweepingthefloor(3.3MET),goingdownstairswithashoppingbag(5MET),walking

whilecarryingalargebox(6MET),andclimbingstairswithashoppingbag(7.5MET).

Activitieswereorganizedinacontinuouspaththatsubjectshadtofollow,witharound30seconds

time for eachactivity and10-secondbreaksbetweenactivities inorder to facilitateaposteriori

segmentation.Acompletepathhadatotaldurationof320seconds.

2.4 Subjects Ten subjects were chosen among the available personnel at the Department of Biological and

Medical Sciences of the University of Udine, Italy, in order to have five female and five male

subjects.Theiragerangedbetween26to40years.

Subjectswereaskedtocarryouttheabove-mentionedactivitypathforthreetimes,eachonewith

the smartphone in adifferentposition (pocket, arm, andwrist).Whenon the armorwrist, the

smartphonewasfixedusinganarmband(suchasthoseusedforsportactivities).

Attheendofthepath,userswerefinallyaskedtoanswerashortsurveyabouttheirexperience

duringtheexperimentation.Thequestionswereaimedatevaluatingtheacceptabilityofwearing

thesmartphoneduringtheactivities.

Thus, foreachof the threepositions,userswererequested to tellwhether thesmartphonewas

uncomfortable, an impediment (meaning that it blocked the execution of the activities), or a

conditioning factor (meaning that it caused the activities to be carried out in an unusualway).

Answerswereona1-5scale(frombesttoworst).

2.5 Analysis Duetothelownumberofsubjects,weonlycalculateddescriptivestatisticalfeatures.

Classificationperformancewasdescribedbymeansofaccuracy,precision,recallandF-measure;

insightsonclassificationerrorswereexaminedbylookingatconfusionmatrices.

We also attempted to consider computational complexity for the sake of real-time use on the

smartphone by measuring the time needed for model building and activity classification (the

latteron350instances).

3 Main outcomes

3.1 Data collection 480 timeblocks for each smartphonepositionwere collected for a total of 1440, that is, 6 (30

seconds)foreachofthe8activities,allmultipliedby10subjects.Figure1showsanoverallview

oftherecordedaccelerometerdataforthewholepathasdonebyonesubject.

(FIGURE1HERE)

3.2 Classification accuracy Table 1 shows the performance of the algorithms for each smartphone position, expressed in

termsofaccuracy,precision,recallandF-measures.

(TABLE1HERE)

Three algorithms demonstrated a classification accuracy greater than 80% for all three

smartphonepositions:RandomForest(84.6-87.4%),Kstar(82.6-89.1%),andMultilayerPerceptron

(82.5-85.4%).Figure2showsacomparisonamongthethreebestalgorithms.

(FIGURE2HERE)

In general, the pocket position showed a slightly higher accuracy (5 algorithms obtainedmore

than 80%accuracy vs 4 forwrist position and 3 for armposition). In absolute terms, the best

recognitionrate(89.4%)wasobtainedwithKstarandthesmartphoneinthepocket.

Most of the classification errors were due to activities that appear similar from a specific

smartphone position (e.g., sweeping and ironing when the smartphone is on the arm). Some

errorswerenotcrucialbecausethedifferenceinMETwasnotgreat(sweeping:2.3MET,ironing:

3.3MET),someothersthoughweremoresevere(e.g.,walkingwithbox:6MET,andwithoutbox:

2 MET). In the latter case, with the smartphone in the pocket, 103 blocks were correctly

recognized (53 aswalkingwithbox, 50without), whereas16were incorrectly classified (9 as

walkingwithbox,7without).Table2showsthetwoconfusionmatricesfortheKstaralgorithm,

pointingouttheabove-mentionedcases.

(TABLE2HERE)

InTable3,asummaryoftheresultsofrelatedstudiesisreportedforcomparisonwithourresults

obtainedwiththeKstaralgorithmandthesmartphoneinthepocket.Thestudieslistedareonly

thosewithcomparableresults,i.e.,withexplicitaccuracyresultsorwithaccuracyobtainablefrom

confusionmatrices.Whenmorethanonealgorithmorpositionwasused,thebestalgorithmand

thepocketpositionareshown.Activitieslistedarethoseconsideredinourstudy.

(TABLE3HERE)

Twomainconsiderationscanbemade.First,theaccuracyofourmethodisgenerallycomparable

with that of other smartphone-based systems and slightly lower than that of systems adopting

multiple sensors or accelerometers. Differences on specific activitiesmay depend on the set of

activitiesconsidered ineach individualstudy,whichmightaccount formoreor lessdifficulty in

classification.Secondly,forspecifichouseholdactivitiessuchasironing,walkingwithabox,and

sweeping,nopreviousstudieswereavailable,althoughotherauthorsexamineddoingthedishes,

vacuuming,foldinglaundry,scrubbing,andeating[23,38].

We also investigated whether the gender of the subject might influence activity classification.

While the resultswere not significantly different, somedifferences betweenmales and females

were found in algorithm accuracy. Recognition of women activities gave better results (on

average: 6%), probably due to themore regular execution of activities bywomen, as shown in

Figure3 for thethreebestalgorithms. Inabsolute terms, thebestrecognitionrate(93.6%)was

obtained for women activities measured when the smartphone was on the arm. The worst

recognitionrate(81.2%)wasinsteadobtainedformaleactivitiescarriedoutwiththesmartphone

onthewrist.

(FIGURE3HERE)

3.3 Computational issues In our preliminary evaluation of computational complexity, the time formodel building ranged

from0.001(Kstar) to4.34seconds (LogisticRegression),while theclassificationof350samples

was always immediate, except for Kstar (3.30s). Table 4 shows the details regarding the time

neededforbuildingthemodel.

Sincethecomputerusedforthetestismorepowerfulthanagenericsmartphone(inparticularfor

floating point operations), maybe Kstar is less suitable for real-time recognition of activities

directlyonasmartphonethanotheralgorithms.However,thisshouldbeprovenwithanad-hoc

implementationonasmartphone,optimizedforitscomputationalresources.

(TABLE4HERE)

3.4 User survey Allusersfilledinthequestionnaireasrequested.Figure4showsthedetailsoftheirevaluationof

theexperiencewiththesmartphone.

Smartphone comfortwas perceivedwith awide range of levels,with a slight predominance of

positiveopinions.Summingup1and2aspositiveopinionsand4-5asnegativeopinions,thewrist

positionturnedouttobethemostcomfortable(8positivevs1negative)andthearmpositionthe

leastcomfortable(6positivevs3negative).

Impedimentandconditioningofactivitieswereevaluatedinverysimilarways,and,ingeneral,it

seems that carrying the smartphonedidnothinderor condition activities, becausenonegative

opinions(4and5)wereexpressed.Asanexample,withthesmartphoneinthewristposition,all

opinions were positive for both impediment and conditioning. The arm position, which was

considered the least comfortable, did not impede or condition activities more than the other

positions.

(FIGURE4HERE)

4 Conclusion Thesmartphone,placedinausualpositionsuchasinapocket,canbeusedtorecognizeseveral

activitiesthroughitsaccelerometer,includingsomehouseholdactivities[23,38].

Although some activities were studied for the first time in our work, our contribution is

preliminary,duetothelimitednumberofsubjectsinvolvedandtothefactthatthesubjectswere

notaskedabouttheirconfidencewiththestudiedhouseholdactivities.

Sincemostofthecurrentmobileandwearabledevicesandapplicationsaredevotedtofitnessand

sports activities, our resultsmay open towards a better understanding of the physical activity

behaviour of populations that arenotdirectly involved in activities explicitly aimedat physical

wellness,butthatdosomephysicalactivityintheirdailylife.

To exploit physical activity recognition, we envisaged two implementation modalities. One

modalityprovidesforapre-calculatedgenericclassificationmodel(subject-independenttraining),

whichtheusercanexploitwithout theneed forspecific trainingandpersonalization.Theother

modality isbasedonan initial subject-dependent trainingphase, inwhich theuserhas to label

his/herownactivitiestohelptheconstructionofapersonalizedclassificationmodel,whichthus

canexploitknowledgeonthesubject(e.g.,preferredsmartphoneposition,gender,etc.).Whilethe

pre-calculatedmodalityisofimmediateuse,thesecondoneisinitiallymorecomplicatedforthe

user, but it certainly provides greater accuracy in recognizing activities. However, in the first

modality,themodelbuildingoccursoffline,whereasinthesecondmodalitythemodelhastobe

builtonthesmartphone,andthiscantakesometime.Classificationhastobedoneinrealtimeor

closetorealtimedirectlyonthesmartphone,andalgorithmshavethustobeoptimizedforthat

(e.g.,[41]).Whilethishastobeconfirmedbyaspecificexperimentation,thebestalgorithminour

experimentation (Kstar) might not be the most adequate algorithm from this point of view.

However,sincedifferencesinaccuracyamongthealgorithmswereminimal,eitherRandomForest

orMultilayerPerceptroncouldbeinsteadimplementeddirectlyonthesmartphone.

The demonstrated capability of recognizing common activities can be exploited in all those

situationswerephysicalactivity iscrucial [1-3], forexample inmetabolicsyndrome[4-5].Since

measurement of physical activities is one of the components to evaluate the possible risk of

metabolicsyndrome,asmartphonecapableofclassifyingandrecordingactivitiesduringtheday

couldcomplementtheIPAQquestionnaire,thusprovidingawayforestimatingphysicalactivityin

a non-invasiveway. However, since there are reports of variable reliability of IPAQ in specific

populations [9,42-44]and,moregenerally,of limitations inmeasuringphysicalactivity through

questionnaires [45], a furtherdevelopment couldbe to effectivelymeasureMETs starting from

smartphonesensorsonly,althoughsomeevidenceexiststhataccelerometeralonecannotprovide

a reliable estimate of energy expenditure [46]. In addition to accelerometer data, whichmight

eventuallycomefromelectronicactivitymonitors[47,48],othersensiblesourcesof information

couldbetheinternalgyroscopeofsomehigh-endsmartphones,oralsothevitalsignsensorsbeing

introduced in forthcoming smartwatches. All of this, throughdata fusion [49],may concur to a

betterrecognitionofdailylifeactivities,althoughthemostsophisticatedandsensor-richsystems

arealsomoreexpensiveandthuslessadequateforreachingawideaudience.

Declarationofinterest:Theauthorreportsnoconflictsofinterest.

5 References [1] World Health Organization. Diet, nutrition and the prevention of chronic diseases. Geneva:

WorldHealthOrganization,2002.

[2] World Cancer Research Fund / American Institute for Cancer Research. Food, Nutrition,

PhysicalActivity, and thePrevention of Cancer: aGlobal Perspective.WashingtonDC:AICR,

2007.

[3] WorldHealthOrganization.Globalhealthrisks:mortalityandburdenofdiseaseattributableto

selectedmajorrisks.Geneva:WorldHealthOrganization,2009.

[4] MiccoliR,BianchiC,OdoguardiL,PennoG,CaricatoF,GiovannittiMG,PucciL,DelPratoS.

PrevalenceofthemetabolicsyndromeamongItalianadultsaccordingtoATPIIIdefinition.

NutrMetabCardiovascDis2005;15:250-4.

[5] CornierMA,DabeleaD,HernandezTL,LindstromRC,SteigAJ,StobNR,VanPeltRE,WangH,

EckelRH.Themetabolicsyndrome.EndocrRev2008;29:777-822.

[6] WillettW.Nutritionalepidemiology.NewYork:OxfordUniversityPress,2013.

[7] WDMcArdle,FIKatch,VLKatch.ExercisePhysiology:Energy,Nutrition,andHuman

Performance.6thed.Philadelhia:LippincottWilliams&Wilkins,2006.

[8] CorderK,EkelundU,SteelRM,WarehamNJ,BrageS.Assessmentofphysicalactivityinyouth.J

ApplPhysiol2008;105:977-987.

[9] CraigCL,MarshallAL,SjostromM,BaumanAE,BoothML,etal.Internationalphysicalactivity

questionnaire:12countryreliabilityandvalidity.MedSciSportsExerc2003;35:1381–1395.

[10] InternationalPhysicalActivityQuestionnaireWebsite.[cited2016Feb10].Availablefrom:

https://sites.google.com/site/theipaq/

[11] AinsworthB,HaskellW,LeonA,JacobsDRJr.,MontoyeHJ,SallisJF,PaffenbargerRS,Jr.

Compendiumofphysicalactivities:Classificationofenergycostsofhumanphysicalactivities

MedSciSportsExerc1993;25:71-80.

[12] AinsworthBE,HaskellWL,HerrmannSD,MeckesN,BassettDRJr,Tudor-LockeC,GreerJL,

VezinaJ,Whitt-GloverMC,LeonAS.2011CompendiumofPhysicalActivities:asecondupdate

ofcodesandMETvalues.MedSciSportsExerc2011;43:1575-81.

[13] CompendiumofPhysicalActivities.[cited2016Feb10].Availablefrom:

https://sites.google.com/site/compendiumofphysicalactivities/

[14] ByrneNM,HillsAP,HunterGR,WeinsierRL,SchutzY.Metabolicequivalent:onesizedoes

notfitall.JApplPhysiol2005;99:1112-1119.

[15] KozeyS,LydenK,StaudenmayerJ,FreedsonP.ErrorsinMETestimatesofphysical

activitiesusing3.5mlxkg(-1)xmin(-1)asthebaselineoxygenconsumption.JPhysActHealth

2010;7:508-516.

[16] BruegelRB.Patientempowerment--atrendthatmatters.JAHIMA1998;69:30-3.

[17] SloanJA,CellaD,HaysRD.Clinicalsignificanceofpatient-reportedquestionnairedata:

anothersteptowardconsensus.JClinEpidemiol2005;58:1217-9.

[18] WorldHealthOrganization.mHealth:Newhorizonsforhealththroughmobiletechnologies.

2011.[cited2016Feb10].Availablefrom:

http://www.who.int/goe/publications/ehealth_series_vol3/en

[19] DaleO,HagenKB.Despitetechnicalproblemspersonaldigitalassistantsoutperformpen

andpaperwhencollectingpatientdiarydata.JClinEpidemiol.2007;60:8-17.

[20] FortneyJC,BurgessJFJr,BosworthHB,BoothBM,KaboliPJ.Are-conceptualizationof

accessfor21stcenturyhealthcare.JGenInternMed2011;26:639-47.

[21] AscherioA,RimmEB,GiovannucciEL,ColditzGA,RosnerB,WillettWC,SacksF,Stampfer

MJ. A prospective study of nutritional factors and hypertension amongUSmen. Circulation.

1992;86:1475-84.

[22] Foerster F, Fahrenberg J. Motion pattern and posture: correctly assessed by calibrated

accelerometers.BehavResMethodsInstrumComput2000;32:450-7.

[23] Bao L, Intille S. Activity Recognition from User-Annotated Acceleration Data. Lect Notes

ComputSc2004;3001:1-17.

[24] ManniniA,SabatiniAM.Machinelearningmethodsforclassifyinghumanphysicalactivity

fromonbodyaccelerometers.Sensors2010;10:1154-1175.

[25] Khan AM, Lee YK, Lee SY, Kim TS. A triaxial accelerometer-based physical activity

recognition via augmented-signal features and a hierarchical recognizer. IEEE Trans Inf

TechnolBiomed2010;14:1166–1172.

[26] LeeM,KimJ,KimK,LeeI, JeeSH,YooSK..Physicalactivityrecognitionusingasingletri-

axis accelerometer. World Congress on Engineering and Computer Science 2009, San

Francisco,USA:14-17

[27] LongX, YinB,AartsRM. Single-accelerometer-baseddaily physical activity classification.

ConfProcIEEEEngMedBiolSoc.2009;2009:6107-10..

[28] RaviN,DandekarN,MysoreP,LittmanML.Activityrecognitionfromaccelerometerdata.

17thconferenceonInnovativeapplicationsofartificialintelligence,Pittsburgh2005;Volume

3:1541-1546.

[29] Parkka J, ErmesM, Korpipaa P,Mantyjarvi J, Peltola J, Korhonen I. Activity classification

usingrealisticdatafromwearablesensors.IEEETransInfTechnolBiomed2006;10:119-128.

[30] MaurerU,SmailagicA,SiewiorekD,DeisherM.Activityrecognitionandmonitoringusing

multiple sensors on different body positions. International Workshop on Wearable and

Implantable Sensor Networks, Cambridge; 2006. Washington DC: IEEE Computer Society;

2006:113-116.

[31] Lester J,ChoudhuryT,BorrielloG.Apracticalapproachtorecognizingphysicalactivities.

LectNotesComputSc2006;3968:1–16.

[32] MiluzzoE,LaneN,FodorK,PetersonR,LuH,MusolesiM,EisenmanS,ZhengX,CampbellA.

Sensing meets mobile social networks: The design, implementation and evaluation of the

CenceMeapplication.6thACMConferenceonEmbeddedNetworkedSensorSystems,Raleigh,

USA2008.NewYork:ACM;2008:337-350.

[33] KwapiszJ,WeissG,MooreS.ActivityRecognitionusingCellPhoneAccelerometers.Human

Factors,2010.

[34] WuW,DasguptaS,RamirezEE,PetersonC,NormanGJ.Classificationaccuraciesofphysical

activitiesusingsmartphonemotionsensors.JMedInternetRes2012;14:e130.

[35] Dernbach S, Das B, Krishnan NC, Thomas BL, Cook DJ. Simple and Complex Activity

RecognitionthroughSmartPhones.8thInternationalConferenceonIntelligentEnvironments,

Guanajuato,México,2012.WashingtonDC:IEEEComputerSociety;2012;214-221.

[36] NishiguchiS,YamadaM,NagaiK,MoriS,KajiwaraY,SonodaT,etal.Reliabilityandvalidity

ofgaitanalysisbyandroid-basedsmartphone.TelemedJEHealth2012;18:292-296.

[37] Saha IK, Dirik AE, Topkara U, Memon N, Gutierrez G, Rao S. Reliability and validity of

accelerometer-based smartphones to assess physical activity. Proceedings of the Annual

MeetingoftheAmericanSocietyofBiomechanics,Providence,USA,2010:571-2.

[38] Edgar SR, Fulk GD, Sazonov ES. Recognition of household and athletic activities using

SmartShoe.ConfProcIEEEEngMedBiolSoc.2012;2012:6382-5.

[39] Brooks AG, Withers RT, Gore CJ, et al. Measurement and prediction of METs during

householdactivitiesin35-to45-year-oldfemales.EurJApplPhysiol2004;91:638-648.

[40] HallM,FrankE,HolmesG,FahringerBP,ReutemannP,WittenIH.TheWEKADataMining

Software:AnUpdate.SIGKDDExplorations2009:11(1),10-18.

[41] Tapia EM, Intille SS, Haskell W, Larson K, Wright J, King A, Friedman R. Real-Time

recognition of physical activities and their intensities using wireless accelerometers and a

heartratemonitor.11thIEEEInternationalSymposiumonWearableComputers,Boston,USA,

2007.WashingtonDC:IEEEComputerSociety;2007;1-4.

[42] Wang C, Chen P, Zhuang J. Validity and reliability of International Physical Activity

Questionnaire-ShortForminChineseyouth.ResQExercSport.2013;84:S80-6.

[43] Hansen AW, Dahl-Petersen I, Helge JW, Brage S, Grønbæk M, Flensborg-Madsen T.

Validation of an Internet-based long version of the International Physical Activity

Questionnaire in Danish adults using combined accelerometry and heart rate monitoring. J

PhysActHealth2014;11:654-64.

[44] OyeyemiAL,OyeyemiAY,AdegokeBO,OyetokeFO,AliyuHN,AliyuSU,RufaiAA.TheShort

International Physical Activity Questionnaire: cross-cultural adaptation, validation and

reliabilityoftheHausalanguageversioninNigeria.BMCMedResMethodol2011;11:156.

[45] ShephardRJ.Limitstothemeasurementofhabitualphysicalactivitybyquestionnaires.BrJ

SportsMed2003;37:197-206.

[46] SasakiJE,HickeyA,MaviliaM,TedescoJ,JohnD,KozeyKeadleS,FreedsonPS.Validationof

the Fitbit wireless activity tracker for prediction of energy expenditure. J Phys Act Health

2015;12:149-54.

[47] VooijsM,AlpayLL,Snoeck-StrobandJB,BeerthuizenT,SiemonsmaPC,AbbinkJJ,SontJK,

Rövekamp TA. Validity and usability of low-cost accelerometers for internet-based self-

monitoring of physical activity in patients with chronic obstructive pulmonary disease.

InteractJMedRes2014;3:e14.

[48] Lee JM, Kim Y,Welk GJ. Validity of consumer-based physical activity monitors. Med Sci

SportsExerc.2014;46:1840-8.

[49] ShoaibM,BoschS,IncelOD,ScholtenH,HavingaPJ.Fusionofsmartphonemotionsensors

forphysicalactivityrecognition.Sensors(Basel).2014;14:10146-76.

Figure1–Anexampleofaccelerometerdataforallactivitiesandsmartphonepositions

Figure2–theaccuracyofthebestthreealgorithmsbysmartphoneposition

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Arm Wrist Pocket

MultilayerPerceptronKstar

RandomForest

Figure3–Accuracyofthebestthreealgorithmsbygenderandbysmartphoneposition

74%

76%

78%

80%

82%

84%

86%

88%

90%

92%

94%

96%

pocket wrist arm pocket wrist arm

F M

RandomForest Kstar MultilayerPerceptron

Figure4–Resultsofusersurvey

0

2

4

6

8

10

1 2 3 4 5

Comfort

comfortable highlyuncomfortable

0

2

4

6

8

10

1 2 3 4 5

Impediment

no Totalimpediment

0

2

4

6

8

10

1 2 3 4 5

Conditioning

no highlyconditioning

Position J48Logistic

RegressionMultilayerPerceptron

NaiveBayes

BayesNet

DecisionTable KStar

RandomForest

LogitBoost

arm Accuracy 74.8 76.3 84.4 74.2 75.7 62.1 87.9 85.6 78.1 Precision 0.64 0.73 0.82 0.64 0.63 0.42 0.84 0.83 0.72 Recall 0.66 0.68 0.87 0.69 0.67 0.56 0.93 0.78 0.73 F-measure 0.63 0.69 0.83 0.64 0.63 0.46 0.88 0.79 0.71wrist Accuracy 76.7 76.8 84.3 76.8 79.5 64.4 83.1 85.2 81.6 Precision 0.73 0.77 0.80 0.64 0.77 0.46 0.73 0.81 0.78 Recall 0.70 0.73 0.86 0.79 0.73 0.66 0.91 0.85 0.79 F-measure 0.70 0.73 0.82 0.70 0.74 0.53 0.80 0.82 0.77pocket Accuracy 80.7 75.6 82.2 66.1 73.6 63.0 89.4 89.4 83.1 Precision 0.84 0.86 0.92 0.90 0.86 0.49 0.96 0.95 0.89 Recall 0.82 0.85 0.91 0.81 0.76 0.73 0.91 0.91 0.89 F-measure 0.82 0.85 0.91 0.84 0.79 0.57 0.93 0.93 0.88

Table1–Performanceofclassifiers(accuracy,precision,recallandF-measure)bysmartphone

position;inboldthemostaccurateclassifierforeachmeasureandposition

clim

bing

stai

rs

ironi

ng

wal

king

with

box

sittin

g

wor

king

at P

C

swee

ping

wal

king

goin

g do

wn

stai

rs

clim

bing

stai

rs

ironi

ng

wal

king

with

box

sittin

g

wor

king

at P

C

swee

ping

wal

king

goin

g do

wn

stai

rs

climbing stairs 55 0 3 0 0 0 1 0 54 0 2 0 0 0 2 2 ironing 0 47 0 2 0 4 0 0 0 45 0 0 0 5 1 1

walking with box 0 0 57 0 0 0 2 1 0 0 53 0 0 0 7 0 sitting 0 1 0 52 4 2 0 0 1 0 0 49 10 0 0 0

working at PC 0 3 0 2 55 0 0 0 0 0 0 1 59 0 0 0 sweeping 5 7 0 2 0 44 2 0 0 3 0 0 0 57 0 0

walking 3 0 3 0 0 2 50 2 1 0 9 0 0 0 50 0 going down stairs 3 0 2 0 0 0 1 54 2 0 2 0 0 0 2 54

ARM POCKET

Table2–ConfusionmatricesforKstaralgorithm,onarmandpocketpositions

position climbing stairs ironing walking

with box sitting working at PC sweeping walking

going down stairs

Multiple accelerometers and sensors

(22) Foerster 2000 5 sensors on sternum, right and left thigh 93.5% 94.2% 96.7% 95.7%

(23) Bao 2004 5 sensors on arm, wrist, knee, ankle, waist 85.6% 94.8% 97.5% 89.7%

(25) Khan 2010 1 sensor at a position closer to the center of mass

99.0% 90.8% 99.0% 99.0%

(27) Long 2009 1 sensor on wrist 80.3% (28) Ravi 2005 1 sensor on a pelvic region 42.9% 97.8% 100%

(29) Parkka 2006

22 sensors on chest, wrist, finger, forehead, upper back, below neck, shoulder, breastbone

96% 79.0%

(31) Lester 2006 7 sensors on shoulder, wrist and waist 84.5% 50.5% 79.8% 80.3%

(38) Edgar 2012 9 sensors on the inside of the wrist and the foot 77.7% 96.7% 100% 95.7%

Smartphone

(32) Miluzzo 2008 average among pocket, hip, necklace 68.2% 94.4%

(33) Kwapisz 2010 pocket 61.5% 95% 91.7% 44.3%

(34) Wu 2012 pocket 69.8% 100% 92.0% 79.4%

(35) Dernbach 2012 not standardized 71.8% 87.3% 60.0% 86.9%

our study (using Kstar) pocket 90.0% 86.5% 88.3% 81.7% 98.3% 95% 83.3% 90%

Table3–AccuracycomparisonofourstudyusingKstaralgorithmwithsmartphoneinthepocketwithpreviousrelevantworks

Classifier Modelbuildingtime(s)J48 0.14LogisticRegression 4.34MultilayerPerceptron 2.90Naivebayes 0.04Bayesnet 0.09DecisionTable 0.22Kstar <0.001RandomForest 0.16LogitBoost 0.24

Table4–timeneededformodelbuilding;inboldthe3mostaccuratealgorithms