Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... ·...

FinemuscularactivityrecognitionusingElectromyography

AlaShaabana

Outline• WhatisElectromyography?

• AlittleBiology.

• SurfaceEMG(sEMG)Signal

• ClassifyingsEMG.

• MyoelectricControlstrategies.

• Ourapproachinfinemuscularactivityrecognition.

Electromyography

• Medicinalelectro-diagnostictechniqueformeasuringandevaluatingtheelectrical output(actionpotential)ofskeletalmuscles.

• Signalscanbeanalyzedtodetectmedicalabnormalities,activationlevels,recruitmentorder,ortoanalyzethethebiomechanicsofhumanoranimalenvironment.

• Twotypes:1. IntramuscularEMG2. SurfaceEMG

IntramuscularEMG

• Typicallyperformedusingmonopolarneedleelectrode– A finewireinserted intoamusclewithasurfaceelectrode forreference,or

– Twofinewires inserted intothemusclereferencedtoeachother.

• Needpre-injectionsandskinpreparations.

• Lessnoise

Invasive,painful.

SurfaceEMG(sEMG)• Surfaceelectrodesonlyprovidealimitedassessmentofthemuscleactivity.– Arrayofelectrodes typicallyused

• Signalisacomposite ofallthemusclefiberactionpotentialsoccurring inthemusclesunderlyingtheskin.

• Pronetonoise,artifacts• Non-invasive

Muscles

• Composedofbundlesofspecializedcells thatareresponsibleforcontractionandrelaxation.– Generateforces,movements,&abilityforexpression.

• 4mainfunctionsofmuscles:1. Producemotion.2. Movingsubstancew/i body.3. Stabilization.4. Generateheat.

Muscles• 3typesofmuscletissue:

1. Skeletalmuscle.2. Smoothmuscle.3. Cardiacmuscle.

• Skeletalmuscles areaformofmuscletissue thatisunderthecontrolofthesomaticnervoussystem.– Voluntarilycontrolled.

• Attachedtothebonebybundles ofcollagen fibers,knownastendons.

• Whenmusclescontractandrelax,theyreleaseatinybioelectricpulsecalledtheActionPotential

sEMG Electrodes

• ElectrodesmeasuringthesEMG signalformanemg channel.• 3maintypes:

– Bipolarconfiguration– Monopolar configuration– Laplacin configuration

• Onechannelbipolarderivationtypical.

• Recentinteresthasalsoincreasedtowardhigh-density sEMG (HD-sEMG).

sEMG Signal

• RawEMGsignalstypicallycomeinasomewhatuselessform.

• Beforebeginningpatternrecognition,severalstepsaretypically followed:1. Cleanup2. Segmentation3. Analysis&preprocessing

sEMG SignalCleanup

• Consistsofseveralsub-steps:

1. Filtration&De-noising.• LPFmosttypical• HPForBandpass alsoused.• WaveletTransforms

2. Rectification.• Halfwave.• Fullwave.

sEMG SignalSegmentation

• Twowindowingtechniques:1. Overlappedwindowing2. Adjacentwindowing

Features

• Featuresselectedcanhavehigher effectonclassificationaccuracythanclassifiertype.

• 3qualitiesdeterminequalityoffeaturespace:1. Maximum classseparability.2. Robustness3. Computational complexity

EMGFeatures

• RawsEMG signalstypicallymappedintosmaller-dimensionfeaturevectors.

• Classifiersperformfaster.– Improvesreal-time properties ofthesystem.

• Canbegroupedinto4categories:– Timedomain(TD)features– Frequencydomain(FD)features– Time-frequency domain(TFD)features– Spatialdomain(SD)features.

OptimalFeatures

• LittleconsensusonstudiestryingtofindtheoptimalfeaturesforclassificationofsEMG signals

• ComparativestudiesshowTDfeaturesachievehigheraccuracyforLDAclassifier.– TFDfeaturesoutperformthemforSVM.– TDfeaturesclassifiedwithLDAhavebeensuggested asoptimal forsEMGclassification.

• Conclusionsnotreliable.– Madeinlow-noise labenvironment.

Timedomain(TD)features

• MostcommonfeaturegroupinsEMG signalclassification.

• Nomathematicaltransformationsneeded.– Fastcalculations, suitable forreal-time– Sensitive tonoiseandartifacts.

• Phinyomark etal.dividedTDfeaturesinto4maintypes1. Energyandcomplexity informationmethods.2. Frequencyinformationmethods.3. Predictionmodelmethods.4. Time-dependence methods.

Timedomain(TD)Features

• SeveralattemptstodeterminetheoptimalTDfeaturevectorforseveralclassifiers.

• Mostcommoncombination:LDAclassifierwithHudgin’s featurevector– Meanabsolute value(MAV)– Waveformlength(WL)– Zerocrossing(ZC)– Signalslopechanges(SSC)

• Lowcomputationalcomplexity,highaccuracy.

Frequencydomain(FD)features

• Calculatedfrompowerspectraldensity(PSD).– Canbedetermined byfiringrateofrecruitedmotorunits ormorphologyofMUAPstravelingalongrespectivemuscle fibers.• Dependsonmusclebeingmeasured.

• Phinyomark studiedthepropertiesof37TDandFDfeatures– TDfeaturessuperiortoFDfeatures.– FDfeaturescomputationally morecomplex, lessaccurateclassification.– WhataboutTD+FDfeaturevector?

• BetterthansingleTDfeaturevector.

Time-frequencyDomain(TFD)Features

• Describesignalinbothtimeandfrequencydomainssimultaneously.

• ComputationallymorecomplexthanTDfeatures.– Implemented withfastalgorithms.– Showntobecapable ofmeeting real-time requirements insEMG classificationusingappropriatedimensional reductionandsegmentation techniques.

• Yieldhighdimensionalfeaturevectorthatrequiresdimensionalityreductiontoincreasespeedandaccuracyofclassification.

Time-frequencyDomain(TFD)Features

• TFDfeaturestypicallyusedinsEMG classificationinclude:– ShorttimeFouriertransforms(STFT)– Discretewavelettransform(DWT)– Continuouswavelettransform(CWT)– Wavelet packettransform(WPT)

• STFTcannotincreasebothtimeandfrequencyresolutionsimultaneously.

• CWT,DWT,WPTovercomethissomewhat.– Goodfrequencyresolution andpoortimeresolution inlowfrequencyband.– Badfrequencyresolution andgoodtimeresolution inhighfrequencyband.– DWTmostpopularduetobeingcomputationally moreefficient

Spatialdomain(SD)features

• Regionsofmuscleactivateddifferentially.– Dependsonthepositionofthejointanddurationandstrengthofthecontraction.

• HD-sEMG measurementshavemadeitpossibletoextractspatialinformationfromsEMG recordings.

• Improvedifferentiationbetweenposturesandforcelevels.– ProvideinformationaboutMUAPsandload-sharingbetweenmuscles.

• Recentlyadopted,fewstudiesinvestigatedanddesignedSDfeatures forthispurpose.

MyoelectricControlStrategiesinsEMG

• 2controlstrategies:1. Patternrecognition based.2. Non-patternrecognition based.

• Patternrecognitionbasedcontrolusesclassifiers.– Mapfeaturevectorstodesiredcommands– Allowsmoreversatilecontrolscheme thannon-patternrecognitionbased

control.

• Nonpatternrecognitionbasedcontrolcomparesvalueofsinglefeaturetopredeterminedthreshold.

Patternrecognition-basedcontrol• Reliesonassumption:

– Classifieriscapableofrecognizinginputvaluesintroducedintrainingsessionandassigneachinputvaluetooneofagivensetofclasses.

• Input:featurevectorscalculatedofthesEMG signal.• Classes:

– Differentcontrolcommandssenttothedevice.– Different“poses”madebymuscles.

• Comparativestudiesshowmostclassifiershavesimilarclassificationaccuracy.– Usingappropriatefeaturesetsandsufficientnumberofchannels.– Bestclassifiersarefasttotrain,simpletoimplement,meetreal-timecontractions.

• LDA,SVM,HMM• LDAmostcommonlyused.

Non-patternrecognition-basedcontrol

• Simplestructure.– Limitation: onlysomanycommandscanberecognized.

• Showntoprovideintuitive interfacefornavigationmenus,wheelchairs,andassistiverobotics.– Allofwhichrequirefewercommandsthanotherapplications, likeprosthesis.

• Methodsincludedinnon-patternrecognition-basedcontrol:– Proportionalcontrol– Onsetanalysis– Finitestatemachines (FSM)

Finemuscularactivityrecognition

• Nowweknowthe“magic”behindmuscularsignalsandEMG.– Wecanbegintotalkaboutwhatweareabletoexplorewiththisknowledge.

• Fineactivityrecognitionisnotyeta“hottopic” likeEMGcontrolsandregularactivityrecognition.– Easytotell ifsomeone isstandingorsitting.– Wedomanyactivities withourfingers.

• Typing,writing,eating,etc.• Manyneuromusculardiseasesshowsymptomsatthefingerlevel

– Parkinson’s, Muscular Dystrophy, MS, Carpel Tunnel

Overallstructure

• Weneedtobeableto:

Collectdata

ExtractFeatures Classify Predict

finger

Myo DataRecognition

• Whiletypingina“naturalisticsetting”, isitpossibletoextractwhatfingersusersareutilizing?– Allowsforfingermovementanalysis– Unobstructive, uninvasive.

• UsingstandardQWERTYkeyboard– Standardfingerpositioning

• Usingtwo MYOarmbands– Collecting EMGdataat200Hz– Collecting IMUdataat50Hz

Myo DataRecognition

• EMGFeatures– TDFeatures:

• MeanAbsoluteValue(MAV)• WaveformLength(WL)• SlopeSignChanges(SSC)• ZeroCrossings(ZC)

– TFDFeatures:• EMGShort TimeFourierTransform• IMUShortTimeFourierTransform

Myo DataRecognition

• IMUFeatures– Averageofacceleration forx,y,zaxes.– Standarddeviation ofeachx,y,zaxes.– AverageAbsoluteDifference.– AverageResultant Acceleration.

• Thesefeaturesdon’ttellusmuchaboutaccelerationchangeswhentyping,sincethemovementsaremostlyalongoneaxis,andareverysmallinthegrandscheme.– Otherfeaturesstillunderconsideration.

Conditions

• WearingaMyo armbandoneacharm,usertypesoutanarticle.

• Typingspeedhastobeslow,inordertogetlargerwindowsizes.– Approx.20words/minute orless

• Userholdsthekeydownwhenpressedforapprox.1second.

“Traditional”classification

Results:• 50%-60%accuracyinidentifyingwhichfingerwasusedatnormal typingspeed(~60wpm).

• ~76%- 80%accuracyatslow typingspeed(~20wpm).• Why?

Hierarchicalclassification

Windowdata

• “chop”dataupintowindows

Typingactivity

• Classifywhethertheuseristypingornot.

Fingersused

•Onceweknowanactivityisoccurring,wecanextrapolatefingers

Collectdata

ExtractFeatures Classify

Hierarchicalclassification• Firstcheckiftypingactivity ishappening.• Ifyes,classify finger.Results:• Typingornottyping:

– Majorityvote.• Windowchoppedintosubwindows.• Subwindows areclassified.

– ~100%accuracyusingLDA,Perceptron,andSVM regardlessoftypingspeed.• Fingerclassification

– 55%-60%accuracyinidentifyingwhichfingerwasusedatnormal typingspeed(~60wpm).– ~78%- 82%accuracyatslow typingspeed(~20wpm).

• WHY?!?!?!

Culprit

• MuscleActionPotentialatsupinators/pronatorsrateis~500Hz

• Myo claimssamplingrateis~200Hz, actuallycloserto~150Hz.

• Windowsizesmaystillbetoosmall.Classificationaccuracysuffers.

Thankyou

TDfeaturevectorsusedinsEMG interfaces