Haptic Touch Screens for Mobile Devices: Feedback ... hui/mobile/papers/Muller-paper.pdf · Haptic…

  • Published on
    02-Aug-2018

  • View
    212

  • Download
    0

Embed Size (px)

Transcript

  • HapticTouchScreensforMobileDevices:

    Feedback&Interaction

    in

    HapticCommunicationandInteractioninMobileContexts

    UniversityofTampere,FI

    11/12/2008

    SebastianMller

  • TableofContentsIntroduction..........................................................................................................................................2Theory..................................................................................................................................................3

    TheHapticSense.............................................................................................................................3WhatInformationtodeliver.............................................................................................................5Howtodeliver..................................................................................................................................5

    Practice.................................................................................................................................................6Summary..............................................................................................................................................9References.............................................................................................................................................9

    IntroductionTouchscreensbringagreatamountofdirectnessintothecommunicationbetweenmanandmachine.

    Buttonscanbedirectlypressedwheretheyareinsteadofmovingamouseandtextcanbewritten

    whereitissupposedtobe,justtonametwoexamples.Eventhoughusageofatouchscreenrequires

    traininganditstechnologydoesbynomeansreplacecompletecomputerarchitectures,touchscreen

    technologycanbehelpfulinavarietyofways.

    Thesameholdsforhaptictechnologiesintouchscreens.Hapticactuatorscanpresentusersafeeling

    theymightbefamiliarwithorcanrelateto.Severaltypesofuserscanbeaddressed:Computer

    savvyuserscaneasilyintegratethenewinformationintotheirworking(orgaming)habits.Haptic

    feedbackisinmanycasesfasterandinanycasemoredirectthanvisualfeedback[10,11].People

    wholivewithandaroundtechnologicaldevicesbenefitfromthevisualindependentandsometimes

    evenattentionindependentaspectsofhaptics.Computernovicesmayappreciatethephysical

    metaphorthatisconstructedwhenpressingabuttonactuallyfeelslikeabuttonandanerroris

    sensible.Thereisathresholdtoovercomewhenpeopleareunfamiliarwithdevices,theirusageand

    functioning,butadditionalinformationchannelscanonlyhelppeopletolearnintuitively.Often,

    thesenovicesareolderpeoplethathavetodealwithotherproblemsthantheaveragetechnical

    gadgetaudience.Visualimpairmentmeansalotinthisdomain.Therearehardlyanytechnical

    devicesthatdonotrequirereadingascreenorblinkingLEDerrormessages.Touchscreensmake

    mosttactileinterfacessuchaskeyboardsredundant.Obviously,itmakeslittlesensetousehaptic

    touchscreensifuserscannotreadanyscreen,buthapticallyenhancedGUIelementsareonlyata

    veryearlystageofinvestigation.Welladaptedfeedbackmaydrawlevelwithvisualinteractionsome

    day.

  • Oneofthedrivingforcesoftouchscreensandonscreenkeyboardsaremobiledevicesand

    applications.Devicesthatrequiretofitintothepocketdonotallowalotofspace,andcomparedto

    themanytechnicalfinessesmodernhandhelddevicescarry,akeyboardwithenoughkeysthatare

    largeenoughtobeusedbyanyusertakesexcessiveamountsofspace.Formanypeople,the

    keyboardisthemostimportantinputdeviceandcannotbereplacedbyasmoothsurfaceofvirtual

    buttons.Ithasalreadybeenshowninexperimentsthattactilefeedbackcansignificantlyimprove

    entryspeedanderrorrates[10,11]uptoempiricalvaluesexperiencedwithnormalkeyboards.

    Mobileapplicationrequiresnotonlythoughtfuluseofspace,butalsocontextsensitivity.Asmobile

    devicescanbetakenanywhere,theycanbeusedinanysituationandthereforenotrelyonsingular

    modesofinteractionorfeedbackchannels.Opposedtothestatickeyboardoneverycomputer,on

    screenkeyboardscandelivermoreinformationthanjustthepositionandstatusofakey,and

    differentthanthevibrationalarmofcommonmobilephones,tactileactuatorshaveamuchbroader

    rangeofactions,flexibleinfrequency,amplitudeandspace.Hapticscreensarenottheanswertoall

    contextsensitiveinteractionquestions,buttheyoffermorereliableinputthanplainonscreen

    keyboardsonsmallerspacethanphysicalkeyboards.Furthermore,ifthehapticcommunicationis

    limitedtofingertip/penscreeninteraction,designersdonotneedtoworryaboutappropriatenessof

    thefeedbackwhichisofteninquestionwhenitisappliedtootherbodyparts[12].

    Apartfromsteppingintothebreachwhereotherinteractiontechniquesfail,hapticfeedbackon

    touchscreensofferawidevarietyofnewapplicationsandinteractionsthatwerenotpossiblebefore.

    SuchasGUIelementsarehapticallyenhanced,socanbasicallyanything.ThehapticTBar[9]isa

    newGUIelementthatwasspecificallydesignedforhaptictouchscreens.Whenthistechniquesteps

    outofvibrotactilemotors,spatiallyarrangedhapticactuatorsfedbysoftwarecouldmakeanything

    tactile.

    TheoryHapticfeedbackontouchscreensisarathernewfieldofstudy,andthereforemanyaspectshavenot

    beenexplored.Thischapterdealswiththephysicalpossibilitiesandespeciallylimitsofthehuman

    touchsenseaswellasthetheoreticalnatureoftouchscreeninteraction.

    TheHapticSense

    Thehapticsensehasbeenexploredforhumancomputerinteractionforamuchlongertime.It

    becamepopularandwellknownasthevibratingalarmincellphones.Comparedtoothersenses

  • oftenusedinhumancomputerinteractionsuchasvisionandaudition,itisconsideredmoredirect.

    Thisadvantageisoftenpaidforwithlackofprecision.Embeddedinadevice,hapticactuators

    usuallycarrytheirmovementovertothewholedevice.However,thefingertipsareoneoftheparts

    ofthehumanbodywiththehighestamountofsomaticsensors.Consideringthatthefingertipsare

    alsothemostprecisetoolyouhaveandtheyareinusewhenworkingonatouchscreen,theyarethe

    perfectchannelforhapticinformation.

    Undercertainconditions,hapticinformationcanbeingesteduptofivetimesfasterthanvisual

    information[1].Thisismostlyduetothefactthathapticinformationgothroughlessprocessing

    stagesthanvisualinformation.Thisindicatesthathapticsdonotmeansomuchtousasvisualsdo.

    However,hapticfeedbackisnotintendedtoreplaceorsimulatevisualinformation.Instead,thetwo

    infactanysecondsensecansupportandinfluenceeachother.TikkaandLaitinen[2]foundthat

    audio[...]biasestheperceptionandincreasestheperceivedstrengthofthehapticstimulus.

    Sincetheuprisingofvirtualkeyboards,thehapticcapacityoffingertipshasbeeninvestigated

    intensively.Justnoticeablemovementsanddispleasingsensationsareimportantlandmarksinhaptic

    feedbackdesign.BarfieldandFurness[]foundthatvibrotactilesensations2millimetresapartfrom

    eachothercanbeidentifiedastwodistinctsensations.Furthermore,ashiftofpositionofa

    vibrotactileeventassmallas0.2mmcanbenoticed.Itisimportanttorememberthattheseresults

    werefoundunderlaboratoryconditions.Touchscreens,however,playanimportantroleinthedesign

    ofmobiledevices(duetothelackofspaceandreplacementofphysicalkeyboardswithvirtual

    keyboards)andconsequentlycannotbeusedasdefinitevaluesasthecontextandenvironmentin

    whichthesedevicesareusedishighlyvariable.

    Oneaspectofhapticfeedbackthatdoesnotapplyto

    thefingertipsistheprivacyeffect:Hapticfeedback

    thatisappliedtootherpartsofthebodysuchasthe

    back,stomachorfeetneedstobemorefinetuned

    forthreereasons.First,manypartsofthebodydo

    nothaveasmanysomaticsensorsasthefingertips

    have.Hapticeventsthereforeneedtobestrongeror

    spatiallymoredifferentiated.Second,mostbody

    partsarenotusedasmuchasthefingersforinput

    (whichis,fromanevolutionaryperspective,the

    reasonwhytheydonothaveasmanyreceptorsas

    thefingers)andtouchiseasierconsideredsensual.Third,touchscreensusuallydisplay

    Illustration1:Homunculusandbrain[3]

  • informationondemand,meaningthatitrequirestheuser'svisualattention.Whentouchscreen

    devicesarecarriedaroundunused,thescreensareusuallyturnedoff,andsoisthehapticfeedback.

    Thedeviceswithhapticfeedbackthataresoldorwillbesoldinthenearfutureonlyrespondtothe

    user'sinput.Futuretouchscreensmightdisplayhapticinformationindependentfromuserinput,but

    sofaritisdifficulttoseeifthesefallundertheprivacyeffectatall.

    WhatInformationtodeliver

    Thehapticinformationspaceconsistsofseveraldimensions.Mostofthemareunusedincommon

    humancomputerinteractionbutneverthelesshighlyusefulwhendesigninghapticicons.Haptic

    iconsaresingleinformationcarryingunitsconsistingofhapticfeatures.Thesefeaturesarespreadin

    thefeaturespaceofthehumansomaticsense.Theprinciplesofhapticicondesignbecomemore

    importantthemoreinformationistobedelivered.Questionsthatariseare:Whatisthesmallest

    perceivablehapticevent?Howmanydifferenttactileeventscanbedifferentiated?Whatisthe

    qualitativeperceptionofspecificfeatures?

    Koskinen,Kaaresoja,andLaitinen[14]conductedanexperimenttofindthemostpleasanttactile

    feedback.Theirresultsrevealalotaboutsubjectiveperception,andmostoftheirresultscanbe

    foundinotherpublicationsaswell:Theparticipantsoftheirstudyfoundthefeedbackfromapiezo

    actuatorslightlymorepleasantthanfromavibrationmotor.Also,themostpleasantstimulihadthe

    samecurrent(46mA),butdifferedmostlyinmaximumdisplacement.Furthermore,vibrationmotor

    stimuliareperceivedmostpleasantbetween13and19milliseconds.Althoughtheseresultscannot

    begeneralizedtoalltouchscreenarchitectures,theauthorsconsiderthemindicatorsforother

    technologies.Inanycase,tacticallyenhancedtouchscreensareperceivedbetterthansimple

    touchscreens.

    Aninterestingobservationaboutthedifferencebetweenactualandperceivedfeedbackcanbemade

    withtheAppleInc.'sMightyMouse1:Whenpluggedin,asmallspeakerinsidethemouseplays

    clickingsoundswhenabuttonispressedorthemousewheel(ScrollBall)ismoved.Iftheenergy

    sourceiscutoff,buttonsandwheelremainsilent.

    Hapticfeedbackontouchscreenshasalreadybeentestedonanumberofapplications.Kaaresoja

    andothers[4]havedemonstratedtheapplicabilityoftactilefeedbackontextentry,textselection,

    scrollinganddraganddropapplications.Theyenhancedsimpleclicks(usuallyleftmousebutton

    clicks)witharesponsefromapiezoelectricactuator.Moreover,draggingthestylusoverthescreen,

    asintextselectionordraganddrop,createdseveraldifferentfeedbacks.Inotherexperiments,

    rotationandzoomingwereenhancedwithatactilefeeling.Italsoseemsconvenienttodifferentiate

    1 http://www.apple.com/mightymouse/

  • betweenleftandrightaswellassingleanddoubleclicks,whenthesystemallowsthis.Clicking

    usuallyreferstotheactivationofGUIelementslikebuttons.Oneofthemosteagerlyawaited

    applicationsofhapticfeedbackisthetactileonscreenkeyboard.Insomedevicesthatarealready

    soldcomprehensively,theselectionofabuttonissimplyconnectedtooneshorthapticevent.If

    properlydesigned,thiseventactuallyfeelslikethepushofabutton.Aswewillseelater(see

    Chapter4),therealreadyexistmuchbettertechniquesofhowtoimitatethebehaviourofaphysical

    keyboard.

    Otherinformationthatcouldbetransferredthroughhapticfeedbackarethenotificationofevents

    thatruninthebackground,time(asthesystemtime,timespentonataskorremainingtimeina

    countdown)orthestrengthoftheinput,ifthetouchscreenispressuresensitive.

    Howtodeliver

    Onaccountofthenoveltyoftheconcept,mostexperimentsconcerningtheperceptionofhaptic

    feedbackaremadewithuniqueprototypesofstandardtouchscreendevicesupgradedwithexternal

    actuators.Thereforenowidelyadoptedrulesaboutwhatexactlyspecifichapticeventsshouldfeel

    likeexist.Generally,hapticpulsesaredefinedbyfrequencyandamplitude.Thehigherboth,the

    strongertheperception.TernesandMacLeanhaveshownthattheperceptionoffrequencyis

    stronglysubjectiveandmightrequireindividualhapticdesign,whereasamplitudeisthestrongest

    differentiatingfactor[5].

    Additionally,hapticiconscandifferinlength,suchasmanycellphonesdifferentiatebetweencalls

    andtextmessages,andspace.Wherevermoresophisticatedhapticfeedbackisrequired,higherlevel

    Illustration2:Interactiontechniqueswithahandhelddevice

  • featuresconsistingofdifferentlowlevelfeatures,suchasrhythm[5],emerge.

    Thedesignofhapticfeedbacknotonlydependsonthephysicallimitsofsensorsandactuators.

    Especiallyinmobiledevices,theappropriatenessofsignalsdependsonthecontextofusage.The

    followingisalistofdesignissuesthatmayarisefromotherdesigndecisions.

    Touchscreentechnologiesarevarious.Somearepressuresensitiveandreactive,some

    requiresolidsurfaces.Mostimportantly,somecanbeusedwithastylus.Inthatcasethe

    hapticeffectneedstobetransferredthroughthestylus.Othersensors,suchascapacitive

    sensors,requiretheuseofskin,i.e.fingertips,towork.

    Devicelayout.Dependingonthelayoutandsizeofthewholedevice,usersmightuseone

    handtoholdthedeviceandtheotherhandforinput.Otherdevicescanbeheldwithboth

    handsandinputhappenswiththethumbs.Theuseofastylusmightindicatetheneedfor

    higheraccuracythantheuseoffingertips.Inthesecases,theusermightwanttoplacethe

    deviceonaflatsurfacelikeatableorrestthewristonthedeviceforstabilization.

    Vibrationsonhardsurfacescancreatealotofnoiseinappropriateundercertainconditions.

    Distractions.Subjectiveperceptionofhapticeventsmaybedifferentdependingonthe

    surrounding.Avibrationthatseemsappropriateunderlaboratoryconditionsmightnotbe

    perceivedatallinapackedsubwaywheretheuserissubjecttothephysicalforcesofthe

    vehicleandfloodedwithothersensoryperceptions.

    Task.Thetaskthatshallbeperformedonthedevicecaninfluenceanystageofdesign.

    Dependingonhowspecializedadeviceis,hapticfeedbackmaybespecializedaswell.

    Hapticfeedbackataconstructionsiteneedstobemuchstronger(ifnotinappropriateatall)

    thaninalectureormusichall.

    PracticeInthischapter,somealreadytestedtechniqueswillbepresented.Thedesignissuesintheprevious

    chapterswereoftheoreticalnature,thefollowingaremorepracticallyorientedissues.

    In2001,FukumotoandSugimura[13]publishedtheirworkaboutattachinganelectrictovibration

    transducertoaPDA.Theyalsoexploredthepossibilitiesanddifficultiesofdesigninghaptic

    feedbackfordifferentsituations,forexamplecomparingahandhelddevicetoalargepublic

    informationterminal.

    In2003,PoupyrevandMaruyamapublishedadesignandimplementationofatactileinterface[6].

  • TheyembeddedaTouchEngine[7]inthetouchscreenofaSonyCliPDA,wheretheactuatorswere

    placedatthesidesofthedisplay.Thedesignofthedeviceallowedtheauthorstoplacetheactuators

    betweenthecoveringglassplateandtheTFTdisplay,thereforerequiringlessenergy.Althoughthe

    displacementoftheglassmeasuredonly0.5mm,averystrongtactilesensationwasreported.The

    authorsdescribethetactilefeedbackintheirdeviceaslocalized,inthesensethatonlytheglass

    undertheuser'sfingerorstylusmoves.Transferofthemovementontotherestofthedevicewas

    inhibitedwithsilicondumpersoneachsideoftheactuators.

    Piezoelectricactuatorsaremadeofpiezoceramiclayers.InthecaseoftheTouchEngine,eachof

    theselayersisonly0.28mthickandcanbeassembledinanysizeandquantity.Thedisplacement

    doesnotdependonthethicknessoftheactuatorbutontheappliedvoltage.Iftheactuatorswere

    thicker,theywouldseparatesurfaceanddisplaywhichresultsintheparallaxproblem.Theparallax

    problemdescribesthatthearrangementoftwoormoreobjectscanappeardifferentfromdifferent

    perspectives.Thefurtherawaydisplayandsurfaceofatouchscreenarefromeachother,themore

    theuserhastopayattentiontolookatthedisplayatarightangle.Thisphenomenencanoftenbe

    observedonATMmachines,astheirscreenisprotectedbyathicklayerofglass,whichmakesthe

    displayedareafurtherawayfromthehardwarebuttonsonthesidesofthescreen.Whenthescreen

    thenpointstooneofthebuttons(toasktheuserforaselection),theassignmentofdisplayareato

    buttonscanbedistracteddependingontheuse...

Recommended

View more >