Let Your Body Move: A Prototyping Toolkit for WearableForce Feedback with Electrical Muscle Stimulation

Max Pfeiffer, Tim Duente and Michael RohsHuman-Computer Interaction Group University of Hannover

max,michael@hci.uni-hannover.de

ABSTRACTElectrical muscle stimulation (EMS) is a promising wearablehaptic output technology as it can be miniaturized consider-ably and delivers a wide range of haptic output. However,prototyping EMS applications is challenging. It requires de-tailed knowledge and skills about hardware, software, andphysiological characteristics. To simplify prototyping withEMS in mobile and wearable situations we present the LetYour Body Move toolkit. It consists of (1) a hardware controlmodule with Bluetooth communication that uses off-the-shelfEMS devices as signal generators, (2) a simple communica-tions protocol to connect mobile devices, and (3) a set of con-trol applications as starting points for EMS prototyping. Wedescribe EMS-specific parameters, electrode placements onthe skin, and user calibration. The toolkit was evaluated ina workshop with 10 researchers in haptics. The results showthat the toolkit allows to quickly generate non-trivial proto-types. The hardware schematics and software componentsare available as open source software.

Author KeywordsEMS; electrical muscle stimulation; toolkit; prototyping;haptic feedback; force feedback; wearable; mobile.

ACM Classification KeywordsH.5.2. Information Interfaces and Presentation (e.g. HCI):User Interfaces—Haptic I/O

INTRODUCTIONInterest in haptic feedback has been long-standing in human-computer interaction. Haptic feedback provides a direct wayto interface with the user’s body. Haptic sensations areevoked by very different phenomena, ranging from skin con-tact with objects to movement of limbs through acceleration.In human-machine interfaces movement may be caused, e.g.,by force-feedback devices and exoskeletons [5]. In virtualand augmented reality haptic feedback can augment interac-tions with virtual objects and make the experience more real-istic and immersive.

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than theauthor(s) must be honored. Abstracting with credit is permitted. To copy otherwise,or republish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee. Request permissions from Permissions@acm.org.

MobileHCI ’16, September 06 - 09, 2016, Florence, Italy.Copyright is held by the owner/author(s). Publication rights licensed to ACM.ACM 978-1-4503-4408-1/16/09$15.00DOI: http://dx.doi.org/10.1145/2935334.2935348

Figure 1. Overview of the EMS toolkit. Left: mobile and wearable de-vices that connect to the EMS control module; middle: custom controlmodule and off-the-shelf EMS generator; right: actuation of muscles.

However, while in general computing technology has a ten-dency of getting smaller, requiring less power, and becomingmobile and wearable, haptic output technologies do not easilyfollow this trend. Haptic technologies often require actuators,mechanics, and considerable amounts of energy. Therefore,many haptic methods are bound to the lab and special usecases, such as surgical systems [22]. These technologies areless suitable for mobile, wearable, and everyday life usage.

A number of efforts have tried to miniaturize haptic devices,e.g., by using electrical muscle stimulation (EMS) [15]. EMSactuates the muscles directly rather than through external ac-tuators. The effect ranges from a small tingle to limb move-ments. EMS applies a small current to particular muscles toelicit a tactile sensation or movement. EMS, also known asfunctional electrical stimulation (FES), has a long history inrehabilitation, massage, and fitness applications [31].

Nowadays, using EMS as a haptic feedback method requiresextensive background knowledge. This includes knowledgeabout the design of circuits that generate or modify EMS sig-nals, the choice of EMS parameters to achieve a certain ef-fect, the placement of the electrodes, as well as the responsebehavior of different muscles. A lot of initial effort is typi-cally spent in building custom hardware and software com-ponents [15, 25, 33]. This raises the entry hurdle for HCI re-searchers who want to use EMS as a haptic feedback method.Easy and fast prototyping is hardly possible.

1

In other domains, such as cross-device interfaces, toolkitsand frameworks like [8, 10] help to reduce the initial effortand enable researchers to quickly focus on HCI-related ques-tions, e.g., regarding novel application scenarios and inter-action techniques. To simplify the use of EMS in HCI re-search we present the “Let your Body Move” EMS prototyp-ing toolkit (Figure 1). It comprises the schematics of a hard-ware control module, software modules, sample applications,and a robust communication protocol for EMS parameters.We discuss EMS parameters and electrode placement. Wealso define the steps of a prototyping process.

A major concern with EMS research is safety. An importantdecision regarding the toolkit was thus to design it aroundexisting massage/EMS/TENS devices as EMS signal gener-ators. The toolkit was tested with four different commercialdevices. A second fundamental aspect was to make the sys-tem as small and unobtrusive as possible to be able to used itin wearable applications. Moreover, the system is designedto easily connect to a wide range of mobile and wearabledevices, such as tablets, mobile phones, and smartwatches(Figure 1). This is achieved by wireless communication overBluetooth low energy (BLE). The output of the EMS device issent to an Arduino-based control module. The control mod-ule manipulates the EMS signal by modifying the intensitiesand on/off-times of two EMS channels. For safety reasonsthe control and communication circuitry is galvanically iso-lated from the EMS signal circuitry, which is connected to theuser’s body. The EMS signal is switched off when the con-nection gets lost. We provide a number of prototyping apps,which run on Android-based tablets and mobile phones andallow to easily conduct Wizard-of-Oz studies. Moreover, weprovide several example applications that may serve as a basisfor custom interfaces to EMS functionality, such as a runner’sapp that provides EMS feedback at particular points on thetrack. The circuit schematics and Arduino code of the controlboard, the Wizard-of-Oz control applications, and the exam-ple applications are provided as open source software1. Giventhe circuit schematics and the parts list, the fully assembledPCB board can be ordered from several manufacturers.

RELATED WORKWe begin by reviewing existing prototyping toolkits and plat-forms. We then focus on EMS basics and physiological back-ground. Finally, we discuss work that successfully uses EMSto generate haptic feedback.

HCI and Haptic Prototyping ToolkitsPrototyping toolkits help to make new technologies accessi-ble and to focus on investigating new concepts and interactiontechniques. For example, Houben and Marquardt [10] presentWatchConnect, a prototyping platform for smartwatch cross-device applications. It simplifies investigating cross-deviceinteraction between watches and desktop systems.

FeelSleeve [35] are tactile gloves that are attached to the backof a tablet to provide vibration effects based on the story theuser is reading. Yannier et al. [35] show that tactile feedback

1Let Your Body Move ToolKit https://bitbucket.org/MaxPfeiffer/letyourbodymove

increases comprehension and memorization. Pohl et al. [27]present a toolkit for simulating different surface structureswith a tangible object. Ledo et al. [14] present the HapticTabletop Puck, which simulates haptic feedback by changingthe friction of the device. They simulate different properties,such as softness, depending on the underlying texture. Ledoet al.’s toolkit covers the hardware-specific implementationwith different layers of abstraction. Similar feedback is gen-erated by TeslaTouch [1] and FingerFlux [34].

On the haptic and force-feedback side Brave and Dahley [3]present a device for haptic remote communication. Ha etal. [9] describe a haptic prototyping system for a single dial.The hardware prototype provides several patterns to turn thedial. They conclude that the system can be used for hapticprototyping in automotive, medical, and gaming contexts.

WoodenHaptics [8] is a toolkit that implements a device sim-ilar to the Phantom Desktop device. It is an open source hard-and software toolkit that allows designers to investigate highfidelity haptic feedback. The WoodenHaptics toolkit can beused for haptic prototyping with a broad range of users. How-ever, this device cannot be used in mobile contexts and has alimited interaction range.

Most current haptic toolkits focus on very specific forms ofhaptic feedback or on specific interactions, such as dials [9],rolls [3], or sliders [32], or they are stationary [8].

To our knowledge, we present the first EMS-based opensource prototyping toolkit. It is suitable for mobile and wear-able interactions, can easily be connected to typical mobiledevices, and is able to deliver a wide range of haptic feedback– from light vibrations to moderately strong movements.

Actuation and Electrical Muscle StimulationIn rehabilitation, massage and fitness applications, EMS hasa long history [36]. Strojnik et al. [31] support patients whilewalking. Lyons et al. [21] correct foot drop problems of palsypatients by lifting up the foot with EMS signals while walk-ing. The NESS Handmaster [11] and the Bionic Glove [28]support grasping of physical objects. Significant researcheffort went into restoring gait, which requires to selectivelystimulate multiple muscles [2].

Electrical muscle stimulation has already been investigatedextensively in HCI. PosessedHand [33] implements fine-grain control of multiple fingers, e.g., for learning an instru-ment. In Muscle-Propelled Force Feedback [15] an EMS de-vice attached to the back of a mobile phone is able to driverealistic forces in a gaming context. Lopes et al. [17] haveshown how to shrink down an eyes-free haptic I/O device tothe size of a wearable bracelet. Pfeiffer et al. [23] propose ac-tuated navigation and modify the walking direction of pedes-trians. Pfeiffer et al. [24] use EMS for immersive interac-tion with public displays. EMS has also been used to providefeedback in gaming [13], mixed reality [6], and virtual real-ity [16] applications. In virtual environments EMS has beenused to support virtual target selection in 3D [26]. EMS hasbeen used to display various object properties when touching,grasping, and punching virtual objects [25]. Finally, Lopes etal. [18] extended real objects with additional affordances.

2

EMS is quite versatile with respect to the form of the signalas well as the placement of the electrode pads and the result-ing movements. We believe that there are still many aspectswhich have not been considered yet and that there is partic-ular potential in areas like learning movements, recognition,and recall. A prototyping toolkit, such as Let Your Body Movecan help to explore this potential.

EMS PROTOTYPING PROCESSBefore considering to use the EMS prototyping toolkit in aparticular project, it is important to clarify if EMS fits therequirements for an envisioned scenario. The specific capa-bilities and limitations of EMS have to be considered. Thedesign space of EMS is large, both in terms of the kind andrange of effects it can create. The EMS design space and therelevant parameters are discussed in the next section.

Using the proposed toolkit requires an initial one-time effort,consisting of building the EMS control module (using thegiven schematic) and installing the control software on theArduino. Each control board can handle two EMS channels.If more channels are needed, multiple instances of the controlmodule have to be built.

For the actual design process we recommend the followingsteps. This process is based on our own extensive experiencein prototyping EMS-based haptic feedback. We assume thatthe design starts with an initial idea regarding an applicationscenario, such as a mobile application giving haptic feedbackto runners, or regarding an interaction technique, such as giv-ing haptic feedback in response to grasping virtual objects ina VR environment. The steps are:

1. Refine the role of haptic feedback in the application sce-nario or interaction technique.

2. Determine the parameters of the EMS design space toachieve the intended effects (EMS parameters, electrodeplacement, calibration requirements).

3. Install the Wizard-of-Oz control app on the experimenter’sdevice (e.g., a tablet) and conduct a Wizard-of-Oz study.

4. Taking into account the lessons learned in the Wizard-of-Oz study, implement a custom prototype for the user’s mo-bile device (mobile phone or smartwatch) that communi-cates with the EMS control module and logs events. Run adetailed study to evaluate the desired phenomena.

5. Iterate (refined prototype, Wizard-of-Oz study on specificaspects, EMS parameters and electrode placement).

The toolkit may be used as-is in brainstorming sessions inorder to inspire Wizard-of-Oz prototypes. The toolkit helpsin the quick production of working prototypes. We used thisapproach in a workshop setting (reported below), but considerit the less common prototyping variant.

When the details of the role of haptic feedback in the envi-sioned scenario or technique have been clarified, the EMSdesign space needs to be considered. An example scenariowould be a wearable application for joggers that actuates therunner’s hand left or right to indicate directional turns. Im-portant aspects to achieve the desired haptic effects concern

the required EMS signal parameters, the placement and sizeof the electrodes, and user-specific calibration. The signal pa-rameters include required and acceptable intensity levels andsignal patterns (e.g., rhythms of movements to communicatea certain kind of information). These steps are in many casessufficient prerequisites for an initial Wizard-of-Oz study thathelps to validate the feedback concept or to quickly try outdifferent feedback design ideas. The Wizard-of-Oz study caneasily be controlled from the experimenter’s Android phoneor tablet in Bluetooth range of the user. However, this initialstudy already requires careful calibration of signal strengthand electrode placement, as EMS strongly depends on thephysiological characteristics of individual users.

The lessons learned can then be incorporated in the design ofa custom software prototype for the user’s device (e.g., thesmartwatch of a jogger). A number of existing applicationsof custom prototypes may serve as a starting point for newsoftware prototypes. The sensors of the user’s device may beused to log specific events and reactions to haptic feedback.This setup can serve as the basis for a more extensive userstudy that aims to evaluate specific phenomena. Of coursethe design process allows for iterating on the prototype, spe-cific aspects or new ideas for haptic feedback, as well as themodification of EMS parameters and electrode placement.

New constraints or requirements may arise, such as the needfor another form factor or additional EMS output channels.While additional channels can easily achieved by addingan additional EMS control module and EMS signal source,changing the form factor may call for deeper a modificationof the hardware design. We consider this beyond the scope ofthe prototyping toolkit as we focus on simplifying the designprocess for HCI researchers, and because the existing formfactor does not constrain the user severely. For example, asthe system uses Bluetooth LE, it can communicate wirelesslywith an unmodified smartwatch.

EMS DESIGN SPACEEMS feedback ranges from a small tingling – similar to thatgenerated by a vibration motor – to the contraction of a mus-cle, which results in the movement of a limb. A weak electri-cal current (typically 10-30 mA) is generated by the EMS de-vice and applied to the user [7]. A lower current only createsa tactile sensation that is noticeable at the place of the elec-trode on the skin. The sensation arises as a side effect of thestimulation of nerves in the skin. The feeling changes overtime due to habituation effects. A stronger current crossesthe skin, reaches deeper tissues, and activates motor nerves,which ultimately results in limb movement. Stimulating themotor nerves leads to a contraction of muscle fibers. The con-tracting muscle pulls a tendon, which is connected to a jointand moves the limb. Current that is higher then 40 mA [7]stimulates the pain receptors.

EMS ParametersTo achieve comfortable haptic feedback the signal parame-ters need to be adapted to the particular user, muscle, anddesired effect. Commodity EMS devices usually output asquare wave signal. Some medical EMS devices have optionsto change the signal form, e.g., to sine, square, or triangular.

3

The intensity of the feedback signal is controlled via the am-plitude, the pulse width (typically 30-800 μs), and the pulsefrequency (typically 1-150 Hz) [30].

Most of the devices allow for setting the amplitude, the pulsefrequency, and the pulse width. For prototyping and Wizard-of-Oz studies a pulse with 100 μs and 150 Hz works reliablyfor many users. For example, we are using the off-the-shelfEMS device Sanitas SEM 43 [29] on program 8 TENS witha pulse width of 100 μs and a pulse frequency of 120 Hz. Toachieve a certain current the amplitude needs to be adjusteddepending on the resistance of the skin, which differs widelyacross users and feedback positions. It also changes withinone user over time. For example, skin resistance changes withskin moisture, which is influenced by sweat.

The EMS signal is applied to the skin via plate electrodes, ad-hesive electrodes, rubber electrodes or vacuum electrodes [7].Conductive gel may be used to decrease skin resistance, but istypically not necessary with adhesive silicone electrodes. Theelectrode size plays an important role in how the current is felton the skin. Smaller pad sizes are preferable as they allow tomore precisely target specific muscles. However, as the cur-rent density (electrical current divided by pad area) increases,nerves in the skin are stimulated more strongly, which lets theuser feel the current more intensely.

Tactile and Force FeedbackTactile feedback is felt at the position at which the current isapplied. The perceived strength depends on the amount ofcurrent, pad size, and density of receptors in the skin. Thetactile feedback can be used itself as silent and invisible feed-back. The signal parameter are similar to force feedback,the intensity is lower and the electrodes does not need to beplaced over a muscle.

However, in this work we are focusing on force feedback. Forevoking a particular movement the pair of electrodes need tobe placed exactly over the muscle that is actuating the tar-get joint. Thus the position of the EMS pads and the posi-tion of the effected movement differ. Some muscles are veryclose together, others overlay each other. The size and theposition of a muscles differ between users. Therefore preciseelectrode placement and a calibration is crucial. Actuatinga single muscle results a single elementary movement. Thiselementary movement may support or counteract a voluntarymovement performed by the user. For example, it might am-plify a grasping motion or slow it down with a counterforce.Elementary movements may be combined to form a gesture.In the following we describe a set of elementary movementsthat can be evoked reliably. Further placements and safetyaspects are described in [4, 29].

Hand up: The extensor digitorummuscle lifts up the hand, as shownon the Figure left. It is connectedthrough a tendon to the upper sideof the hand. When the electrodesare not properly placed on this mus-cle, the signal either actuates the ex-tensor carpi ulnaris (lateral side) orone of the thumb extensors (medial

side, extensor pollicis brevis or extensor pollicis longus),which move the thumb inside. The muscles in the lower armare placed very tightly together and an exact placement is dif-ficult. The effect also depends on the rotation and posture ofthe hand. The skin with the electrodes shift differently thanthe muscles underneath.

Hand down: The pads need to beplaced over the flexor digitorumprofundus, as this muscle pulls thewrist down. An imprecise place-ment also activated flexor digitorumsuperficialis, which bends the prox-imal interphalangeal (PIP) joints,i.e., it lets the upper finger segmentsclaw. The flexor digitorum profun-dus does not actuate the thumb. To

bring the thumb inside during this movement the thumb mus-cle (flexor pollicis longus) needs also be activated. Actuatingall these muscles together results in a fist.

Lower arm up: To lift the lower armup the biceps brachii muscles (ca-put longum and caput breve) needto be actuated. On this larger mus-cle the pad may be placed verti-cally (proximal–distal) or horizon-tally (lateral–medial). The im-age shows the horizontal placement.With the horizontal placement weachieved better actuation results,

because the current crosses the whole muscle. However, thehorizontal placement may result in a tingle in the palm of thehand, in which case the vertical placement should be chosen.

Arm up: The arm is primarily liftedup by the deltoid muscle. When ac-tuating it together with the bicepsbrachii muscles, the arm moves abit forward. To avoid this the shoul-der muscles (infraspinatus and teresminor) can be actuated to pull theshoulder backward a bit. Depend-ing on the user’s fitness the deltoidmuscle fatigues after a few actua-

tions and hardly lift the arm up.

Foot up: Activating the tibialisanterior muscle leads to lifting upthe foot. The electrodes shouldbe placed on the upper part of thelower leg on the outer side of thetibial bone (shinbone) as shownon the Figure on the left. Thelargest effect occurs, when thefoot is swinging in the air. Longerelectrodes that are placed vertically

(proximal–distal) reduce the tactile side effect at the leg.

4

Leg up: When sitting the lower legcan be lifted up (moved ventral–proximal) by actuating the quadri-ceps muscle of the thigh. Thequadriceps is a large muscle that canbe reached easily by surface elec-trodes. This large muscle requires arelatively large current. Therefore,a larger pad should be used. Careshould be taken when actuating this

muscle while walking, as it easily blocks the leg, which maycause the user to stumble.

EMS UsageWe have found the elementary movements described above towork reliably. Many other elementary movements and com-binations of movements are possible [4, 29]. The reliabilityof EMS actuation depends on several factors.

The actuated muscle may change its form or it may moveunder the skin such that the surface electrode is not longerprecisely placed over the muscle. If the electrode is placedon the boundary of the muscle, a periodic oscillation may oc-cur, in which the muscle is repeatedly pushed away from theelectrode while contracting, receives less current, relaxes, andmoves back into the current. These effects need to be takeninto account when elementary movements are combined togestures or if force feedback should be applied during a vol-untary motion of the user.

Moreover, the muscle performance changes over time. De-pending of the size and fitness of a muscle after a set of repe-titions or continued actuation muscle fatigue occurs [12].

Placing the electrodes on tissues with a lower resistance thanthe muscle will bypass the muscle. This can lead to unex-pected effects such as actuating the biceps brachii muscleswhen one electrode is positioned on the lower arm and theother in the crook of the arm. In this case the pads need to beslightly replaced.

Safety IssuesThere are potential medical risks that need to be taken into ac-count when using EMS for generating haptic feedback. Thehaptic feedback designer and also the user have to be awareof these potential risks. Feedback designers and users are re-quired to consult the manual of the EMS device and to followthe safety recommendations carefully. The user’s safety hasthe highest priority!

An important point for safety is that the electrodes must neverbe placed on the front torus near the heart. EMS devices mustnot be used in combination with a pacemaker or by peoplewho have any heart disease. It is recommended that pregnantwoman, people with epilepsy, with cancer, after a surgery,with sensitive skin, or with a skin disease should not useEMS, or only after consulting their physician [29].

Moreover, extremely high EMS signals stimulate the noci-ceptors (pain receptors) [7]. The pain threshold depends onthe user and on the position of application. Therefore, the

maximum EMS level should be calibrated individually andfor each placement.

Reports from the domain of fitness training have shown thatlong activation periods with high EMS levels can result inmuscular fatigue and overtraining of the actuated muscle.

Running EMS experiments typically requires the approval ofa local ethics board. For our workshops at WorldHaptics [19]and CHI [20] on this toolkit and the associated prototypingprocess the participants filled in an informed consent form.No ethics approvals were required by the conferences.

User CalibrationEvery user is different! Because of anatomical differences auser-dependent placement of the electrodes for each muscleis necessary. The perception of signal strength, the maximumcomfortable signal level, and the minimum level at which ac-tuation starts all differ widely across users. The skin resis-tance differs depending on tissue sickness, body fat percent-age, skin moisture, and hair.

Typically, users do not have any prior experience using EMS.A calibration phase should start with a brief introduction ofEMS and a discussion about the safety issues. When upcom-ing questions have been answered the users should be prac-tically introduced to EMS step by step: Initially, a muscleshould be tested that is easy to reach, such as the extensor dig-itorum on the posterior forearm. The expected effect shouldbe described by the experimenter beforehand and the usershould be allowed to adjust the signal intensity level on theirown. This helps to determine the maximum comfortable levelfor a particular user. Then other muscles can be calibratedsimilarly by placing the electrodes and increasing the inten-sity level. The maximum comfortable level for each mus-cle should be recorded and never exceeded during the study.Finally, combined gestures can be tested with the calibratedstrengths.

Reporting EMS ExperimentsImportant information that needs to be reported for repro-ducibility of EMS experiments include the used muscle ormuscle group (with the function/actuation effect of the mus-cle), the placement of the electrodes, as well the types andsizes of the electrodes. Also the used EMS parameters, suchas amplitude, pulse frequency, and pulse width, we well asmeasured current, should be reported per user. Voltage andcurrent should be measured before and after the experiment,since the skin resistance can change.

EMS PROTOTYPING TOOLKITThe EMS prototyping toolkit is a composition of hardware,software, and methods to enable prototyping with EMS asshown in Figure 1. On the hardware side we provide a con-trol module that manipulates the EMS signals of off-the-shelfEMS devices (Figure 2). It manipulates the strength and du-ration of the EMS signals and communicates wirelessly viaBluetooth low energy (BLE). The EMS control module iseasy to build given the circuit schematics and parts list. Theindividual components are controlled by an Arduino Nanoover I2C (Figure 3). We developed a simple ASCII-based

5

Figure 2. EMS control module: Overview of the components and func-tionality.

protocol to modify the signal parameters. On the softwareside we implemented four simple Android prototyping appsthat communicate with the EMS module (Figure 1, left).

The schematics, parts list, the Arduino source code, and thesource code of the Android apps are available online.2

Toolkit HardwareWe typically use the Breuer Sanitas SEM 43 as the signalgenerator. However, we tested the system with three otherEMS devices. The control module reduces the signal inten-sity of the EMS device. The EMS device is calibrated withthe maximum acceptable intensity for a particular user. Forsafety reasons we designed two galvanically isolated circuits,the EMS circuit and the control circuit. The EMS circuit isconnected to the signal lines of the EMS device and the hu-man body, as shown in Figure 2, right. The control circuit isconnected to the Arduino (Figure 2, left), but also to the BLEmodule, and a 9V or USB power supply.

EMS circuit: The galvanic isolation of the EMS circuitis implemented with an optically isolated MOSFET driver(VOM12713) as shown in Figure 3. It generates a currentto drive two MOSFETs (25NF204) in the EMS circuit, whichreduce the signal intensity of the EMS signal and switch it onand off. One MOSFET is used for the positive and one forthe negative half-wave of the EMS signal. These MOSFETsare connected to one channel of the EMS device (EMS gen-erator). The regulated resistance of the MOSFETS should bebetween 100Ω and 10 kΩ, because otherwise the EMS de-vice switches off automatically. For safety reasons we alsoincluded two photo relays (LH1546ADF5 in Figure 3, top-center) to instantly cut off the EMS circuit when switchingthe module off.

Control circuit: In the control circuit the input of the MOS-FET driver is leveled by a 1 kΩ I2C digital potentiometer

2Let Your Body Move ToolKit https://bitbucket.org/MaxPfeiffer/letyourbodymove3VOM1271 www.mouser.com/ds/2/427/vom1271t-244790.pdf4STD25NF20 www.mouser.com/ds/2/389/DM00079534-470123.pdf5LH1546ADF www.mouser.com/ds/2/427/lh1546ad-254173.pdf

220Ω

I2C

100nf

Galvanic isolation

Electrode

Electrode

G

G

S

S

D

D

Channel 1

220Ω

1.5MΩ

Figure 3. Circuit of the EMS control module for one EMS channel.

Command Values Sample Description

Channel 0-1 C0 Set channel 0 or 1

Intensity 0-100 I56 Set intensity in %

On-Time 1-50000 T2000 Set the on time in ms

Activate/Go G G Activate the command

Table 1. EMS Control Protocol (ECP).

(AD52526). The Arduino controls the photo relays and theBluetooth communication through a standard BLE chip7.When the Bluetooth connection fails, the EMS channels getdisabled. For the BLE module and for each EMS channel weadded a control LED (Figure 2).

Protocol of the ToolkitAs long the EMS modules are not connected they announcetheir Bluetooth name. Based on the name the prototypingapps can find the device and connect to the “EMS-Service-BLE1” service. The service accepts ASCII strings to changethe EMS parameters. The protocol of the service is shown inTable 1.

For example, if the message “C0I100T750G” is sent to theservice, channel 0 will be activated at full intensity for750 ms. Resending “G” activates the same channel with theprevious parameters again. Each parameter can be changedindividually. If a new “on-time” message is received while achannel is active, the new time will be updated immediately,even if it is shorter. For example, if the signal is on and thenew time is set to 50 ms, it will deactivated after 50 ms. Themaximum on-time that can be set is 1000 ms. For safety rea-sons we use a relatively short on-time of 750 ms and resend“G” before it expires to keep the signal alive. When the appli-cation fails to renew the lease, the EMS channel stays activeuntil the on-time expires. The protocol can easily be extendedto add further commands, e.g., for changing the service nameor the connection code. However, this basic protocol is suffi-cient to generate a wide variety of feedback. For example, itis possible to generate slow movements (by slowly increasingthe signal strength) or oscillating movements (by modulatingthe signal with the sine function).

6AD5252 www.mouser.com/ds/2/609/AD5251_5252-246267.pdf7RN4020 www.mouser.com/ds/2/268/50002279A-515512.pdf

6

APPLICATION SCENARIOSSince the EMS module uses BLE it can connect to any de-vice that supports BT 4.0 or higher. The sample apps wereextensively used for prototyping and in multiple user stud-ies. We have successfully run these apps on Android devices,such as tablets, mobile phones, and smartwatches. Below,we present four prototyping apps that can easily be extended:(1) an app for Wizard-of-Oz prototyping that can connect theEMS module and apply different parameter sets, (2) an appfor connecting multiple devices to actuate a set of muscles,(3) an app that connects to a bracelet wearable device, and(4) a smartwatch app.

Wizard-of-Oz prototyping:The Wizard-of-Oz prototyp-ing app can connect to oneEMS module and activatetwo channels. Our EMSdevice can generate twoindependent signals. Fourelectrodes are needed to ac-tuate two muscles. The appactivates a channel while the

user presses a button. The signal intensity may be adjustedwith a slider. Different patterns may be chosen: Immediatelyon, square wave signal with a 1 s period, sawtooth signal(linearly increasing intensity, then off and starting again),sine wave signal, and inverse sawtooth signal (linearlydecreasing, then on). Finally, there is a text field for sendingcustom protocol messages. Each time the button is pressed,one custom message is sent.

Connecting Multiple De-vices: Actuating multi-ple muscles requires addi-tional control modules andEMS devices. This is use-ful for more complex hap-tic feedback, like graspingor lifting an object. Eachbutton activates one chan-nel while it is pressed. Itis also possible to changethe intensity and use indi-

vidual messages. The app can be used as a “gesture key-board” using chording.

Myo Remote Control: Theremote control app showshow to use data from ex-ternal sensors such as theMyo8 to control muscleactivation. In this case themobile device serves as agateway between the EMSmodule and the Myo de-vice. This setup enables aremote control for another

person: Attaching the EMS pads to one person and having

8Myo 360 www.myo.com

the Myo device worn by a second person. The detected move-ments of the second person can then be transmitted to the firstperson and replayed or otherwise mapped.

Event triggering: Thesmartwatch app runs ona Moto 3609 and extendsthat device with forcefeedback. For specificnotifications, such as analarm, it can actuate amuscle. In this sense theuser becomes the outputdevice for notifications.For example, when analarm is ringing the user

starts waving the hand. Different movements or rhythms maybe used for different kinds of notifications.

EVALUATIONTo evaluate the toolkit we conducted a one-day workshopwith 10 participants (2 female) at the WorldHaptics [19] con-ference. The goal of the workshop was to investigate ap-plication scenarios involving haptic feedback and to teachresearchers from different disciplines how to use EMS. Inpreparation of the workshop we built 11 EMS control mod-ules and deployed the Wizard-of-Oz app on 11 Androidtablets and mobile phones. The workshop consisted of a de-sign session on how to generate application scenarios and ahands-on Wizard-of-Oz prototyping session to test the pro-posed ideas.

Workshop ProcedureThe workshop started with a detailed overview of EMS inHCI research. We explained how EMS works and how tosafely use it. We followed up with an EMS experimentationsession, in which every participant had the chance to feel theinvoluntary muscle contractions that it caused. All partici-pants got a prototyping toolkit consisting of an EMS module,four self-adhesive electrodes, an EMS device, and a mobiledevice with the Wizard-of-Oz prototyping app. We asked theparticipants to perform a simple wave gesture and ended thissession with an EMS-based aloha wave.

With the experience of how EMS works, we run a 10 minutebrainstorming session. Each group selected one scenario andrealized it in a simple Wizard-of-Oz prototype. After under-standing which muscles they needed to stimulate to cause acertain movement, the participants combined the elementarymovements to EMS-based gestures.

At the end of the day the prototypes were presented in a ple-nary session. We video-recorded the prototyping session andthe final presentations for later analysis.

ResultsWe analyzed the video recordings of the prototyping sessionand the final presentation to identify the sub-results and seewhich application scenarios the participants tried with theWizard-of-Oz app.9Moto 360 www.motorola.de/products/moto-360

7

Figure 4. A participant controls another participant to take a photo.

During prototyping the participants tried different scenariossuch as single grasping, lifting the thumb up, changing theface expression, and remotely controlling a person to take aphoto.

The participants divided their initial idea first into several dif-ferent elementary movements such as opening and closing thehand and lifting the arm up. They then tried each movementone after the other. Finally, they combined the movements tocomplete gestures like grasping and releasing.

For example before Group 2 came up with their final appli-cation scenario they tried a remote control for a “selfie stick”shown in Figure 4. They also broke down their gesture intoelementary movements and combined them for the remotecontrol. They used the biceps and triceps to move the stickleft or right, and the muscles in the lower arm to lift the handup. Group 1 started to actuate the hand, then the upper arm,and finally they added more body parts, such as the foot. Thegroups acted straightforward to investigate the prototypes.

For some participants it was hard in the beginning to findthe right muscle. By performing the wanted movements theyexplored the rough position of the muscle. Afterwards theyplaced the electrodes and tested the position while slowly in-creasing the intensity. When it was not the expected effectthey replaced the electrodes until they got the right move-ment.

After the prototyping, Group 1 investigated a learning,Group 2 a remote control, and Group 3 a healthcare scenario.Group 1 with four participants called their learning scenario“piano-player.” The “learning device for piano” proposed tohelp a piano player to coordinate and time key hits, verticalmovements over the keyboard, and handling the piano footpedal (Figure 5a). To implement this idea they used six mus-cles. They explained that the leg muscles are “to control thepedal movements up and down” and the muscles in the arm“to control wrist movements.” Finally, they used the shoul-der muscles for “translating across the piano.” During theirpresentation they used new wordings such as “my shoulder isonline.” One participant was actuated while the other threecontrolled two muscles each. They first demonstrated eachactuation of the combined movement sequentially and thensimultaneously.

Group 2 investigated the “Play Pictionary” game scenario.They mentioned, that “this is a new step of Pictionary [...]after you game a little bit, you can control your friends whilethey will play Pictionary.” Group 2 introduced a remote paint-ing approach, where two players control one other player whopaints the picture (Figure 5b). One muscle is used to lift thehand up and one to push it down to draw vertical lines. To

draw horizontal lines the muscles in the upper arm pushes thehand left (biceps muscles) and right (triceps muscles). Eachplayer used one control module to actuate two muscles fordrawing each dimension. For drawing, both players tried tosynchronize the drawing. The group named this as a “fungame” with a completely new kind of gameplay.

Group 3 with again three participants composed a healthcarescenario that they called “diet control app.” This applica-tion supports people for balanced diet and helps to avoid un-healthy food shown in Figure 5c. The participants explained,“when it comes to a healthy option” ... “then you will take it”and an “unhealthy [option] will be avoided.” In the scenariothe user had to choose between different drinks. When tryingto grasp an unhealthy drink the hand was pushed away. Onthe other hand healthy drinks will be grasped and lifted up au-tomatically. Further when the user was just across a healthydrink “it will pull your hand down a bit.” They mentioned thatthe environment needs to be tracked to detect the location ofeach object. Group 3 used five muscles: Two muscles in thelower arm were used to grasp the bottle and one to open andpush the hand away from the drink. For lifting the hand thebiceps muscles were used. Finally for pushing the hand downtoward to a drink the triceps was used. To activate the move-ment one participant controlled the three muscles of the handusing two devices. The other participant lifted the hand up orpulled it down. While investigating the scenario the partici-pants investigated a kind of music book to activate the rightmuscle at the right time.

DISCUSSIONThe participants quickly got familiar with the toolkit and withEMS as a haptic feedback method. During the workshop theparticipants used most of the functionality of the toolkit aswell as all of the provided devices. In most cases the par-ticipants were able to locate a muscle for a certain move-ment, place the electrode pads, and actuate the muscle withthe toolkit.

The participants used the actuation time (on/off) as main ma-nipulation factor. A precise and smooth movement or hold-ing at a certain position was hard to achieve. To preciselycontrol the speed and the amount of movement a closed con-trol loop is necessary, as well as a way to adjust the intensityof the muscle contraction. It the took the participants somepracticing to find the right timing to combine the elementarymovements to a full gesture.

All groups came up with non-obvious ideas from differentareas. We observed new wordings, such as “shoulder is of-fline.” The main challenges the groups faced were related tothe timing to perform gestures and the precision of the actu-ation. For the timing the participants used multiple devicesside by side and actuated one muscle per finger. The preci-sion problem was more difficult to solve. The participants’strategy was to avoid very precise movements in their proto-types. For example Group 2 did not implement filigree fingermovements of piano players. They rather focused on grossmovements like moving the arm across the keyboard. Mostof the movements and gestures that the participants tried to

8

Figure 5. Final results EMS-based prototyping session (a) learning scenario “piano-player”, (b) game scenario “play pictionary”, and (c) healthcarescenario “diet control app.”

actuate worked reasonably well. To be as inclusive as possi-ble, the workshop was thematically quite broad. We believethat more specific ideas would have been generated in a morefocused workshop, considering, e.g., specific scenarios, spe-cific muscle groups, particular gestures, or particular applica-tion scenarios from everyday life.

The prototyping toolkit and process described in this paperaim to lower the entry hurdle for using EMS-based hapticfeedback in HCI research. There is an initial effort in produc-ing or ordering the EMS control module. The Eagle files forthe circuit and the parts list are available on the Let Your BodyMove ToolKit bitbucket project Web site. After this initialstep fast prototyping with a wide range of haptic feedback ispossible. The results from the workshop show that the toolkitcan be used in prototyping sessions with very little overhead.We successfully used the same toolkit and prototyping pro-cess in several user studies to perform simple movements aswell as complex gestures. Moreover, the toolkit is applica-ble for generating EMS-based haptic feedback in mobile andwearable usage scenarios as well as in virtual environments.

However, the presented toolkit does not shift the responsibil-ity away from the experimenter. When using EMS safety is-sues should always be a prime concern. Ethical aspects needto be kept in mind as well.

As the toolkit is available in open source form, it may freelybe modified. The Arduino software and the Android appscan be extended with new functionality. Bluetooth makes iteasy to connect new devices and to realize further applicationscenarios. For example, a mobile phone may act as a gatewayfor remote communication. The circuit may also be adapted.

CONCLUSIONLet Your Body Move is a full working prototyping toolkit forEMS-based haptic feedback. EMS enables a wide range ofhaptic output – from a slight tingling to limb movements –in mobile and wearable situations. The toolkit and associ-ated prototyping process considerably simplify the usage ofEMS-based feedback in mobile interactions. Because it usesBluetooth LE for communication the toolkit can easily becombined with mobile and wearable devices, such as mobilephones, wristbands, and smartwatches.

The software components and hardware schematics are avail-able in open source form. We describe suitable EMS parame-ter settings and pad placements for different kinds of elemen-tary movements. We discuss calibration requirements andsafety aspects. Further we present technical aspects of thetoolkit hardware and communications protocol. We give fourexamples of combining mobile and wearable devices with thetoolkit for realizing various scenarios.

The proposed toolkit was evaluated in a workshop setting.After a brief introduction the participants were able to de-velop interesting application ideas using EMS as a hapticfeedback technology. The participants were able to success-fully use most of the features of the toolkit. However, theevaluation also uncovered certain problematic areas, such asthe difficulty of temporal and spatial precision in Wizard-of-Oz prototyping. An idea to overcome these problems is toprovide scripted Wizard-of-Oz inputs rather than completelymanual control. We aim to extend the toolkit with such addi-tional functionality and use it in future prototyping sessionsand user studies.

REFERENCES1. Bau, O., Poupyrev, I., Israr, A., and Harrison, C.

Teslatouch: Electrovibration for touch surfaces. In Proc.UIST’10 (2010), 283–292.

2. Belda-Lois, J.-M., del Horno, S. M., et al. Rehabilitationof gait after stroke: A review towards a top-downapproach. Jour. NeuEng. and Reha. 8, 66 (2011).

3. Brave, S., and Dahley, A. intouch: A medium for hapticinterpersonal communication. In Ext. Abstr. CHI ’97(1997), 363–364.

4. Centofanti, S. K. Application of Muscle/NerveStimulation in Health and Disease. SpringerNetherlands, Dordrecht, 2008, ch. Electrical, 69–116.

5. Dollar, A. M., and Herr, H. Lower extremityexoskeletons and active orthoses: challenges andstate-of-the-art. Robotics, IEEE Transactions on 24, 1(2008), 144–158.

6. Farbiz, F., Yu, Z. H., Manders, C., and Ahmad, W. Anelectrical muscle stimulation haptic feedback for mixedreality tennis game. In posters SIGGRAPH’07 (2007).

9

7. Fialka-Moser, V., Ebenbichler, G., and Gillert, O.Elektrotherapie. Pflaum Physiotherapie. Richard PflaumVlg GmbH, 2005.

8. Forsslund, J., Yip, M., and Sallnas, E.-L.Woodenhaptics: A starting kit for craftingforce-reflecting spatial haptic devices. In Proc. TEI’15(2015), 133–140.

9. Ha, S., Kim, L., Park, S., Jun, C., and Rho, H. Virtualprototyping enhanced by a haptic interface. {CIRP}Annals - Manufacturing Technology 58, 1 (2009),135–138.

10. Houben, S., and Marquardt, N. Watchconnect: A toolkitfor prototyping smartwatch-centric cross-deviceapplications. In Proc. CHI’15 (2015), 1247–1256.

11. Ijzerman, M. J., Stoffers, T. S., In’t Groen, F., Klatte, M.A. P., Snoek, G. J., Vorsteveld, J. H. C., Nathan, R. H.,and Hermens, H. J. The NESS Handmaster orthosis:restoration of hand function in C5 and stroke patients bymeans of electrical stimulation. Journal of rehabilitationsciences 9, 3 (1996), 86–89.

12. Ito, K., Shioyama, T., and Kondo, T. Lower-limb JointTorque and Position Controls by Functional ElectricalStimulation ( FES ). Complex Medical Engineering(2007), 239—-249.

13. Kruijff, E., Schmalstieg, D., and Beckhaus, S. Usingneuromuscular electrical stimulation for pseudo-hapticfeedback. In Proc. VRST’06 (2006), 316–319.

14. Ledo, D., Nacenta, M. A., Marquardt, N., Boring, S.,and Greenberg, S. The haptictouch toolkit: Enablingexploration of haptic interactions. In Proc. TEI’12(2012), 115–122.

15. Lopes, P., and Baudisch, P. Muscle-propelled forcefeedback: Bringing force feedback to mobile devices. InProc. CHI’13 (2013), 2577–2580.

16. Lopes, P., Ion, A., and Baudisch, P. Impacto: Simulatingphysical impact by combining tactile stimulation withelectrical muscle stimulation. In Proc. UIST’15 (2015),11–19.

17. Lopes, P., Ion, A., Mueller, W., Hoffmann, D., Jonell, P.,and Baudisch, P. Proprioceptive interaction. In Proc.CHI’15 (2015), 939–948.

18. Lopes, P., Jonell, P., and Baudisch, P. Affordance++:Allowing objects to communicate dynamic use. In Proc.CHI’15 (2015), 2515–2524.

19. Lopes, P., Pfeiffer, M., Rohs, M., and Baudisch, B. Letyour body move: electrical muscle stimuli as haptics.Prog. IEEE World Haptics (2015).

20. Lopes, P., Pfeiffer, M., Rohs, M., and Baudisch, P.Hands-on Introduction to Interactive Electrical MuscleStimulation. In CHI EA ’16 (2016), 944–947.

21. Lyons, G., Sinkjaer, T., Burridge, J., and Wilcox, D. Areview of portable FES-based neural orthoses for thecorrection of drop foot. IEEE Trans. on Neu. Sys. andReha. Eng. 10, 4 (2002), 260–279.

22. Maeso, S., Reza, M., Mayol, J. A., Blasco, J. A., Guerra,M., Andradas, E., and Plana, M. N. Efficacy of the davinci surgical system in abdominal surgery comparedwith that of laparoscopy: a systematic review andmeta-analysis. Annals of surgery 252, 2 (2010),254–262.

23. Pfeiffer, M., Dunte, T., Schneegass, S., Alt, F., and Rohs,M. Cruise control for pedestrians: Controlling walkingdirection using electrical muscle stimulation. In Proc.CHI’15 (2015), 2505–2514.

24. Pfeiffer, M., Schneegass, S., and Alt, F. Supportinginteraction in public space with electrical musclestimulation. In Proc. UbiComp’13 (2013), 5–8.

25. Pfeiffer, M., Schneegass, S., Alt, F., and Rohs, M. Letme grab this: A comparison of ems and vibration forhaptic feedback in free-hand interaction. In Proc. AH’14(2014), 48:1–48:8.

26. Pfeiffer, M., and Stuerzlinger, W. 3D Virtual HandPointing with EMS and Vibration Feedback. 3DUI 2015(2015), 117–120.

27. Pohl, I. M., and Loke, L. Touch toolkit: A method toconvey touch-based design knowledge and skills. InProc. TEI’14 (2013), 251–258.

28. Prochazka, A., Gauthier, M., Wieler, M., and Kenwell,Z. The bionic glove: An electrical stimulator garmentthat provides controlled grasp and hand opening inquadriplegia. Archives of Physical Medicine andRehabilitation 78, 6 (1997), 608–614.

29. Prorelax. TENS+EMS DUO.http://www.sanitas-online.de/web/_dokumente/GAs/

therapie/752.907-0212_SEM43.pdf. Accessed:2016-02-04.

30. Reilly, J. P. Applied Bioelectricity: from ElectricalStimulation to Electropathology, vol. 20. Springer, 2009.

31. Strojnik, P., Kralj, A., et al. Programmed six-channelelectrical stimulator for complex stimulation of legmuscles during walking. Biomedical Engineering, IEEETransactions on, 2 (1979), 112–116.

32. Swindells, C., MacLean, K. E., Booth, K. S., andMeitner, M. J. Exploring affective design for physicalcontrols. In Proc. CHI’07 (2007), 933–942.

33. Tamaki, E., Miyaki, T., and Rekimoto, J. Possessedhand:Techniques for controlling human hands using electricalmuscles stimuli. In Proc. CHI’11 (2011), 543–552.

34. Weiss, M., Wacharamanotham, C., Voelker, S., andBorchers, J. Fingerflux: Near-surface haptic feedback ontabletops. In Proc. UIST’11 (2011), 615–620.

35. Yannier, N., Israr, A., Lehman, J. F., and Klatzky, R. L.Feelsleeve: Haptic feedback to enhance early reading. InProc. CHI’15 (2015), 1015–1024.

36. Zhang, D., Guan, T. H., Widjaja, F., and Ang, W. T.Functional electrical stimulation in rehabilitationengineering: A survey. In Proc. iCREATe’07 (2007),221–226.

10