Artificial prosthetic limbsProblems and solutions for connecting brains

and robots

Advanced Seminar

submitted byConstantin UhdeNicolas Berberich

NEUROSCIENTIFIC SYSTEM THEORYTechnische Universitat Munchen

Prof. Dr Jorg Conradt

Supervisor: Viviane Ghaderi, Ph.D.Final Submission: 07.07.2015

Abstract

Current motorized limb prostheses provide rudimentary functionality for the appli-cation in every day life. Together with poor cosmetic appearance this is the reasonwhy a large percentage of amputees do not use their prosthetic device regularly. Thisseminar paper seeks to present an overview of current state of the art research onneural interfaces. The focus lies on non-invasive recording with EMG and especiallyHigh Density EMG sensors. Additionally, different machine learning and patternrecognition algorithms for the decoding of the recorded signals are discussed. Finally,promising research directions for advanced prosthesis control will be discussed.

2

CONTENTS 3

Contents

1 Introduction 5

2 State of the Art of BCI technology 72.1 BCI Input Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.1.1 Invasive Brain Recording . . . . . . . . . . . . . . . . . . . . . 72.1.2 Noninvasive Brain Recording . . . . . . . . . . . . . . . . . . . 92.1.3 Indirect Brain Recording . . . . . . . . . . . . . . . . . . . . . 102.1.4 Supplementary Technologies . . . . . . . . . . . . . . . . . . . 12

2.2 BCI Output Technologies . . . . . . . . . . . . . . . . . . . . . . . . 132.2.1 Prosthetic Limbs . . . . . . . . . . . . . . . . . . . . . . . . . 132.2.2 Robotic Arms . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.2.3 Exoskeletons . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

3 EMG Method 173.1 Signal source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173.2 Electrodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183.3 Recording . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

3.3.1 Signal contamination . . . . . . . . . . . . . . . . . . . . . . . 193.4 High density EMG . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

3.4.1 Hardware requirements . . . . . . . . . . . . . . . . . . . . . . 203.4.2 Curse of dimensionality . . . . . . . . . . . . . . . . . . . . . . 213.4.3 Detection of bad electrode signals . . . . . . . . . . . . . . . . 22

4 Decoding Algorithms for EMG-based Prostheses 234.1 Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234.2 Feature Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244.3 Machine learning approaches . . . . . . . . . . . . . . . . . . . . . . . 26

4.3.1 Standard Algorithms . . . . . . . . . . . . . . . . . . . . . . . 264.3.2 Neuro-inspired Algorithms . . . . . . . . . . . . . . . . . . . . 27

4.4 Pattern Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

5 Summary and Outlook 315.1 Advances and Problems . . . . . . . . . . . . . . . . . . . . . . . . . 315.2 Promising Research Areas . . . . . . . . . . . . . . . . . . . . . . . . 31

4 CONTENTS

5.2.1 Deep Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . 315.2.2 Bidirectional closed-loop neuroprostheses . . . . . . . . . . . . 325.2.3 Multi-modal Approaches . . . . . . . . . . . . . . . . . . . . . 325.2.4 Semi-Autonomous Control . . . . . . . . . . . . . . . . . . . . 33

List of Figures 35

Bibliography 37

5

Chapter 1

Introduction

The first part of this composition gives an overview of current technological ap-proaches for interfacing brains and computers, so called brain-computer-interfaces(BCIs). This is a very general term, because the ”brain side” of the interface canmean getting the data directly by measuring brain functions but it can also mean,that the cognitive activity is measured indirectly by muscle activation or other cues.Likewise, the ”computer part” of the interface can stand for a standard personalcomputer, a robotic manipulator or a prosthetic device like a cochlea implant or anartificial limb. In this report we focus mainly on the non-invasive EMG technologyfor the input and an artificial limb for the output. While we believe, that there isno clear best BCI, we do think that there are technologies that perform better orbest for certain fields of applications. As the main concern of this report is machinesthat can replace a missing limb, we will take a deeper look into artificial limbs thesecond part of chapter 2 and into the EMG technology in chapter 3, with a focuson High Density EMG. We intend to show why neuroprosthetic systems that arebased on those two technologies are very promising in devices that amputees needin terms of functionality and control but also flexibility and appearance.In chapter 4 we examine the methods that are used to process the EMG data, decodeit and infer meaning by means of machine learning algorithms and send control com-mands to the artificial limb. Different algorithms, both classic and neuro-inspired,will be presented and compared in order to achieve a better understanding of thismost critical part of modern BCIs.The last part of this report will summarize the achievements of EMG-based neuro-prosthetic technology as well as its most dominant problems. Based on this analysisa short outlook on future research will be presented.

6 CHAPTER 1. INTRODUCTION

7

Chapter 2

State of the Art of BCI technology

This chapter seeks to summarize current state of the art technology that is beingused for brain-computer-interfaces and robotic limb prostheses.

2.1 BCI Input Technologies

There is a multitude of methods for interfacing brains and computers that havespecific strengths and weaknesses and are thus used for many different applications.Since the topic of this seminar paper is the use of BCIs for the control of prostheticlimbs, the most important criteria that the technologies have to be evaluated againstare real-time applicability, high precision, flexibility and adaptability as well as theability to control multiple degrees of freedom intuitively.

2.1.1 Invasive Brain Recording

The best way to acquire high quality signals of the neural activity is to surgicallyplace electrodes on the surface of the brain below the skull. However those ap-proaches are dangerous for the patient not only because they involve break of skinbut also due to the body rejecting a foreign object that can lead to infection.One method is Electrocorticography (ECoG), also called intracranial EEG (iEEG),where electrodes are placed directly on the brain surface to record the activity of thecerebral cortex. It was initially developed at the Montreal Neurological Institute inthe 1950s to treat patients with epilepsy to identify the regions of the cortex thatgenerate the epileptic seizures. The signals that are measured by the extracellularmicro-electrodes are the local field potentials (LFPs) which are the sum of the syn-chronized postsynaptic voltages produced by multiple neurons in the vicinity of thechip. This leads to a spatial resolution of 1 cm and a temporal resolution of ap-proximately 5 ms [AJS+05]. If depth electrodes are used in conjunction with a highsampling rate (more than 10 kHz) it is possible to measure the action potentialsof individual neurons. The reason for this massive increase in resolution is, thatthe potentials are primarily produced by cortical pyramidal cells which lie several

8 CHAPTER 2. STATE OF THE ART OF BCI TECHNOLOGY

layers below the surface of the cortex. Using ECoG, low voltage, high frequencycomponents can be detected, that cannot be seen in scalp EEG.The most notable research project in area of invasive neural interfaces is BrainGate[brab], which resulted in the BrainGate brain implant system that is currently testedin clinical trials.

Figure 2.1: The BrainGate neural interface [braa]

The BrainGate system consists of a sensor which is implanted directly onto thebrain’s motor cortex and uses 100 hair-thin gold microwire electrodes mounted on asmall silicon array to record the neural activity of individual neurons. Those signalsare then sent to a metallic pedestal which is embedded in the skull and from there toan external decoder device where they are interpreted and used to control a roboticarm [Pat09]. This setup has become well known for a project conducted in 2012[HBJ+12] where a subject with tetraplegia (a paralysis of all limbs and torso) whowas enrolled in the BrainGate2 pilot clinical trial, uses the system to steer a roboticarm precisely enough to pick up and drink from a bottle of coffee, see Figure 2.2.This was the first time, she was able to drink unaided in 15 years.The implant was a 4mm x 4mm, 96-channel microelectrode array, which was im-planted 5 years earlier in the dominant hand area of the motor cortex and enabledthe decoding of the local ensemble spiking signals. For the output device, a DLRLight Weight Robot III was used which enabled a robust finger position and graspingof the object by automated joint impedance control. This robotic arm system willbe further discussed in section 2.2.2. Despite the impressive results of the system, ithas several drawbacks. Most noticeable the external decoding device severely limitsthe patients mobility and thus partly counteracts the systems main goal of givingback autonomy to the patient . Furthermore the recorded data has to be compressedbefore it can be sent to the decoder resulting in loss of potentially useful information.The connector component that goes through the skin entails a high risk of infection.

2.1. BCI INPUT TECHNOLOGIES 9

Figure 2.2: A tetraplegic BrainGate2 clinical trial participant drinking from a bottleusing [nat][HBJ+12]

Although the participant had lower recorded spike amplitudes and fewer channelscontributing signals than during the first years of recording, it is notable, that thereach and grasp control could be achieved at all 5 years after the implantation ofthe intracortical array.Even though the team is working on a novel wireless neural recording microsystemto address those issues, invasive BCIs will still be less preferable to non-invasivesystems because of the necessary surgical intervention entailing medical risk andhigh cost. Therefore the remainder of this paper will focus on non-invasive BCItechnologies.

2.1.2 Noninvasive Brain Recording

Functional magnetic resonance imaging (fMRI)

Functional Magnetic Resonance Imaging (fMRI) is used as diagnostic tool in hospi-tals and research. It is a variation of MRI and utilizes echo-planar imaging for fastscanning of the whole brain by means of several cross section images. It not onlyhighlights active areas in the brain but is even capable to detect changes in activitydue to its high scanning speed of up to 1-2 s [SWC+08]. The source to the fMRIsignal is the magnetic difference in oxygenated blood compared to deoxygenatedblood. The so called BOLD (blood-oxygenation-level dependent) effect lags behindthe neuronal activity by 3-6 s and describes the selective supply of active neuronswith oxygenated blood. The varying concentrations cause electromagnetic pulsesfrom the fMRI to be reflected in corresponding strength . One has to find a com-promise between scanning speed and resolution to achieve meaningful results. The

10 CHAPTER 2. STATE OF THE ART OF BCI TECHNOLOGY

voxel (3D pixel) size correlating with the aforementioned speed measures around 3x 3 x 5 mm3 and contains millions of neurons.The recorded signals have to be preprocessed to account for movement artifactscaused by breathing and voluntary movement as well as signal dropouts and mag-netic distortions. The processed information is then classified by different machinelearning algorithms to detect specific behaviors and reactions.Feedback via video or sound can be utilized to detect specific areas that are respon-sible for vision, cognitive and affective tasks. Different voluntary movements help todetect specific motor areas. It is also possible to close the loop and let participantsself-regulate specific areas of their brain though automated feedback. Closed-loopfMRI-BCIs enable a multitude of applications such as study of functional reorgani-zation and plasticity as well as psychophysiological treatment [SCV+07]. Howeverthe huge spatial dimensions of the fMRI scanner render it useless for most BCIapplications.

Electroencephalography (EEG)

The biggest advantage of using electroencephalography (EEG) is its fast responsetime allowing temporal resolutions in the millisecond-range Modern EEG systemsare capable of recording at sampling rates of more than 20.000 Hz. EEG is mostlyused non-invasive and directly measures the voltage fluctuations from the neurons.To achieve this form of signal acquisition, multiple electrodes are placed on thescalp. Those sensors are densely placed compared to the distance to the sourcewhich results in several electrodes recording the same signal. This means thatadvanced statistical methods have to be used to disentangle the signals for optimalperformance. Compared to ECoG signals, the spatial resolution of EEG recordings ismuch lower because of the low conductivity of bone. EEG has already proven usefulin many clinical applications, e.g. for the diagnosis of epilepsy, sleep disorders andcoma.

The disadvantages of the EEG methods are that it is very noisy and that the signal-to-noise ratio is very challenging because the relevant brain activity is small com-pared to the the brain background activity and other interfering artifacts. Further-more most EEG systems have limited spatial resolutions. It is also problematicthat EEG poorly measures neural activity that occurs below the upper layers ofthe brain, because it is functioning by detecting LFPs. To be usable for Brain-Computer-Interface applications, you have to calibrate the EEG by using prior dataabout the person and the task. The correct placement of the dozens of electrodesand the application of various gels to keep them in place further impedes its use.

2.1.3 Indirect Brain Recording

Interfacing technology that doesn’t involve the central nervous system directly hasseveral advantages regarding prosthetic applications. Following, current approaches

2.1. BCI INPUT TECHNOLOGIES 11

will be discussed.

Electromyography (EMG)

Instead of directly recording the neural activity of the brain, one can instead recordthe amplified electrical activity produced by muscle cells. This method is calledelectromyography (EMG).EMG is used in medical applications such as neuromuscular monitoring during gen-eral anesthesia as well as detection of neuromuscular diseases.There are two approaches to Electromyography. The noninvasive method, calledsEMG, utilizes electrodes that are placed on the skin above the muscle in question.The invasive iEMG method relies on electrodes that are directly inserted into themuscle tissue. While these iEMG electrodes reduce crosstalk (signal pollution bynearby muscles) and enable more precise signal extraction, they are less practical touse, since one has to undergo surgery with all its possible complications and long-term signal quality can not be guaranteed[MCR10]. Therefore, most studies andcurrent prostheses rely on non-invasive sEMG electrodes.The sEMG signal is noisy but relatively robust due to the fact that the whole mus-cle mass that is observed, contracts almost at the same time and thus generatesa strong signal. This noninvasive recording is a popular and comparatively cheapmethod of control for myoelectric prostheses such as dexterous hand prostheses asshown in [CvdS09]. Until now, commercial prostheses only used few sEMG chan-nels to control several degrees of freedom of a prosthesis, which results in severalpredefined grasps which have to be selected by other means than the recorded sig-nal. However, current research shows that more dexterous and intuitive control ispossible [CvdS09][LAK+12][CZMC08]. As, for the presented reasons, EMG is oneof the best recording technologies for neuroprosthetic applications that are based onneural motor signals, we will discuss it in depth in the next chapter.

Electroneurography (ENG)

In recent studies, more direct means of control and feedback have been the focus ofresearch. While recording muscle activity is a convenient and reliable way of access-ing nerve signals, interfacing the nerves directly brings some tangible advantages.For this approach, there is no natural amplifier that allows for distant electrodes.Therefore all ENG electrodes have to be implanted in direct vicinity to, or even intothe target nerves. This however allows for a bidirectional approach that can recordand induce signal spiketrains directly as most somatic peripheral nerves are of mixednature and carry both motor and sensory strands. Often used are cuff electrodesthat surround the nerve, longitudinally implanted infrastructural electrodes (LIFEs)and transversal intrafascicular multichannel electrodes (TIMEs)[RCP+14] [MCR10].Results in [TSK+14] and [RCP+14] show, that ENG based direct feedback allowsfor very natural feedback in prosthetic control. Furthermore these studies hint onlongevity of signal quality, which is crucial for an invasive approach to be viable.

12 CHAPTER 2. STATE OF THE ART OF BCI TECHNOLOGY

Figure 2.3: EEG recording setup [eeg]

2.1.4 Supplementary Technologies

Visual tracking

In many advanced neural interface systems, visual tracking is applied in order togather the arm position of the human subject for training the classifier with su-pervised machine learning algorithms. In [SNF15] a tracking system was used toidentify the center instant of each movement. In a window of about 2 seconds aroundhis time instant, the EMG data was recorded that is later used to represent the ges-ture. In [VCvdS11], a Vicon MX motion tracking system was used, consisting of4 passive markers fixed on the subject’s wrist and 6 near-infrared cameras. Thissystem allowed to reconstruct the object’s position and orientation in real time.

Targeted Reinnervation (TR)

Often, after limb amputation, there are not enough muscles left to control all degreesof freedom of a complex prosthesis. The now unused efferent nerves that were partof the amputated limb may serve a purpose again, by surgically reconnecting themto unused muscle groups. This procedure is called Targeted Muscle Reinnervation(TMR) and is used in combination with EMG recording of the newly connectedmuscles. Research shows that this allows, after a brief training phase, for morenatural control of the prosthesis, compared to a regular EMG approach. [KDL+04].Similar to TMR, afferent sensory nerves can be reinnervated by connecting them tosensory areas of the skin. This approach is called Targeted Sensory Reinnervation(TSR) and can partly restore sensation in the amputated limb [MCR10].

2.2. BCI OUTPUT TECHNOLOGIES 13

2.2 BCI Output Technologies

BCIs and especially EMG based systems provide excellent possibilities for researchersto interface the peripheral nervous system with various robotic support technologies.

2.2.1 Prosthetic Limbs

The first prosthetic limbs were created 700 BC. The earliest finds were static andmainly of aesthetic nature. Later, body-powered variants were developed to providethe user with more natural control. These prostheses are comparatively cheap andallow for basic feedback to the controlling muscles but are aesthetically less pleasingthan current aesthetic prostheses. Furthermore, prolonged use applies strain anddevices often require unnatural looking movement, to control [MCR10]. To over-come those problems, externally powered prostheses were proposed after the firstWorld War. This class of devices is divided into switch- controlled and myoelectricprostheses. The first commercial myoelectric device was developed in 1960. Theswitch-based approach is controlled similar to body-powered devices, and allows fora quick learning phase but lacks the feedback that body-powered prostheses provide[MCR10]. Myoelectric devices rely on the electric signals that muscle activity cre-ates. Currently leading providers of myoelectric prostheses are Otto Bock with athree-finger grasping system and Touch Bionics, whose hand-prosthesis iLimb pro-vides five independently controllable fingers [MCR10]. In most cases, prostheses arecontrolled by only few muscle groups. This asks for complex control systems, thatallow the user to harness the utility with limited input possibilities. Therefore manyprostheses act context sensitive and signals from muscles and context are merged toallow for the right reaction.

Figure 2.4: The myoelectric controlled virtu-limb system from Touch Bionics [vir]

14 CHAPTER 2. STATE OF THE ART OF BCI TECHNOLOGY

2.2.2 Robotic Arms

Intuitively teleoperated robot arms are useful, when the controlling person isn’table to use direct manipulation. This, among others, is the case for robot-assistedsurgery, paralyzed patients or manipulation tasks in remote hazardous environments.Contrary to limb prosthetics for trans-radial amputations, upper limb movementneeds to be considered. This requires further investigation since most EMG-basedconcepts focus on grip-strength, where digit position is less important [VCvdS11]. Toavoid positioning problems, an implementation shown in [VBVDS13] utilizes velocitybased control and achieves good results in precision as well as intuitive control. Theutilization of modern robotic arm systems can improve the overall performance ofa neural interface system. In [VCvdS11] as well as in the Braingate trials describedabove, a DLR Light Weight Robot III (DLR LWR III) was used (see Figure 2.2),equipped with motor position, joint position and joint torque sensors in each jointto reproduce complex arm and hand movements. The use of joint torque sensorscan enable a robust finger position and grasping of the object by automated jointimpedance control as well as special soft-robotic features like collision detection andreaction by detecting external forces. Those safety features are necessary for human-centered applications. The mixing of advanced implant technology with advancedrobotic technology displays a potential area of advancement in neural interfaces. Inorder to reduce cognitive load, computational complexity and to increase intuitivebehavior, some precise movements can be controlled semi-autonomous by the roboticoutput device. Furthermore, the DLR LWR III has 7 degrees of freedom (DoFs),just as many as the human arm has and thus enables more natural movements.Since in 3D space, there are only 6 DOFs (3 positional DoFs and 3 orientationalDoFs) that can be controlled, this means that the robotic system is redundant andpermits null space motions as well as the opportunity to use the additional degreeof freedom to avoid singularities or to minimize energy. An example of a null spacemotions in humans is, when an object is being held in a constant orientation whilemoving the elbow.In [VCvdS11] the LWR III has been extended by the DLR-HIT Hand II, a 5-fingeredhand with torque sensors in each of its 15 joints. This enabled the hand to performpreprogrammed grasp motions controlled by joint impedance control whenever thedecoded grasp force exceeds a certain threshold.

2.2.3 Exoskeletons

BCI-based systems can be used to support several different clinical rehabilitationprocesses. People with degenerative muscle diseases such as sarcopenia can be sup-ported with robotic exoskeletons that receive movement information from sEMGelectrodes. Such systems are able to support activities of daily living (ADL) andcan be used to help retrain muscles by regulating support strength, similar to anelectric bicycle. One such system was showcased at the opening ceremony of the

2.2. BCI OUTPUT TECHNOLOGIES 15

world soccer championship 2014 when a paraplegic Brazilian used an exoskeleton tokick a ball.Furthermore, problems like phantom movement and pain in amputees can be workedon by enabling feedback through ENG based electrodes in combination with syn-thetic skin prototypes. Other applications of exoskeletons are in factories for liftingof heavy objects and in the military to enable soldiers to move faster and have betterendurance despite their heavy equipment.

16 CHAPTER 2. STATE OF THE ART OF BCI TECHNOLOGY

17

Chapter 3

EMG Method

EMG recording is the most widely used non-invasive recording technique for thecontrol of powered prostheses. As introduced before, it does not directly measurethe brain activity but instead senses the electric potentials that are generated bymuscle cells. In this section we will present the EMG recording method in depthand discuss the recent advances in High Density EMG research.

3.1 Signal source

To extract information from muscle activity, one has to understand the basic con-cepts behind the signal generation. The basic functional element of a muscle is calledthe motor unit (MU). Each motor unit is connected to an α-motoneuron. The neu-ron branches out and contacts all fibers of the MU. These connections are calledthe neuromuscular junctions. The signal from the neuron causes depolarization inthe fibers which travels in both directions from the junction and generates electriccurrent fields. These cause potential changes that can be measured by electrodes.Since all fibers in a MU are controlled by the same nerve, they react as a unit. Thesignal a MU generates is called motor unit action potential (MUP). The resultingsignal of a contracting muscle is the sum of all its simultaneously activated motorunits.Before this signal can reach an electrode, it has to travel the remaining tissue onthe way. This shapes frequency content with low-pass characteristics and dampenssignal amplitude which is mostly affected by distance and decays based on an inversepower relationship.The signal summation of the MUs partly causes phase cancellations. MUs are onlyweakly synchronized which results in random cancellation effects. This results in lessthan proportional increase in amplitude with rising MU numbers as well as variableamplitudes and thus has to be considered for signal recording and analysis.In addition, research has shown that MUs are organized in groups called neuromus-cular compartments, which can be controlled individually by the central nervoussystem. To differentiate between compartments, correct relative electrode placement

18 CHAPTER 3. EMG METHOD

is crucial and several simultaneous recording sites may be beneficial [SRSVD10].

3.2 Electrodes

Electrodes used in sEMG recordings can be differentiated into dry and wet, polar-izable and non-polarizable as well as single- and multichannel electrodes. The mostcommonly used sEMG electrode is based on a silver surface and silver-chloride elec-trolyte due to its good performance, non-toxicity and low manufacturing cost.Most iEMG electrodes are based on four wires for differential recording, which isknown as quadrifilar electrode. Newer concepts are based on thinfilm technology andis build with silicon wafers. The so-called Implantable Myoelectric Sensors (IMESs)are long-term friendly, can be produced in bigger numbers and result in less prob-lems during muscle contraction as they are permanently held in fibrous scar tissue[FYSK08].

3.3 Recording

Figure 3.1: Distinct sEMG amplitude relations for selected gestures recorded by 4electrodes placed around the forearm [BFBG14]

There exist several different approaches to control prosthetics with EMG signals.Current commercial prostheses mostly use the dual-site and single-site approach.The former uses the signals of different muscles for flexion, extension or pronation

3.4. HIGH DENSITY EMG 19

and supination. The latter method utilizes one signal source to control both direc-tions by assigning levels of muscle contractions in one recording site[MCR10].Signal recording is commonly achieved by monopolar and single-differential setups.Monopolar recording uses a single electrode recording the muscle signals and a refer-ence electrode which is placed away from the source. The single-differential conceptis based on two electrodes that are placed along the muscle fiber at a fixed inter-electrode distance to record the local difference.Typical EMG signal amplitudes measure around 100µV [CvdS09].

3.3.1 Signal contamination

To get clean and repeatable measurements, one has to factor in several possible errorsources during recording sessions. The biggest positive influence on signal quality isachieved by correct electrode placement. Distance to the active motor units in themuscle should be constant and as short as possible since the signal degrades sub-stantially when electrodes are placed further away. In addition, bipolar electrodeshave to be aligned correctly along the muscle fiber between the innervation zoneand fiber tendon intersection to allow for correct measurements [SRSVD10]. Figure3.2 displays signal change for different bipolar electrode locations along the muscle.Crosstalk from other muscles influences signals too. This is mostly dependent onthe distance to the contaminating muscle, as well as its size. Another source of errorare artifacts caused by movement of electrodes and connectors. These contamina-tions are normally expressed at around 10 Hz. Electrical equipment can influencemeasurements at 50Hz or 60Hz, dependent on the used power supply.

3.4 High density EMG

Current research heralds High Density (HD) EMG as the next step in signal record-ing, made possible by improvement in grid electrode technology and the develop-ment of more powerful microprocessors. Signals from electrode arrays allow for hightemporal and spatial accuracy while being robust to electrode shift[SNF15] as theycreate complete maps of muscle activity, contrary to recordings of single points ofinterest. The goal is to optimize prosthetics use with intuitive and powerful controlstrategies that are based on HD-sEMG recording arrays. The change in the utilizednumbers of electrodes is made clear when comparing [VCvdS11] where 9 electrodeswere used and [BFBG14] with only 4 electrodes to [SNF15] which used as many as192 electrodes.Array setups can incorporate up to 350 recording channels or more, as shown in[RMMA+12a]. With this amount of electrodes, the possibility for bad signals rises.Thus, those electrodes have to be detected and ignored, to ensure ongoing overallsignal quality. In classical pattern recognition algorithms, a faulty channel can causeserious degradation in classification accuracy, and even if the malfunctioning chan-

20 CHAPTER 3. EMG METHOD

Figure 3.2: Different bipolar electrode locations along the muscle resulting in varyingamplitude and frequency content [MCR10]

nel is detected and removed, it still requires a re-training of the classifier.

This calls for algorithms that are optimized for massive amounts complex signalstreams and computational efficiency. Reduction of computational burden can forexample be achieved by reducing the input samples online via its inter-sample dis-tance before classifying them with an algorithm while retaining classification accu-racy, as shown in [CvdS09]. Furthermore, the rate at which samples are recordedhas direct influence on performance and has to be optimized for execution time andaccuracy. New machine learning approaches for meeting the challenges of HD EMGswill be discussed in section 4.

3.4.1 Hardware requirements

With high number of electrodes, computation cost rises substantially. This has to beconsidered since current machine learning-based algorithms already demand power-ful systems. For commercial implementations, algorithms have to run on embeddedsystems which are limited in power, compared to desktop systems.For a rough guess of the computational demand, the system specifics of [SNF15]were used. Considering the grid of almost 200 electrodes that record 2048 samplesper second, each of which is A/D converted to 12 bit, a data stream of 5.000.000bits or 600 kB, that has to be processed every second for real-time capability. Newersystems use as many as 500 electrodes with sampling frequencies of more than 4096

3.4. HIGH DENSITY EMG 21

Hz. This results in high requirements for processors in terms of speed. The EMGsignal preprocessing is a somewhat independent step that has to be done in all sEMGapplications. For this reason electrodes like the Ottobock 12E200 have integratedfilters and amplifiers to create a suitable output signal for embedded systems around±3.3V [BFBG14]. HD-EMG arrays do not provide enough space for local ampli-fication chains and thus need a dedicated unit for this step. Current research onHD-EMG signals utilizes bidimensional electrode arrays produced by LISiN Politec-nico di Torino, Italy [HMF14] or fabricated ones [RMMA+12a] in conjunction withone specific amplifier from OT Bioelettronica, Italy, the EMG-USB2 [RMMA+12a][SNF15] [HMF14], which is capable of recording 265 channels at up to 10240 Hzsampling rate. While this device is categorized as desk equipment, the companyalso offers portable solutions with 32 channels and 2000Hz sampling frequency. Theresults in [BFBG14] show performance of a Support Vector Machine with linear andRBF kernel which was implemented on an ARM Cortex M4 microcontroller, workingwith 4 electrodes. Execution time scaled linearly with implemented training vectorcount and resulted in up to 15 ms computation time at around 680 vectors. Theteam explicitly state that they used SVM with kernels for its low computationaldemand and good behavior with high dimensional input vectors.Unfortunately, current literature lacks precise computational performance analysisfor high density electrode arrays in combination with various machine learning al-gorithms.

3.4.2 Curse of dimensionality

Adding additional dimensions, the same number of data points becomes less in-formative since they have to describe more variables simultaneously. In fact, thevolume of the possible input space expands exponentially with linearly rising di-mension count. This bloats filesize and required input information that has to beprocessed. With electrode arrays, information dimensionality rises alarmingly sinceevery electrode represents a dimension of the input vector. Still, prosthetic setupsare required to transfer and utilize this information in real time. Furthermore spaceconstraints are dictating the maximum power of the controller that has to classifyand regress the data. It is not viable to route all information from the electrodearray directly into the machine learning algorithm.For data transfer means, incoming information can be compressed and reduced.There exist several approaches to this problem, with one originating in image com-pression. The research team from [IFM14] applied lossless and lossy jpeg compres-sion to electrode array information, which helps with transferring information fromlarge electrode arrays to the target microcontroller. This approach achieved 59.3%filesize reduction in the lossless case and up to 90.8% in a lossy approach whilekeeping the Signal to Noise Ratio at 21.19 dB.Another international team [Cic14] worked on high dimensional data in a more gen-eral way and derived a novel approach to dimensionality reduction based on fault

22 CHAPTER 3. EMG METHOD

tolerant Tensor Networks and Tensor Train decomposition that allow for strongcompression and parallel computation in machine learning context. It also allowsfor existing low dimension algorithms to be applied to high dimensional data. Stillthis approach requires some a priori knowledge about the data and close observationof numerical stability.

3.4.3 Detection of bad electrode signals

Measurement setups with few electrodes require close attention to signal quality ofevery single placed electrode due to sparse feature dimensions. For dense electrodearrays, it is not possible to create good contact for all electrodes, thus there arealways some channels, that do not record useful signals. Since those electrodes sendsignals nonetheless, one has to detect them in order to exclude those channels fromthe machine learning pipeline.Manual detection of bad channels is a very time con-suming task that requires some experience and may not be viable in complex setups.There are several different approaches to detecting erroneous electrodes automat-ically among which the team of Monica Rojas-Martinez [RMMA+12a] proposes aconcept, that selects based on spectral and amplitude features as well as comparisonto neighboring electrodes. The implementation utilizes a threshold based relativepower analysis of low frequency components from 0Hz to 12Hz and power-linecomponents at 50Hz as well as RMS signal power. Detection rates of 86% wereachieved.

23

Chapter 4

Decoding Algorithms forEMG-based Prostheses

For the decoding of the EMG signals, different classes corresponding to differentarm movements are trained by machine learning algorithms. In general, this is doneby matching the recorded and preprocessed signals to the specific gestures. Thisway, labeled data is obtained that can be transformed to a suitable feature spaceand then be used by an appropriate supervised learning algorithm. The algorithmcomputes a classifier which normally can be seen as a decision function that mapsnew, unlabeled data to one of the many classes of gestures. This step is calledpattern recognition and is comprised of three main processing parts as illustrated inFigure 4.1:

1. Preprocessing

2. Feature Extraction

3. Feature Classification

Similar to the machine learning process, the recorded EMG data is first preprocessed.This includes filtering the signals and cleaning the data from outliers produced byfaulty electrodes. In a following step, distinct features are extracted that can beused for training a classifier by machine learning as well as to classify new recordedand unlabeled EMG signals by pattern recognition.

4.1 Preprocessing

The recorded raw signals have to be preprocessed for use in control algorithms. Firstof all, the measured signals are amplified with a typical gain of 500 [SNF15](EMG-USB2, OT Bioelettronica). In a following step, the frequencies of the actual signalshave to be acquired. A bandpass filter, e.g. a fourth-order Butterworth bandpassfilter, is applied to the recordings which extracts signals between 10 to 500 Hz. This

24 CHAPTER 4. DECODING ALGORITHMS FOR EMG-BASED PROSTHESES

Figure 4.1: EMG-based Neural Interface [emg]

approach also gets rid of DC offset since offset can be seen as a low frequency. Toprepare the data further, it has to be converted into digital information. For this,12,16 or 24 bit AD converters are used, with 1024 to 2048 samples per second which istypical for EMG application[SNF15][MCR10].After the relevant frequencies are available, information can be extracted in threemajor ways. All three implementations rely on a time window. This window canbe extracted disjoint, sequentially or overlapping. The extreme case is a slidingwindow over the data.Window size correlates with intervals of 150-200 ms to getgood matching results while retaining acceptable control delay [JDMF12]. Windowtype and size have direct influence on the number of samples, that have to be pro-cessed and thus need to be considered in respect to additional real-time constraintsas well as usability since natural movement is continuous and not a series of distinctfeatures[JDMF12]. In many applications, the samples are also normalized by sub-tracting the mean and dividing by the standard deviation in each dimension beforebeing used for the training [VCvdS11][VBVDS13].

4.2 Feature Extraction

In the past, most research on pattern recognition for myoelectric control was focusedon small numbers of EMG channels [SNF15]. Due to improvements in grid electrodetechnology and the increase in computational prowess of microprocessors the non-invasive High Density (HD) EMG method is gaining popularity. The higher amountof recorded data enables the use of different features as well as different machinelearning algorithms. Earlier work on sparse EMG recordings used time domain(TD) features [HPS93] or transformed the signals to feature spaces like the Fourieror wavelet domains [F+08][AIK11]. However those are only the most frequently usedfeatures. Other possible features whose parameters can be computed easily and fastare mean absolute value (MAV), variance (VAR), standard deviation (SD), zerocrossing (ZD) and slope sign change (SSD) [AIK11]. All of them can be later used

4.2. FEATURE EXTRACTION 25

to train a neural network classifier. In [SNF15] the research group of Dario Farinafrom the University of Goettingen has proposed the use of spatial features, based onthe observation that samples recorded in proximity are correlated, meaning that theyare likely to have similar values. Therefore the recorded HD EMG signal matricesare first divided in time windows. For each time window, a variogram is computedas a graph of the semivariance against the distance between pairs of locations in thenetwork. Originating from other applications like image processing or geostatistics,the variogram serves as a statistical measure for the spatial correlation of the signals.This method is convenient for HD EMG recordings as they supply the high spatialresolution that it needs. In fact, the minimum number of pairs for the estimation ofthe variogram is approximately 50. The experimental variogram is computed withthe empirical semivariance:

γ(h) =1

2n(h)

n(h)∑i=1

[z(xi) − z(xi + h)]2

Here, z(x) is the observed value at location x and z(x+h) is the value at a neigh-boring point displaced by h in the direction of x. In Figure 4.2 the representationof EMG root mean square (RMS) values on a matrix of 192 electrodes and the cor-responding experimental variogram are shown, as published in [SNF15]. The RMSrepresentation is the most simple EMG feature, as it is only computed as the RMSvalue of each electrode over several intervals of recording the center instant of themovement. Often, it is only used as an intermediate step in a process of obtainingmore stable feature vectors like the variogram. Trial (a) has been recorded for awrist flexion, (b) for a wrist extension and (c) represents the response when thesubject performs a forearm supination. Since the experimental variogram has beenevaluated in the main directions, x and y, it can be represented as a surface. It hasbeen shown by researchers in other fields of study, that the larger the number ofspatial measurement points, in this case the amount of electrodes, the more preciseis the estimation of the variogram [WO07]. Tests conducted by [SNF15] showed,that variogram features allowed to eliminate channels during the test phase with-out retraining. Even when a large proportion of the channels were omitted, theclassification accuracy was still high. This shows, that spatial features are muchmore powerful and robust against malfunctioning electrodes and shift than classictime-domain features in HD EMG applications.The research done by [RMMA+12b] confirms this result. They compared the appli-cation of single bipolar signals to HD EMG maps and obtained the result, that HDEMG allows better identification of tasks as well as lower effort levels. For featureextraction, they used automatic detection of low quality signals which are muchmore common in systems with many electrodes like HD EMG. They determinedthree different characteristics of low-quality signals e.g. that their energy may bemuch higher or lower than that of neighboring monopolar channels. All three caseswere numerically defined and applied to threshold methods in order to find the low-

26 CHAPTER 4. DECODING ALGORITHMS FOR EMG-BASED PROSTHESES

quality signals, which were subsequently removed and interpolated based on RMSvalues of neighbor channels in the HD EMG map.

Figure 4.2: Matrix of EMG RMS values and the corresponding experimental vari-ograms [SNF15]

4.3 Machine learning approaches

4.3.1 Standard Algorithms

Support Vector Machine - SVM

The most commonly used machine learning algorithm for the computation of theclassifier for EMG data is the Support Vector Machine [VCvdS11][SNF15]. An SVMseparates classes by means of hyperplanes that maximize the margin to the nearestsamples of the two classes, the so called ’Support Vectors’. The hyperplane resultsfrom the use of the so called ’kernel trick’. The application of a kernel functionallows to operate in a high-dimensional feature space without computing the co-ordinates of the data in that space. Instead, only the inner products between theimages of all pairs of data in the feature space have to be calculated. The choiceof the right kernel function can allow to linearly separate two classes in the kernelspace, that could not be linearly separated in the original feature space. The Gaus-sian kernel, that was used in [VCvdS11] is one of the most popular ones. When itis used, the SVM tries to build an approximation of the underlying function as asum of Gaussian functions centered on a subset of Support Vectors, which are verydistinct training samples. In [SNF15] it was found, that a linear kernel resulted inthe best classification accuracy. In [BFBG14] a radial basis function (RBF) kernelwas used.In order to optimize the recognition accuracy of the SVM classifier, suitable hy-perparameters have to be found. Grid-search and k-fold cross-validation have beenproven valuable for this task.

4.3. MACHINE LEARNING APPROACHES 27

Support Vector Machines have the drawback, that they can only find a decisionhyperplane between two classes. In BCI applications, the number of classes rep-resenting the different gestures is usually much higher resulting in the ’multi classproblem’. A solution to this issue is the one-versus-one (OVO) approach used in[SNF15] where classifiers are build for each pair of training features or the one-versus-all (OVA) approach used in [BFBG14] which involves training a single classifier perclass. The SVM method is usually very efficient since only the support vectors areused for the optimization instead of every single training vector.

Linear Discriminant Analysis - LDA

The Linear Discriminant Analysis (LDA) is a machine learning algorithm which canbe used to find a linear classifier by constructing a linear combination of features.Similar to the Principal Component Analysis (PCA), the LDA transforms patternsto a format where the different classes can be separated more easily. To determinethe LDA transformation function, the eigenvalues and eigenvectors of the class co-variance matrices are used. Because it is a statistic method, it performs well forimprecise measurements. Compared to other classifiers, LDA requires little compu-tation and less memory which makes highly qualifies it to be used for the decodingof HD EMG data. However it suffers form the problem of singularity [KAJAA09].

In [SNF15] the performance of different features paired together with either anSVM or an LDA was evaluated. The results that were unfortunately not shown inthe paper, indicated, that Support Vector Machines are the better classifiers to beused in conjunction with variogram features, whereas using a Linear DiscriminantAnalysis yields better results when the features are in the form of RMS, Time-Domain (TD) or Time-Domain-Autoregressive (TDAR). The interesting finding hereis, that features and classifiers can not be randomly chosen, but should be chosenin pairs that work well together.

4.3.2 Neuro-inspired Algorithms

Neural Networks

Inspired by biological neural networks, artificial neural networks are one of the mostpowerful machine learning algorithms as they entail the capability to learn fromexamples and have a highly parallel and regular structure. Furthermore, they areuniversal approximators, meaning that they can be used to approximate any functionthat transforms input data to output data. An artificial neural networks consists ofinterconnected nodes representing neurons, that compute the weighted sum of theirinputs and then use a transfer function to map the result to either 1 or 0, in thebiological analogy this corresponds to whether the neuron fires. When building aneural network architecture for a specific application, there are several parametersthat can be attuned, e.g. the number of hidden layers and the number of neurons in

28 CHAPTER 4. DECODING ALGORITHMS FOR EMG-BASED PROSTHESES

each of them. Unfortunately, there is no specific way to calculate the optimal num-ber of hidden neurons, so it must be chosen based on the best classification result intrials with different numbers of neurons. The research group in [VBVDS13] traineda neural network consisting of two hidden layers which both consisted of 10 neuralnodes. In [AIK11] a single layer MLP (multi-layer perceptron) with 10 neurons inthe hidden layer was used. Besides, an appropriate transfer function can be chosen.Popular choices are the sigmoidal and the tangential function. In [VBVDS13] andin [AIK11] a tansig transfer function was used.Once an architecture for the neural network has been chosen, it is trained with thebackpropagation algorithm together with an optimization method like gradient de-scent. In [VBVDS13] as well as in [AIK11] the Levenberg-Marquardt optimizationwas used. The result of this training process are weights of the inter-neural connec-tions that produce the smallest possible error when comparing the training outputdata to the training input data applied to the neural network. Once the optimiza-tion method and thus the weights have converged, new input data can be classifiedby simply applying it to the input layer of the network. To avoid overfitting, thetraining data should be divided into training, validation and testing data. Earlystopping methods can help to further improve the generalization of the network andto reduce the computation time, especially if there are more than sufficient trainingsamples.A thorough overview of many different neural network architectures that have beenapplied to EMG signals have been presented in [AIK11], e.g. finite impulse re-sponse neural networks, fuzzy clustering neural networks and adaptive neuro-fuzzyinference systems.

Neuro-inspired Models

Most of the current neuroprosthetic technologies are based on the ”black box” mod-els described in the previous sections, which means, that they are parametric modelsthat are built by using prerecorded input-output relationships. Another type of ap-proaches are neuro-inspired models like the one proposed in [HBT+15]. They utilizelearning rules that are derived from the biological mechanisms in the respectivebrain area. Since we have good knowledge of the modular functional connectivityof the cerebellum, has been used to verify the approach in [HBT+15]. Advantagesof neuro-inspired approaches However there are also downsides to the use of neuro-inspired learning rules. Their models ”may be limited by incomplete anatomical andphysiological mapping or by the diffuse localization of the function to be replaced.”[HBT+15] Black-box models on the other hand don’t rely on a good understand-ing of the biological background and can thus be applied to very complex circuitry.Though, they need prior sampling of the input-output signal relationships that areused for training. Furthermore, models that use input-output learning are producingindividual-specific adaptations which means that they implicitly include representa-tions for previously learned tasks. However, those ”parametric models are incapable

4.4. PATTERN RECOGNITION 29

of learning new adaptations to support new behaviors” [HBT+15].

4.4 Pattern Recognition

While the training phase can easily be done offline, the recognition of new motioncommands via EMG signals have to be online and ideally in real-time.The method that has to be used to map an new, unlabeled feature vector to the classof the correct gesture, depends on the classifier computed in the training phase. Inthe easiest case, each class has a representative feature pattern vector for instanceconsisting of the RMS values of all electrodes. After being preprocessed and turnedinto feature vectors, new EMG signals can be classified by Nearest-Neighbor classifier(or k-Nearest-Neighbor, when each class consists of several representative vectors).For this method, a of several distance functions can be used, most commonly theEuclidean distance or the Mahalanobis distance.If a SVM has been used to compute the classifier, the new feature vector has to beapplied to the decision function in order to find the correct class of the intendedgesture.In the case of a neural network classifier, the feature vector has to be be applied tothe input layer of the network, the resulting class is then computed by the networkand displayed at its output layer.

30 CHAPTER 4. DECODING ALGORITHMS FOR EMG-BASED PROSTHESES

31

Chapter 5

Summary and Outlook

5.1 Advances and Problems

The research in myoelectric prostheses has come a long way. The quasi standardof simple clamp like motorized prostheses with only one degree of freedom is be-ing rethought. Achievements in research start to appear in commercial products.Among them are multi-electrode interfaces, touch based context sensitive hand pros-theses and complex control strategies for multiple applications. Many more are stillbeing researched on, such as direct brain interfaces, dense surface EMG arrays anddirect closed loop interfaces. Proportional force control and other implementationsbased on advanced classification algorithms like neural networks may soon make anappearance in products and will help to make every day life a lot easier for those inneed. Users will see their powered prosthesis as an enrichment and may some daybe able to use them in the most natural way.Until then, there do remain hurdles to be overcome. Algorithms have to be optimizedfor high dimensional inputs, embedded controllers have to meet rising computationaldemands. sEMG signal based control has to become more user friendly and intu-itive. The sole focus on pattern classification performance may be misleading sinceit has been shown that matching performance correlates only weakly with controllerusability [JDMF12]. Also, durable long term application in invasive methods has tobe researched thoroughly.The remaining sections will discuss some of the mentioned topics in more detail.

5.2 Promising Research Areas

5.2.1 Deep Learning

Deep Learning is a very new area of study that has gained significant popularityin the last years. In 2012 the Google Brian Project Team, lead by the renownedmachine learning expert Andrew Ng, used a deep learning algorithm to make sense

32 CHAPTER 5. SUMMARY AND OUTLOOK

of several million images from Youtube. One of their results was, that the algorithmlearned the visual concept (representation) of a cat, shown as an image.Currently, deep learning is mainly used in computer vision and in speech recognitionapplications. Many of the biggest software companies like Google, Facebook andBaidu are investing heavily into deep learning research and have hired many of thefields leading scientists. Since the hierarchical representations used in deep learningare very similar to those in the neocortex, which might be a reason for the hugesuccess of deep learning in computer vision, one could assume that it will workequally well for finding representations for neural signals from the motor cortex.First advancements in this area have already been made by [VBVDS13], where adeep neural network consisting of two nonlinear hidden layers have been used todecode EMG data.

5.2.2 Bidirectional closed-loop neuroprostheses

Current prostheses rely solely on vision feedback to control grip strength and timing.This concept requires close attention in all grasping tasks and causes problems withchoosing the right grip strength. The most natural way of closing the loop is touse existing structures and re-enable unused efferent sensory nerves for joint posi-tion control and touch sensation. First experiments with touch feedback show verypromising results as researchers were able to reliably induce targeted local touchsensation[TSK+14] and subjects were able to grasp fragile objects without crushingthem[RCP+14]. Subjects also reported very pleasing and intuitive control in com-parison to conventional EMG-based prostheses. Finer control over touch feedbackas well as other types like temperature and joint position sensing are very promisingareas of research and may revolutionize the use of myoelectric prostheses in the fu-ture. Refinement in sEMG signal analysis enables to sense exerted force which canthen be used to control grip strength in prostheses

5.2.3 Multi-modal Approaches

The concurrent use of HD EMG sensors, EEG sensors and supplementary systemslike eye tracking can help to increase precision and robustness, especially for am-biguous sensor data and in cases where single electrodes are malfunctioning. Thisalso means potentially more natural control and less mental effort for end users.However, for advancements in this area, some challenges must be tackled. HighDensity EMG recordings alone already pose tough processing power requirementsto the decoding device. For a multi-modal approach not only the chips’ performanceneeds to be increased, but also their parallel computing capacity. This might go verynicely with the aforementioned deep learning algorithms which are increasingly runon graphics cards due to their huge amount of processing cores compared to currentCPUs.Furthermore, the recorded and decoded data from the different sensor types has to

5.2. PROMISING RESEARCH AREAS 33

be integrated intelligently to make a real difference to the single-modal approach.This could be done by sensor fusion algorithms.

5.2.4 Semi-Autonomous Control

As discussed in 2.2.2, we believe that the use of advanced robotic control methodslike impedance control can help to increase the performance of the neural interfaces.Semi-autonomous control also reduces the cognitive load of the subject by makingthe interaction more intuitive. When the able-bodied perform complex motion taskslike dancing or sports, they don’t plan every single muscle contraction but plan theirmovements on a higher level and let ’muscle memory’ do the specifics.Lastly, semi-autonomous human-friendly control algorithms based on sensing of ex-ternal forces and torques lead to the level of safety that is necessary for an applicationof the brain-robotic system in daily life.

34 CHAPTER 5. SUMMARY AND OUTLOOK

LIST OF FIGURES 35

List of Figures

2.1 The BrainGate neural interface [braa] . . . . . . . . . . . . . . . . . . 82.2 A tetraplegic BrainGate2 clinical trial participant drinking from a

bottle using [nat][HBJ+12] . . . . . . . . . . . . . . . . . . . . . . . . 92.3 EEG recording setup [eeg] . . . . . . . . . . . . . . . . . . . . . . . . 122.4 The myoelectric controlled virtu-limb system from Touch Bionics [vir] 13

3.1 Distinct sEMG amplitude relations for selected gestures recorded by4 electrodes placed around the forearm [BFBG14] . . . . . . . . . . . 18

3.2 Different bipolar electrode locations along the muscle resulting invarying amplitude and frequency content [MCR10] . . . . . . . . . . 20

4.1 EMG-based Neural Interface [emg] . . . . . . . . . . . . . . . . . . . 244.2 Matrix of EMG RMS values and the corresponding experimental var-

iograms [SNF15] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

36 LIST OF FIGURES

BIBLIOGRAPHY 37

Bibliography

[AIK11] Md Rezwanul Ahsan, Muhammad Ibn Ibrahimy, and Othman OKhalifa. Electromygraphy (emg) signal based hand gesture recogni-tion using artificial neural network (ann). In Mechatronics (ICOM),2011 4th International Conference On, pages 1–6. IEEE, 2011.

[AJS+05] Eishi Asano, Csaba Juhasz, Aashit Shah, Otto Muzik, Diane CChugani, Jagdish Shah, Sandeep Sood, and Harry T Chugani. Originand propagation of epileptic spasms delineated on electrocorticogra-phy. Epilepsia, 46(7):1086–1097, 2005.

[BFBG14] Simone Benatti, Elisabetta Farella, L Benini, and E Gruppioni. Anal-ysis of robust implementation of an emg pattern recognition basedcontrol. BIOSIGNALS 2014, 2014.

[braa] Braingate neural interface. https://en.wikipedia.org/wiki/

BrainGate. Accessed: 2015-07-07.

[brab] Official braingate website. http://braingate2.org/. Accessed:2015-07-07.

[Cic14] Andrzej Cichocki. Era of big data processing: a new approachvia tensor networks and tensor decompositions. arXiv preprintarXiv:1403.2048, 2014.

[CvdS09] Claudio Castellini and Patrick van der Smagt. Surface emg in ad-vanced hand prosthetics. Biological cybernetics, 100(1):35–47, 2009.

[CZMC08] Christian Cipriani, Franco Zaccone, Silvestro Micera, andMaria Chiara Carrozza. On the shared control of an emg-controlledprosthetic hand: analysis of user–prosthesis interaction. Robotics,IEEE Transactions on, 24(1):170–184, 2008.

[eeg] Eeg recording setup. http://www.psychologie.uzh.ch/

fachrichtungen/plasti/Labor/EEG-03.jpg. Accessed: 2015-07-07.

38 BIBLIOGRAPHY

[emg] The emg neural interface pipeline. http://www.intechopen.com/

source/html/18212/media/image5_w.jpg. Accessed: 2015-07-07.

[F+08] Todd R Farrell et al. A comparison of the effects of electrode implan-tation and targeting on pattern classification accuracy for prosthesiscontrol. Biomedical Engineering, IEEE Transactions on, 55(9):2198–2211, 2008.

[FYSK08] Dario Farina, Ken Yoshida, Thomas Stieglitz, and Klaus Peter Koch.Multichannel thin-film electrode for intramuscular electromyographicrecordings. Journal of Applied Physiology, 104(3):821–827, 2008.

[HBJ+12] Leigh R Hochberg, Daniel Bacher, Beata Jarosiewicz, Nicolas YMasse, John D Simeral, Joern Vogel, Sami Haddadin, Jie Liu, Syd-ney S Cash, Patrick van der Smagt, et al. Reach and grasp by peoplewith tetraplegia using a neurally controlled robotic arm. Nature,485(7398):372–375, 2012.

[HBT+15] Roni Hogri, Simeon A Bamford, Aryeh H Taub, Ari Magal, PaoloDel Giudice, and Matti Mintz. A neuro-inspired model-based closed-loop neuroprosthesis for the substitution of a cerebellar learning func-tion in anesthetized rats. Scientific reports, 5, 2015.

[HMF14] A Holobar, MA Minetto, and D Farina. Accurate identification ofmotor unit discharge patterns from high-density surface emg andvalidation with a novel signal-based performance metric. Journal ofneural engineering, 11(1):016008, 2014.

[HPS93] Bernard Hudgins, Philip Parker, and Robert N Scott. A new strategyfor multifunction myoelectric control. Biomedical Engineering, IEEETransactions on, 40(1):82–94, 1993.

[IFM14] Cinthia Itiki, Sergio S Furuie, and Roberto Merletti. Compressionof high-density emg signals for trapezius and gastrocnemius muscles.Biomedical engineering online, 13(1):25, 2014.

[JDMF12] Ning Jiang, Strahinja Dosen, Klaus-Robert Muller, and Dario Farina.Myoelectric control of artificial limbs–is there a need to change focus.IEEE Signal Process. Mag, 29(5):152–150, 2012.

[KAJAA09] Rami N Khushaba, Adel Al-Jumaily, and Ahmed Al-Ani. Evolution-ary fuzzy discriminant analysis feature projection technique in myo-electric control. Pattern Recognition Letters, 30(7):699–707, 2009.

BIBLIOGRAPHY 39

[KDL+04] Todd A Kuiken, GA Dumanian, RD Lipschutz, LA Miller, andKA Stubblefield. The use of targeted muscle reinnervation for im-proved myoelectric prosthesis control in a bilateral shoulder disartic-ulation amputee. Prosthetics and Orthotics International, 28(3):245–253, 2004.

[LAK+12] Minas V Liarokapis, Panagiotis K Artemiadis, Pantelis T Katsiaris,Kostas J Kyriakopoulos, and Elias S Manolakos. Learning humanreach-to-grasp strategies: Towards emg-based control of robotic arm-hand systems. In Robotics and Automation (ICRA), 2012 IEEE In-ternational Conference on, pages 2287–2292. IEEE, 2012.

[MCR10] Silvestro Micera, Jacopo Carpaneto, and Stanisa Raspopovic. Con-trol of hand prostheses using peripheral information. Biomedical En-gineering, IEEE Reviews in, 3:48–68, 2010.

[nat] The first successful trial of using a robotic armto drink. http://scitechdaily.com/images/

Using-thoughts-to-control-a-robotic-arm.jpg. Accessed:2015-07-07.

[Pat09] Shailaja Arjun Patil. Brain gate as an assistive and solution providingtechnology for disabled people. In 13th International Conference onBiomedical Engineering, pages 1232–1235. Springer, 2009.

[RCP+14] Stanisa Raspopovic, Marco Capogrosso, Francesco Maria Petrini,Marco Bonizzato, Jacopo Rigosa, Giovanni Di Pino, JacopoCarpaneto, Marco Controzzi, Tim Boretius, Eduardo Fernandez,et al. Restoring natural sensory feedback in real-time bidirectionalhand prostheses. Science translational medicine, 6(222):222ra19–222ra19, 2014.

[RMMA+12a] Monica Rojas-Martınez, Miguel A Mananas, Joan F Alonso, et al.High-density surface emg maps from upper-arm and forearm muscles.J. Neuroeng. Rehabil, 9(85.10):1186, 2012.

[RMMA+12b] Monica Rojas-Martınez, Miguel A Mananas, Joan F Alonso, et al.High-density surface emg maps from upper-arm and forearm muscles.J. Neuroeng. Rehabil, 9(85.10):1186, 2012.

[SCV+07] Ranganatha Sitaram, Andrea Caria, Ralf Veit, Tilman Gaber,Giuseppina Rota, Andrea Kuebler, and Niels Birbaumer. Fmri brain-computer interface: a tool for neuroscientific research and treatment.Computational intelligence and neuroscience, 2007, 2007.

40 BIBLIOGRAPHY

[SNF15] Antonietta Stango, Francesco Negro, and Dario Farina. Spatial cor-relation of high density emg signals provides features robust to elec-trode number and shift in pattern recognition for myocontrol. Neu-ral Systems and Rehabilitation Engineering, IEEE Transactions on,23(2):189–198, 2015.

[SRSVD10] Didier Staudenmann, Karin Roeleveld, Dick F Stegeman, and Jaap HVan Dieen. Methodological aspects of semg recordings for forceestimation–a tutorial and review. Journal of Electromyography andKinesiology, 20(3):375–387, 2010.

[SWC+08] Ranganatha Sitaram, Nikolaus Weiskopf, Andrea Caria, Ralf Veit,Michael Erb, and Niels Birbaumer. fmri brain-computer interfaces.Signal Processing Magazine, IEEE, 25(1):95–106, 2008.

[TSK+14] Daniel W Tan, Matthew A Schiefer, Michael W Keith, James RobertAnderson, Joyce Tyler, and Dustin J Tyler. A neural interface pro-vides long-term stable natural touch perception. Science translationalmedicine, 6(257):257ra138–257ra138, 2014.

[VBVDS13] Jorn Vogel, Justin Bayer, and Patrick Van Der Smagt. Continuousrobot control using surface electromyography of atrophic muscles.In Intelligent Robots and Systems (IROS), 2013 IEEE/RSJ Interna-tional Conference on, pages 845–850. IEEE, 2013.

[VCvdS11] Jorn Vogel, Claudio Castellini, and Patrick van der Smagt. Emg-based teleoperation and manipulation with the dlr lwr-iii. In In-telligent Robots and Systems (IROS), 2011 IEEE/RSJ InternationalConference on, pages 672–678. IEEE, 2011.

[vir] The virtu-limb from touch bionics with emg recording.http://www.touchbionics.com/sites/default/files/

image-gallery/virtu-limb_gallery4.jpg. Accessed: 2015-07-07.

[WO07] Richard Webster and Margaret A Oliver. Geostatistics for environ-mental scientists. John Wiley & Sons, 2007.