Context-Aware Learning from Demonstration: Using Camera Data toSupport the Synergistic Control of a Multi-Joint Prosthetic Arm

Gautham Vasan and Patrick M. Pilarski

Abstract— Muscle synergies in humans are context-dependent—they are based on the integration of vision, sen-sorimotor information and proprioception. In particular, visualinformation plays a significant role in the execution of goal-directed grasping movements. Based on a desired motor task, alimb is directed to the correct spatial location and the posture ofthe hand reflects the size, shape and orientation of the graspedobject. Such contextual synergies are largely absent frommodern prosthetic robots. In this work, we therefore introducea new algorithmic contribution to support the context-aware,synergistic control of multiple degrees-of-freedom of an upper-limb prosthesis. In our previous work, we showcased an actor-critic reinforcement learning method that allowed someone withan amputation to use their non-amputated arm to teach theirprosthetic arm how to move through a range of coordinatedmotions and grasp patterns. We here extend this approach toinclude visual information that could potentially help achievecontext-dependent movement. To study the integration of visualcontext into coordinated grasping, we recorded computer visioninformation, myoelectic signals, inertial measurements, andpositional information during a subject’s training a roboticarm. Our approach was evaluated via prediction learning,wherein our algorithm was tasked with accurately distinguish-ing between three different muscle synergies involving similarmyoelectric signals based on visual context from a robot-mounted camera. These preliminary results suggest that evensimple visual data can help a learning system disentanglesynergies that would be indistinguishable based solely on motorand myoelectric signals recorded from the human user and theirrobotic arm. We therefore suggest that integrating learned,vision-contingent predictions about movement synergies intoa prosthetic control system could potentially allow systems tobetter adapt to diverse situations of daily-life prosthesis use.

I. INTRODUCTION

Humans have an astounding ability to learn a varietyof motor skills, ranging from tying shoelaces, threading aneedle to shooting a basketball. A wide number of interactingelements are involved in learning such skills. In order to beproficient in multiple motor skills, humans learn to gathertask-relevant sensory information efficiently and use it indecision making and the selection of control strategies.

When any muscle or muscle group is activated, the result-ing movement is dependent on the context—the relationshipbetween muscle excitation and movement is variable and this

This research was undertaken, in part, thanks to funding from the CanadaResearch Chairs program, the Canada Foundation for Innovation, and wassupported by the Alberta Machine Intelligence Institute (Amii), AlbertaInnovates – Technology Futures (AITF), the Government of Alberta, andthe Natural Sciences and Engineering Research Council (NSERC).

G. Vasan is with Kindred Inc. P. M. Pilarski is with the Departmentof Computing Science and the Department of Medicine, University ofAlberta, Edmonton, AB T6G 2E1, Canada. G Vasan contributed duringhis M.Sc. at the University of Alberta. Please direct correspondence to:vasan@ualberta.ca; pilarski@ualberta.ca

variability is known to be context conditioned [1]. It is alsowell known that vision provides critical sensory informationin the execution of goal-directed reaching and graspingmovements. The dorsal stream in our visual cortex is directlyinvolved in estimating position, shape and orientation oftarget objects for reaching and grasping purposes. It providesus with the ability to interact with our environment in a quick,reliable fashion [2].

By contrast, the myoelectric control of artificial limbs(specifically upper-limb prostheses) relies predominantly onthe activity of muscle tissue to dictate the movement ofrobotic actuators within the prosthetic chassis [3]. Motorneurons transmit electrical signals that cause muscles tocontract. These signals, also known as electromyographic(EMG) signals, contain information about the neural signalssent from the spinal cord to control the muscles. Despitedecades of research and development, myoelectric control ofupper-limb prostheses still has not reached its full potential,as a large proportion of amputees stop using myoelectricprostheses due to non-intuitive control, lack of sufficientfeedback, and insufficient functionality [4]. A fundamentalremaining issue is that there is a significant mismatch be-tween the number of functions available in modern poweredprosthetic arms and the number of functions an amputee cancontrol at any given moment [5].

Apart from EMG signals traditionally used for the myo-electric control, there are a number of other physiologicaland non-physiological signals that could be used to assist inthe control of multiple prosthetic degrees-of-freedom (DOF).To use additional signals effectively involves building aprocessing unit which performs sensory and motor informa-tion processing to effect a more rich behavioral repertoire.Using additional sensors is important, as there are typicallya limited number of viable sites on an amputated arm forrecording EMG signals. This limits the possible space ofmovements to discrete, gross movements. We hypothesizethat a prosthesis controller can emulate highly synergisticmovements traditionally considered the responsibility of theuser if equipped with non-physiological sensors (e.g., visionand tactile sensors). By combining sensory information fromdifferent sources, we expect a controller should be able todetect and analyze the current context, plan the movementstrategy, and simultaneously and proportionally control mul-tiple DOF available in the prosthesis.

In our previous work [6], [7], we outlined a learning fromdemonstration (LfD) approach that could potentially allowan amputee to use their non-amputated arm to teach theirprosthetic arm how to move in a natural and coordinated

Fig. 1: Schematic showing the flow of information duringthe training period for a vision-enhanced LfD approach.

fashion. This could also be considered as a different typeof pattern recognition method that could include most ifnot all aspects of traditional pattern recognition methods.But our prior approach still has an upper bound on thenumber of synergistic movements that could be learnedby the reinforcement learning (RL) agent. For example, inthe case of a transhumeral amputation (someone missingtheir hand, forearm, and elbow), generated EMG signals arenot separable enough to identify various grasping patterns.While the LfD approach described in Vasan and Pilarski[6] learns to match a particular synergistic movement to theuser’s muscle signals, it cannot learn to discriminate betweencontext-dependent grasping patterns that are characterized byhighly similar EMG signals. In order to achieve situation-dependent movement based on muscle excitation, we suggestthe control system should also be given relevant contextualinformation and meta-data about the user, the robotic limband its environment.

In the present work, we explore how additional sensoryinformation can be exploited to improve synergistic controlof a multi-DOF prosthetic arm. More specifically, we equipthe prosthetic arm with artificial vision to perceive the stateof the user, prosthesis and the environment. Using thisadditional information, we expect the controller should beable to learn to distinguish between different grasp patternsaccording to the context and user’s intentions (communicatedusing EMG signals as shown in Fig. 1). We describe anonline approach based on reinforcement learning that couldpotentially achieve context-dependent motion if presentedwith contextually relevant learned or hand-designed visualrepresentations. When integrated with our prior work, webelieve such an approach can learn to use feedback fromthe user or the control environment to continually update oradapt its control policy. This implies that an amputee couldin principle teach his/her prosthesis to unlearn and relearnunique reaching and grasping patterns for different target

Fig. 2: Experimental setup which includes a high-definitioncamera, Bento Arm, Thalmic Myo Armband and Cyber-Touch II. The Bento Arm as used in our trials had 5 activeDoFs including shoulder rotation, elbow flex/extend, wristpronation/supination, wrist flex/extend and hand open/close.

objects or settings. In this preliminary study, for clarity wefirst test our hypothesis using real-time prediction learningexperiments. As described by Pilarski et al. [8], such learnedpredictions can be integrated into a prosthetic control systemin straightforward ways to enhance multi-joint prosthesisuse, or act as a method by which to select the informationthat should be presented to a more complex control learningalgorithm.

II. RELATED WORK

Markovic et al. [9] introduced a sensor fusion and com-puter vision based control approach for the context-awarecontrol of a multi-DOF prosthesis. Their work uses a com-bination of sensing units, comprising myoelectric record-ing, computer vision, inertial measurements and embeddedprosthesis sensors (position and force), to develop a con-troller that could allow a multi-DOF prosthesis to performsimultaneous, coordinated movements. The method relies onsensor fusion which allows for the perception of the user(proprioception), the environment (exteroception) and theirinteraction, leading to simultaneous, proportional control ofmultiple DOFs through context-dependent behavior. Ghaz-aei et al. also introduce a classification-based approach tograsping that leverages the representational power of deepnetworks [10].

III. METHODS

Similar to the experiments described by Vasan and Pilarski[6], we assume a setting in which our target users haveundergone a unilateral, transhumeral amputation. To providea proof of concept in this setting, we first study the casewhere a non-amputee participant has one biological hand(right hand), and one robotic arm (left hand) that they wishto train to appropriately respond to the commands beinggenerated by the muscle tissue in the user’s upper left arm(simulating the residual limb). Compared to our prior work[6], there are two significant changes in our experimentalsetup—first, we mount a Logitech HD 1080p webcam ontop of the robotic arm (see Fig. 2 (top)) and second, weuse a Thalmic Myo armband instead of the Delsys TrignoWireless system to obtain EMG signals.

A. Hardware

Bento Arm: The Bento Arm is a myoelectric training toolto assess and train upper-limb amputees in how to use theirmuscle signals prior to being fit with myoelectric prostheses[11]. We use a 5-DOF Bento Arm for the purposes of ourexperiments. It is a 3D printed prototype with DynamixelMX Series servos.

EMG Data Acquisition: We used a 8-Channel ThalmicMyo armband to record EMG signals from our subjects. TheMyo armband is fashioned as a wearable gesture controland motion control device that can be worn around theforearm/upper arm. It also provides inertial measurementswhich can be used to calculate rotation and translation withrespect to a fixed frame of reference.

Motion Capture Glove: The desired joint angle configura-tions for wrist flexion/extension (θ∗wf ) and hand open/close(θ∗h) were defined by the subject using a CyberTouch IIsystem (CyberGlove Systems LLC) worn on the hand of theirtraining arm.

Computer Vision: We used a Logitech C920 HD ProWebcam mounted on top of the Bento Arm. The cameracaptured images at 50 frames-per-seconds. Examples of theimages obtained from the camera are shown in Fig. 3.

The experimental task was divided for the participantinto multiple segments as follows: grasp the object on thetable, bring it up by flexing the elbow (as if the object isbeing examined closely), extend the arm and bring it downand finally drop it on the table. While the EMG controlcommands remained the same for flexing and extendingthe elbow, the corresponding wrist and hand trajectoriesfor manipulating each object was different. The hand, wristflexor and wrist rotator joints (denoted by θh, θf and θrrespectively) were correlated with the angular position of theelbow joint (denoted by θe) such that there exists a policythat could map any given elbow position uniquely into acombination of higher dimensional joint movements. Thegoal for the machine learning approach was to accuratelydifferentiate and predict the three different muscle synergiesshown by the user for grasping a large blue ball, a red sponge,and a smaller yellow smiley ball as shown in Fig. 3.

Fig. 3: Three different objects for our experiment—a largeblue ball, a red sponge and a small yellow smiley ball—withdifferent grasp patterns. Raw input images were processedand provided to the learning system as input features.

Phase I: Recording training data

In this phase, the able-bodied participant was instructed toexecute the task as a repetitive sequence of simple reachingand grasping movements that were mirrored by both theircontrol and training arms (for an amputee, this would corre-spond to trying to perform identical movements using theirnon-amputated arm and the prosthetic arm). The training armdemonstrated the desired movement and grasp pattern to theprosthetic arm. During training, the elbow of the Bento Armwas actuated via proportional myoelectric control from thesubject’s control arm, while the wrist and hand of the BentoArm were actuated via direct teleoperation—i.e., the BentoArm copied the training arm’s movements as reflected to thecontralateral side. As shown in Fig. 1, Fig. 2 (bottom) anddescribed above, we recorded computer vision information,EMG signals from the Myo Armband and desired anglesfrom the subject’s training arm (wrist and finger joints) usingthe motion capture glove and inertial measurement system.The participant demonstrated each reaching and graspingmovement for ∼ 15min. The entire trial lasted ∼ 45min.

This task was similar to the one described in Vasan andPilarski [6], in which we had our participants demonstratea single, desired movement and grasp pattern in 3 DOFto an actor-critic control learner. In this experiment, ourparticipant instead demonstrated the wrist and grasp patternsfor manipulating three different objects.

B. Learning Predictions with Temporal-Difference Methods

Temporal difference (TD) learning uses changes or differ-ences in predictions over successive time steps to drive thelearning process. It learns how to predict a quantity that de-pends on future values of a given signal [12]. TD algorithmsare most commonly used in reinforcement learning to predictthe expected return of a reward signal.

More recently, Sutton et al. described a generalized predic-tion learning approach based on temporal-difference learning[13]. They were successful in learning temporally extendedpredictions about non-reward sensorimotor data. It has alsobeen demonstrated that learned temporally extended pre-dictions can accurately forecast signals during prosthesiscontrol by both amputees and able-bodied subjects [8],[14]–[16]. Predictions were phrased as a linear combinationof a learned weight vector, here denoted wp, and a stateapproximation φp of current and next states sp and s′prespectively. Predictions P for a given signal rp were thencomputed using Pp = wp

Tφ, where wp was updated on eachtime step according to the TD error δp using the followingincremental procedure, with parameters αp (learning rate), γp(discounting parameter) and λp (decay rate of the eligibilitytrace ep):• δp ← rp + γpwp

Tφp(s′p)− wpTφp(sp)

• ep ← γpλep + φp(sp)• wp ← wp + αpδpep

IV. REAL-TIME PREDICTION LEARNING USING GENERALVALUE FUNCTIONS (GVFS)

In this experimental setting, the EMG control signals fromthe user are extremely (and deliberately) similar across thethree context-dependent control tasks. Artificial vision is theonly distinguishing input feature that could help a learningsystem differentiate between the tasks. Learning accuratepredictions about the desired target trajectories for eachobject gives a clear measure by which to show a system’sability to ascertain this simple form of context. As such, weuse standard temporal-difference learning [12] of GeneralValue Functions (GVFs, [13]) general to allow the system tomake predictions about the desired joint angles.

We created three GVFs for predicting the three signalsof interest θ∗j in the robotic system. We make temporallyextended predictions at a short time scale (1.0s) about threetarget joint angles—wrist rotation, wrist flexion and handopen/close. The learning agent was presented with a signalspace consisting of the following:• Elbow joint angle and velocity 〈θe, θ̇e〉• Object specific features: input rgb images were con-

verted into hsv format and analyzed for how oftencertain colors appear and in what proportions they are to

be found in different types of images. As a very simplecontextual representation based on visual features notavailable to standard prosthetic hardware, the dominantrange of ‘r’, ‘g’ and ‘b’ values of the correspondingobjects was then used to classify the target object. E.g.,based on the number of blue pixels within a particularthreshold, target objects could be uniquely classified andused as a single integer signal to the learner.

• The EMG control signal, given as the difference be-tween the mean-absolute-value of the EMG signalsobtained from Myo sensors 3 and 8 (placed directlyover the biceps and triceps respectively).

We used tile coding for linear function approximation [12].Our state representation consisted of 32 incrementally offsettilings (width=1) for better generalization. Each tiling had aresolution level NR = 10. The binary feature vector of length5,000,000 was hashed down to a memory size of 8192 andwe also added a bias unit which was always active. At everytimestep, 4 continuous signals were provided to the tile coderand m = 33 features were active. The learning parameterswere set as follows: α = 0.1/m, γ = 0.99, λ = 0.7. Weightvectors w, e were initialized to zero. Each GVF received itstarget joint angle θ∗ as the cumulant rp.

Performance of the learning system was measured basedon its ability to predict desired joint angles. All learningalgorithms were run on a Lenovo Y700 Laptop with IntelCore i7-6700HQ @2.60GHz x 8 and 8GB RAM. We usedthe Robot Operating System (ROS) Kinetic on Ubuntu 16.04to send and receive information and commands from theBento Arm, CyberTouch II and the Thalmic Myo armband.All sensorimotor information was communicated betweendifferent systems using ROS topics. We recorded all thesensorimotor information (from ROS topics) using rosbags.Rosbags avoid deserialization and reserialization of the mes-sages. After recording, we can playback the data in a timesynchronized fashion and simulate real-time sampling andlearning conditions for different hyper-parameter choices.

V. RESULTS

As shown in the results, the system was able to success-fully anticipate the joint trajectories initiated by the subjectfor all three different objects. Accurate predictions wereobserved after 5 − 6min of real-time sampling (simulatedby playing back recorded data and synchronized using times-tamps) and learning. Here the normalized predictions P̄p at atime scale of 1.0s (colored lines) are compared against cor-responding target joint angles (grey lines). The prediction Pp

is dependent on the timescale (i.e., time constant) of returnpredictions determined by γ (i.e., the discount factor). Forcomparison with raw signals, normalized return predictions(P̄p) are therefore scaled according to the time constant, i.e.,P̄p = Pp ∗ (1 − γp). Fig. 4 shows an example of wristrotation angle prediction after two offline learning passesthrough 10mins of recorded training data from the subjectdemonstrations. The agent was tasked with learning threetarget joint angles in parallel—wrist flexion, wrist rotationand hand open/close. The training phase lasted for ∼ 90mins

Fig. 4: Comparison of target (grey line) and predictions (colored lines) of wrist rotation trajectories over training and testingperiods. This plot shows the joint angle predicted by the TD learner for the able-bodied subject during training and testing.

Fig. 5: Comparison of target (grey line) and predictions (colored lines) of wrist flexion trajectories over training and testingperiods. This plot shows the joint angle predicted by the TD learner for the able-bodied subject during training and testing.

Fig. 6: Comparison of target (grey line) and predictions (colored lines) of gripper hand trajectories over training and testingperiods. This plot shows the joint angle predicted by the TD learner for the able-bodied subject during training and testing.

(two offline passes through 15mins of recorded data for eachdemonstration) and the testing phase lasted for ∼ 15mins(one offline pass through 5mins of recorded data for eachdemonstration). Fig. 5 and 6 show examples for normalizedjoint angle predictions for wrist flexion and gripper handrespectively over training and testing periods.

VI. DISCUSSION

A. Prediction in Adaptive Control

Highly skilled motor behavior relies on our brain learningboth to control its body as well as predict the consequencesof this control. Flanagan et al. studied the relation betweenpredictions as control during motor learning [17]. They founddifferent time scales of learning for predictions and control,with predictions being learned much faster than control.Pilarski et al. integrated learned anticipatory predictionsinto the control the actuators of a multi-joint prosthesis foruse by amputees, especially amputees with limited signalrecording sites on their amputated limbs [8]. They wereable to make accurate, anticipatory predictions at differenttimescales about various joint angles, dynamics, and EMGsignals. Their integration of real-time prediction and controllearning promises to speed up control policy acquisition,allow unsupervised adaptation in myoelectric controllers, andfacilitate coordinated, synergistic movements in a mutli-DOFprosthetic limbs. In this paper, we test the ability of oursystem to learn accurate, temporally abstracted predictionsabout the actuator positions of the joints controlled by thelearning agent when supplemented with contextual visualinformation. In the future, we hope to implement a similarintegration of real-time prediction learning and control tolearn a larger repertoire of motor behaviors, either throughthese predictions being provided to a control system as state

information (as per Pilarski et al. [8]), or via the identificationof more complex visual features that can be provided directlyto the control learning approach of Vasan and Pilarski [6].

B. Sensor Fusion for Context-Aware Control

As discussed in Vasan and Pilarski [6], the LfD paradigmis not applicable only to prosthetic arms, but could also beextended to wearable robots such as exoskeletons, poweredorthotics, supernumerary limbs, functional electrical stimula-tion systems, and other assistive rehabilitation technologies.While these devices face challenges in both hardware andsoftware design, a major challenge is that the robot usuallylacks the capability to adequately recognize the actions andintentions of the human user. Consequently, it cannot assistthe user appropriately, a drawback that has been especiallyemphasized in the rehabilitation robotics domain.

In most wearable robots, many sensors are alreadybuilt into the device, such as joint angle sensors, electro-physiological measurements such as electromyography(EMG) or electroencephalography (EEG), or alternativelymechanical sensors or inertial measurement units (IMUs)placed on a part of the body that is not covered by thewearable robot. We suggest that, in the same way as visionwas used in the present work, we can and should combinethis multi-modal information (combining different sensortypes) to better learn and adapt to the needs of the user.

In the rehabilitation robotics domain, the degree of control(DOC) for a device can greatly outnumber the number ofinput channels the user has available. For example, in thecase of an amputee user, the disparity between the DOCand the number of available input signals greatly increasesas the level of amputation increases (e.g., those with tran-shumeral amputations can provide even fewer control signals

than those with transradial amputations). Unfortunately, thismismatch makes the control of wearable robots difficult andtedious. As suggested here for the case of visual sensingduring grasping, sensor fusion can potentially alleviate someof the issues associated with controlling large DOFs with asmall subset of input signals.

In addition to the standard electro-physiological signals,IMUs and joint angle measurement units, it may be fruitful toadd artificial vision, gaze vectors (to know where the user islooking) and tactile sensation systems (for example, a cameraand capacitive touch systems respectively) to robotic devicesprostheses. These systems could provide useful sensoryinformation to the learning agent that could be used to betterperceive the environment and needs of the user.

C. Representation Learning

In this paper, we showed that it is feasible to learn todistinguish between the desired grip trajectories for threedifferent objects by the addition of very simple vision-basedfeatures. A main contribution of this work is to highlight theutility vision-based features can have for building contex-tual LfD algorithms in the prosthetic setting. As would beexpected, altered lighting, displacements and other factorswould not be well handled by the current proof-of-conceptlearning representation used in the present work. While outof scope for the present comparisons, and as describedbelow, there are numerous state-of-the art representationalapproaches, both learned and engineered, that could beimplemented to well support prediction and control learningin prosthetic LfD.

Modern day prostheses could receive a huge density ofdata about the user, their physiological and psychologicalneeds and their environment. For example, camera data oreven additional sensors on the socket of a prosthesis canreadily provide enough contextual information to allow anactor-critic RL system to produce varied motor synergiesin response to similar EMG signals from the user—e.g., asystem can use additional sensor and state information tohelp manage the user’s degree-of-freedom problem, gener-ating synergies that artfully align to different situations inthe user’s daily life. It is therefore important that efficientways of structuring prosthetic data are developed to betterrepresent context to a machine learning prosthetic controlsystem without facing the curse of dimensionality. For exam-ple, representation learning methods built on Kanerva Codingcould potentially be used to better handle this large numberof real-world state signals [18].

While the idea of using a single state representationto better leverage the multi-modal sensory information isextremely appealing, it’s been shown that different functionapproximators can be better at learning about different typesof data. For example, convolutional neural nets (CNNs) havebeen widely successful in image classification and objectdetection datasets [19]. Similarly, motor primitives have beensuccessful in encoding rhythmic and discrete movements.Recurrent neural networks (RNNs) have also been extremelysuccessful in speech recognition and text translation [20].

Though with its own inherent challenges, one simple way tocombine all the sensory information is to extract the featuresfor each modality separately, then input all features into asingle sensor fusion algorithm.

D. Study Limitations and Future Developments

The experimental evaluation in this study was designedspecifically to test the hypothesis that a learning system’sability to make accurate predictions about context-dependentjoint trajectories in the prosthetic LfD setting will increasewhen supported by visual sensor information or visualscene features. Therefore, the scope of this pilot study waslimited to having our participant perform a discrete set offunctionally relevant movements (e.g., grasping and liftingdistinctly colored, differently oriented objects) designed toapproximate those facing a multi-DOF prosthesis user, whilealso taking into account the constraints of the setup (e.g.,desk mounted Bento Arm, and vision sensor mounted on topof the arm instead of on the socket or the terminal device).We are fully aware of the importance of and state of the artin standardized evaluations of the functional effectiveness ofpowered prostheses, and ongoing work is exploring how ourLfD approach generalizes across different tasks, objects, andsettings via validated eye and motion capture metrics.

VII. CONCLUSION

Our approach was able to learn contextually-accuratepredictions from joint trajectories demonstrated by an able-bodied user though the integration of simple visual featuresinto the state space of the prediction learning algorithm.These results therefore suggest important extensions to thegenerality of our previously published methods for LfD inthe prosthetic setting, wherein a prosthetic user with onlylimited control channels at their disposal may still be ableto effect a variety of synergistic, coordinated movements ina situation-appropriate fashion. This suggestion remains tobe demonstrated in practice through precise control learningfollow-up experiments, and provides a path to research intolong-term control adaptation. An interesting extension to thiswork is to explore the use of contextual-prediction archi-tectures along with ACRL or alternative control approachesin complex real-world activities with gold-standard outcomemetrics, and to further evaluate predictions for context-awarecontrol adaptation with a population of participants withamputations.

ACKNOWLEDGEMENT

We thank Craig Sherstan, Kory Mathewson, Michael Rory Dawson,Jaden Travnik, McNiel Keri and Vivek Veeriah for many helpful discussions.

REFERENCES

[1] M. T. Turvey, H. L. Fitch and B. Tuller, “The Bernstein perspective,I: The problems of degrees of freedom and context-conditionedvariability,” in Human motor behaviour: An introduction, Ed. J.A.S.Kelso. Hillsdale, NJ: Lawrence Erlbaum Associates, 1982.

[2] E. Chinellato, A.P. del Pobil, “The Neuroscience of Action andPerception.” In: “The Visual Neuroscience of Robotic Grasping,”Cognitive Systems Monographs, vol 28. Springer, Cham, 2016.

[3] C. Castellini, P. Artemiadis, M. Wininger, et al., “Proceedings ofthe first workshop on peripheral machine interfaces: Going beyondtraditional surface electromyography,” Frontiers in Neurorobotics, vol.8, no. 22, Aug. 2014.

[4] B. Peerdeman, D. Boere, H. Witteveen, et al., “Myoelectric forearmprostheses: State of the art from a user-centered perspective,” J. Rehab.Res. Dev, vol. 48, no. 6, pp 719–738, 2011.

[5] C. Sherstan, “Towards prosthetic arms as wearable intelligent robots,”M.Sc Thesis, University of Alberta, Dept. Computing Science, 2015.

[6] G. Vasan and P. M. Pilarski, “Learning from demonstration: Teaching amyoelectric prosthesis with an intact limb via reinforcement learning,”Proc. IEEE Int. Conf. on Rehab. Robotics (ICORR), London, 2017, pp.1457–1464.

[7] G. Vasan,“Teaching a powered prosthetic arm with an intact arm usingreinforcement learning.” M.Sc Thesis, University of Alberta, Dept.Computing Science, 2017.

[8] P. M. Pilarski, T. B. Dick, and R. S. Sutton, “Real-time predictionlearning for the simultaneous actuation of multiple prosthetic joints,”in Proc. IEEE Int. Conf. on Rehab. Robotics (ICORR), Seattle, USA,June 24–26, 2013, pp. 1–8.

[9] M. Markovic, S. Dosen, D. Popovic, B. Graimann and D. Farina,“Sensor fusion and computer vision for context-aware control of amulti degree-of-freedom prosthesis,” Journal of Neural Engineering,vol. 12, no. 6, 2015, pp. 066022.

[10] G. Ghazaei, A. Alameer, P. Degenaar, G. Morgan, and K. Nazarpour,“Deep learning-based artificial vision for grasp classification in myo-electric hands,” Journal of Neural Engineering, vol. 14, no. 3, 2017,pp. 036025.

[11] M. R. Dawson, C. Sherstan, J. P. Carey, et al., “Development ofthe Bento Arm: An improved robotic arm for myoelectric trainingand research, in Proc. of MEC’14: Myoelectric Controls Symposium,Fredericton, New Brunswick, August 18–22, 2014, pp. 60–64.

[12] R. S. Sutton and A. G. Barto. Reinforcement learning: An introduction.MIT Press Cambridge, Massachusetts; London, England, 1998.

[13] R. S. Sutton, J. Modayil, M. Delp, et al., “Horde: A ScalableReal-time Architecture for Learning Knowledge from UnsupervisedSensorimotor Interaction,” Proc. of 10th International Conferenceon Autonomous Agents and Multiagent Systems (AAMAS), May 2-6,Taipei, Taiwan, 2011, pp. 761–768.

[14] P. M. Pilarski et al., “Adaptive artificial limbs: a real-time approach toprediction and anticipation,” IEEE Robotics & Automation Magazine,vol. 20, no. 1, pp. 53–64, March 2013

[15] A. L. Edwards, J. S. Hebert, P. M. Pilarski, “Machine learningand unlearning to autonomously switch between the functions ofa myoelectric arm,” 6th IEEE RAS/EMBS International Conferenceon Biomedical Robotics and Biomechatronics (BioRob), June 26-29,2016, Singapore, pp. 514-521.

[16] A. L. Edwards, M. R. Dawson, J. S. Hebert, C. Sherstan, R. S. Sutton,K. M. Chan, P. M. Pilarski, “Application of real-time machine learningto myoelectric prosthesis control: A case series in adaptive switching,”Prosthetics & Orthotics International. vol. 40, no. 5, 573-581, 2016.

[17] J. R. Flanagan, P. Vetter, R. S. Johansson, and D. M. Wolpert,“Prediction precedes control in motor learning,” Current Biology, vol.13, no. 2, pp. 146–150, 2003.

[18] J. B. Travnik and P. M. Pilarski “Representing high-dimensionaldata to intelligent prostheses and other wearable assistive robots: Afirst comparison of tile coding and selective Kanerva coding”, 2017International Conference on Rehabilitation Robotics (ICORR), pp.1443–1450, 2017.

[19] A. Krizhevsky, I. Sutskever, G. E. Hinton, “Imagenet classificationwith deep convolutional neural networks”, Advances in Neural Infor-mation Processing Systems 25, 2012, pp. 1097–1105.

[20] A. Graves, A.-r. Mohamed, and G. Hinton, “Speech recognitionwith deep recurrent neural networks”, 2013 IEEE Int. Conference onAcoustics, Speech and Signal Processing, May 2013, pp. 6645–6649.