Embed Size (px)
Citation preview
DIVERSE STUDIES WITHIN A MULTI LEVEL
CONTROLLER FRAMEWORK OF PROSTHETIC KNEE
BY
YONATAN HUTABARAT
A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE
REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE
( ENGINEERING AND TECHNOLOGY )
SIRINDHORN INTERNATIONAL INSTITUTE OF TECHNOLOGY
THAMMASAT UNIVERSITY
ACADEMIC YEAR 2018
Ref. code: 25615822043435QIF
DIVERSE STUDIES WITHIN A MULTI LEVEL
CONTROLLER FRAMEWORK OF PROSTHETIC KNEE
BY
YONATAN HUTABARAT
A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE
REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE
( ENGINEERING AND TECHNOLOGY )
SIRINDHORN INTERNATIONAL INSTITUTE OF TECHNOLOGY
THAMMASAT UNIVERSITY
ACADEMIC YEAR 2018
Ref. code: 25615822043435QIF
ii
Acknowledgements
I would like to express my sincerest gratitude to my advisor, Assoc. Prof.
Dr. Waree Kongprawechnon and also to my co-advisor Dr. Kittipong Ekkachai for their
patience, motivation, support, many great chances, and overall precious guidance and
knowledge throughout my research and study since the very first day.
Beside my advisors, I would also like to thank my thesis committees, Asst.
Prof. Dr. Itthishek Nilkamhang and Dr. Wutthiphat Covanich for their constructive
questions, valuable comments and precious suggestions.
I would also like to thank all of SIIT staffs, particularly staffs of the School
of ICT for their kind help and hospitality during my study.
I am also grateful for study and research funding sources, the Thammasat
University (TU) – Excellent Foreign Student (EFS) Scholarship for granting a full
scholarship on my graduate studies, also the National Research University Project by
Office of the Higher Education Commission and Thammasat University for partly
funding my research.
Most important above all, a never ending thankfulness I offer to Almighty
God, Lord Jesus Christ, for the blessing, guidance and strengths in every step of my life
and in any challenges I faced so far.
Finally, I would like to thank my dearest family; my dad, my mom, my
brother and sister, and my beloved girlfriend for their patience, love, understanding, and
constant support during my study overseas. This thesis is dedicated to them.
Ref. code: 25615822043435QIF
iii
Abstract
DIVERSE STUDIES WITHIN A MULTI LEVEL CONTROLLER FRAMEWORK
OF PROSTHETIC KNEE
by
YONATAN HUTABARAT
Bachelor of Engineering, Universitas Gadjah Mada, 2015
Master of Science, Sirindhorn International Institute of Technology, 2018
This thesis covers a diverse study within the proposed framework of multi
level controller on a prosthetic knee device. Intent recognizer as high level controller is
presented with a study case in sit to stand movement using the proposed method in the
form of heuristic experimental based rule transition. Impedance based controller as
midde level controller is proposed using the non-parametric model in the form of
feedforward neural network to estimate the necessary knee torque needed. Lastly,
reinforcement q-learning control as the low level control is proposed to generate
necessary input voltage to the MR damper device, with a constrain only in a swing
phase of gait cycle.
The result shows that the heuristic experimental based rule transition can
clearly detect the phase transition in sit to stand movement and also can early detect the
critical seat-off event using only ground reaction forces and knee angle data. In middle
level controller, the result shows that the feedforward neural network with double
staged delayed input node structure and using less input parameter can estimate the
necessary knee torque in a gait cycle within 2.67% of NRMSE. Meanwhile, a model
free reinforcement q-learning control proposed with user designed reward function as
the low level controller shows that this control structure can adapt to various walking
speed and have an overall better performance compared to the open-loop control, while
some of walking speed perform better than the neural network predictive control.
Keywords: multi level control, prosthetic knee, heuristic approach, feedforward neural
network, reinforcement q-learning control.
Ref. code: 25615822043435QIF
iv
Table of Contents
Chapter Title Page
Signature Page i
Acknowledgements ii
Abstract iii
Table of Contents iv
List of Tables vi
List of Figures vii
1 Introduction 1
1.1 Statement of Problem 1
1.2 Purpose of Study 2
1.3 Scope of Thesis 3
1.4 Thesis Structure 4
2 Literature Review 5
2.1 Intent Recognizer in Human Movement 5
2.2 Control Algorithm in Prosthetic Knee 6
3 Heuristic Experimental Based Approach : Study Case in Sit to Stand
Movement
9
3.1 Sit to Stand Movement 9
3.2 Experimental System 9
3.2.1 Data Collection and Processing 11
3.3 Rule Based Intent Recognizer 12
3.4 Result and Analysis 15
4 Parametric and Non Parametric Approach : Study Case in Knee
Torque Estimation
19
Ref. code: 25615822043435QIF
v
4.1 Dynamics Modelling 19
4.1.1 Two Degree of Freedom Newtonian Calculation Approach 19
4.1.2 Four Degree of Freedom Lagrangian Calculation Approach 20
4.2 Neural Network Based Modelling 22
4.2.1 Model Design and Data Preparation 23
4.2.2 Training Algorithm and Strategy 24
4.3 Simulation and Analysis 25
5 Machine Learning Approach : Reinforcement Learning Control in
Swing Phase
30
5.1 System and Environment Model 30
5.1.1 System Description 30
5.1.2 Environment Model 32
5.1.3 Data Used 33
5.2 Q-Learning Control 34
5.2.1 Q-Learning 34
5.2.2 User Designed Reward 35
5.3 Simulation and Results 38
6 Conclusions and Recommendations 46
6.1 Conclusions 46
6.2 Recommendations 47
References 49
Appendix 52
Appendix A 53
Ref. code: 25615822043435QIF
vi
List of Tables
Tables Page
3.1 Rule based of detection events in STS movement 15
3.2 STS time with event detection and quantitative analysis of related
parameter on ten performed trial
16
4.1 Case model presented with input variation 23
4.2 Selected optimum size of hidden nodes and its corresponding training
MSE for every presented case
26
4.3 NRMSE achieved by each case in every walking speed test data 26
4.4 Comparison between methods presented in estimating knee torque 29
5.1 Comparison between user adaptive, NNPC, and q-learning control 44
Ref. code: 25615822043435QIF
vii
List of Figures
Figures Page
1.1 Multi level controller framework proposed in this thesis 3
3.1 Experimental set up in a gait laboratory utilizing eight sets of Vicon
MX-T series and two ANTI force plates
10
3.2 Flowchart of data processing used in this experiment 11
3.3 Applied FIR 4th order low-pass Butterworth filter to the GRF data 12
3.4 Initiation of forward body sway represented by CLAV and STRN
markers in torso with its effect to GRF-AP on subject’s foot
13
3.5 Body weight transfer from sit to stand position represented by CoM
coordinate that fall between the LTOE and LANK position
14
3.6 Dataset T1 used in this study consist of θK (top), GRF-AP (middle),
and GRF-Ver (bottom) with the phase transition event E1-E3
17
3.7 Time detection comparison between the presented rule based method
and true seat off measurement from sit force plate
18
4.1 Free body diagram of two DOF model 20
4.2 Four DOF model evaluated with Lagrangian approach 21
4.3 Structure of FNN used to estimate knee torque 24
4.4 The effect of hidden nodes size to MSE on each presented cases 25
4.5 Comparison between FNN prediction and validation data with
absolute error for (a) Case 1, (b) Case 2, (c) Case 4 and (d) Case 6.
Vertical dash line indicate toe off event and mark the phase transition
27
5.1 Structure of MR damper model 31
5.2 Controller structure of MR damper 31
5.3 Double pendulum as environment model to simulate swing phase 32
5.4 Knee angle data used at swing phase with various speed 33
5.5 General reinforcement learning structure 34
5.6 t as an exponential function with 4n 36
5.7 The proposed user designed reward function as a function of tE 38
Ref. code: 25615822043435QIF
viii
5.8 Block diagram of the proposed q-learning control 39
5.9 Effect of learning rate with constrained iteration to NRMSE 40
5.10 Effect of learning rate to convergence and NRMSE 41
5.11 Training process of multispeed under one control policy simulation 42
5.12 Comparison between the proposed q-learning control (black line),
user-adaptive control (green dashed line), and NNPC (red line)
44
Ref. code: 25615822043435QIF
1
Chapter 1
Introduction
1.1 Statement of Problem
Analysis of various discrete movements is important to evaluate a certain related
biomechanics parameter involved in creating those movement. Those analysis could be
used further to various biomechanics related application such as recognizing a certain
movement performed by the subject, estimating biomechanics parameter that are hard
to directly measure, and also constructing a control framework and algorithm for
medical devices such as prosthetic knee.
Prosthetic knee is used to replace the knee function in trans-femoral amputee. It
is aimed to mimic the biological knee function for weight support and locomotion.
From the perspective of prosthetic knee mechanism, it can be divided into two major
categories, i.e. pure mechanical and computerized prosthetic knee. Based on the
actuator used, computerized prosthetic knee can be divided into semi-active prosthetic
which use the actuator that can only generate passive force such as magnetorheological
(MR) damper, and also active prosthetic that can generate both active and passive force
needed during executing certain movements. One of the advantage of using
computerized prosthetic knee is the flexibility to design the control algorithm suitable
for desired application. There are many control algorithm has been designed for this
type of prosthetic knee [1-11].
In control theory, modelling of the system is one of the important aspect.
Generally, there are two main approach to model a system, namely parametric approach
and non-parametric approach. Parametric approach usually employs a number of
mathematical equations derived from the actual system. The higher the complexity of
a system, the more parameters required to be defined in advance to solve those
equations. The challenge occurred when some of the parameters are hard to obtain.
Approximation based on certain knowledge and experience of some parameter is
usually done to compensate this problem, but this technique can lead to bad model
design. On the other hand, non-parametric approach can overcome this matter. A
commonly used method in non-parametric modelling is neural network. Neural
Ref. code: 25615822043435QIF
2
network typically learns the relation between input and output of a system. It does not
require the detail knowledge of a system, thus physical parameters is not necessary
known as it will learn through training process from the input-output relation. The
important aspect to be noted of using this approach is gathering the data that can cover
the overall system representation to train the network.
On the biomechanics point of view, human movement is governed by the
coordinated musculoskeletal system and nervous system. Human nervous system
recognizes our intent to do specific movement and the musculoskeletal system will act
accordingly to our specific intention and coordinate related joints and muscles to
perform the movement. On the case of trans-femoral amputation as the main concern
of this thesis topic, the amputee loss one of the most important joint in human body, the
knee joint, to support body weight and perform locomotion. Thus, to mimic the
biological knee, the prosthetic knee used by the amputee should be able to replace the
function of the loss nervous and musculoskeletal system on the leg.
It can be concluded that in designing control system prosthetic knee, a
comprehensive framework must be developed to accommodate not only from the
control and system point of view, but also the biomechanics point of view.
1.2 Purpose of Study
In this thesis, diverse studies are conducted to explore various techniques
regarding the proposed control framework in prosthetic knee depicted in Figure 1.1.
The control framework is proposed as multi level controller, which consists of high
level controller designed to function as intent recognizer, middle level controller that
influenced by the intent recognizer and also depends on whether it is designed as
trajectory or impedance based control, and lastly a low level controller designed to
calculate the necessary voltage command to the prosthetic knee system based on the
information obtained from both of previous controller and prosthetic signals/
measurements.
Intent recognizer studied in this thesis is narrowed down to a specific discrete
movement, i.e. sit to stand movement. An experiment on sit to stand movement is
designed and performed to capture related trajectories and forces data to build a
heuristic experimental based rule transition aimed to used less parameters and suitable
Ref. code: 25615822043435QIF
3
for prosthetic knee application as the intent recognizer on this specific movement. On
the other hand, impedance based controller is also studied as the middle level controller.
A non-parametric model in the form of feedforward neural network is proposed to
estimate knee torque during gait cycle. Several model is constructed using the available
gait data with different number of input to be compared in terms of the performance
index and the effect of certain input used in the model. Qualitative comparison between
the presented non-parametric model and the dynamics model are also presented. Lastly,
a machine learning algorithm is proposed as the direct actuator controller on this thesis,
where MR damper is studied as the actutator. The proposed controller is then compared
to the traditional open-loop control and existing machine learning control algorithm.
Figure 1.1 Multi level controller framework proposed in this thesis
1.3 Scope of Thesis
Scope of thesis is generally within the framework of multi-level controller of
prosthetic knee, where each level is studied seperately instead of unifiedly. Scope is
divided into three study cases presented on chapter 3 to chapter 5, which specifically
aimed on the following points:
1. To propose a heuristic, experimental derived rule transition as intent recognizer
in sit to stand movement,
Ref. code: 25615822043435QIF
4
2. To propose a non-parametric approach in estimating knee torque throughout a
gait cycle and qualitatively compare to parametric approach,
3. To explore a possibility of direct reinforcement learning control to semi-active
prosthetic knee in swing phase.
1.4 Thesis Structure
This thesis is organized following the structure as follows :
Chapter 1 explains the problem statement, purpose of this study and also scope
along with limitations of the study.
Chapter 2 discusses the review of existing method and approach on each study
case presented in this thesis, i.e. sit to stand movement with heuristic rule based
method, comparison of parametric and non-parametric approach on estimating
knee torque, and reinforcement q-learning control of prosthetic knee in swing
phase.
Chapter 3 proposes a heuristic rule based approach directly derived from
experiment data to detect the sit to stand transition.
Chapter 4 proposes a non- parametric approach constructed from experiment
data to estimate knee torque, and qualitatively compare it to parametric
approach.
Chapter 5 explores a machine learning based control approach in the form of
reinforcement q-learning to directly control a semi-active prosthetic knee device
in swing phase.
Chapter 6 provides concluding remarks and recommendations for future
research direction.
Ref. code: 25615822043435QIF
5
Chapter 2
Literature Review
In the previous chapter, it is stated that diverse studies within the proposed
control framework are the main objective of this thesis. In this chapter, several literature
review are conducted as the first step of doing this study. Literature review covered in
this chapter are consists of the study on intent recognizer in human movement, existing
control algortihm in prosthetic knee, and also tools used in this study such as various
application of neural network.
2.1 Intent Recognizer in Human Movement
In this thesis, sit to stand movement is the movement to be analyzed. There are
several studies had been investigated regarding intent recognition in this movement.
Banerjee et al [12] used fuzzy clustering techniques to detect the transition between STS
from a sequence of image frames. They used silhouette extraction technique that
employs shadow removal for greater accuracy and then extract the image moments. The
technique employed in this paper is an image processing based technique that are
suitable for fall risk assessment in elderly care but will not be practical for prosthetic
application.
Zijlstra et al. [13] used a time series analysis technique to detect STS transition
based on a novel body fixed sensor compared to the force plate based analysis in older
adults and patients with Parkinson’s disease. The novel body fixed sensor were placed
at vertebrae L2-L4 to measure the parameters of trunk movement such as vertical
velocity and acceleration of the trunk. These parameters then analyzed in a time series
manner while the subjects were doing the STS movement to define the parameters at
changing events. The force plate based analysis were using the force plate beneath the
chair as the basis of their seat off event. While the time series techniques employed in
this paper demonstrated a high agreement between both techniques in detecting the
transition, this also will not be practical to the prosthetic application as the placement of
the sensors were in the trunk of the subject and the force plate based analysis depends
on the force plate beneath the chair.
Ref. code: 25615822043435QIF
6
Varol et al. [14] used a Gaussian Mixture model based intent recognizer to
distinguish sitting from standing and develop a finite state based controller to adjust the
powered transfemoral prosthesis. In this study, the data that were used to build the model
is consists of seven signals, i.e. joint positions along with velocities at the knee and ankle,
forces at heel and ball of the foot, and sagittal plane moment at the socket.
One of the objective of this study is to develop a rule based algorithm to detect
the phase transition in STS movement using less parameters consists of knee angle and
ground reaction force (GRF) data. Design of this study is a single subject experimental
design performing STS movement. Data of STS movement were recorded using a
motion capture system and force plate. The obtained data were used to develop a rule
based algorithm to detect the STS transition.
2.2 Control Algorithm in Prosthetic Knee
There has been various studies regarding control algorithm in prosthetic knee.
Kalanovic et al. [1] proposes a feedback error learning (FEL) neural network approach
as the control structure of a powered prosthesis. The advantages of using this approach
are it can be used to identify inverse dynamics of a simple single joint movements of an
arbitrary trans-femoral prosthesis, which can be used to track an arbitrary trajectory or a
specific walking patern. However, to acheive the convergence of neural network
weights, learning rate need to be adjusted as it is very sensitive and no known method
other than trial and error that can guarantee the weights to converge to the best value.
Another advantage is the good response of the controller when a peturbation such as
change in hip torque or ground reaction generated by the user, even though network have
to be retrained to account for the known peturbation.
Another study conducted by Herr and Wilkenfeld [2] propose a user-adaptive
control for a variable-damping electronic knee. This approach utilizes data from sensors
located on the knee axis to adapt the knee damping value to match with the amputee's
gait. The result show that the proposed controller perform better to match with biological
gait compared to the mechanical passive knee. However, this structure lack of the ability
to recognize user intent because of only local sensor on the knee is being used. Another
thing to note is that this control structure is an open-loop control.
Ref. code: 25615822043435QIF
7
A study combining four-bar link mechanism that often used in passive knee with
MR damper is presented in [3]. In this study, modelling and control proposed are a
parametric approach, thus contains many parameters to be defined. The result show that
the intellegent control proposed in this study is able to follow the gait tracking of the
healthy side of amputee leg, although there is a certain delay.
MR damper has been widely used in various application. In robotics field, the
study investigated by Garcia et al. [4] proposed a combination of MR damper and series
elastic actuation for locomotion control for all-terrain robot. There are two control
schemes used in this study, i.e. direct joint force control employing a PID control to
generate current command to the amplifier module and a cascade controller in the
amplifier module. The result show that the proposed combined actuator and control can
acheive a natural looking motion and also can reduce 20% of power in braking knee
mechanism. In a biomechanics field, MR damper has been used as a variable impedance
knee mechanism in orthosis for a specific stance flexion control in a pathological gait as
investigated by Bulea et al. in [5].
A variable stiffness control as proposed by Wentink et al. [6] has been
investigated in modelling study of prosthetic knee to restore knee buckling during
stance. Rotational stiffness is controlled to prevent excessive knee flexion by increasing
the rotational stiffness. This study show a useful addition of control strategy in stance
phase.
In stance phase control, knee torque is one of the important parameter to be
evaluated. In prosthetic application, applied torque in prosthetic knee is important to
provide a normal gait throughout the gait cycle. Moreover, in stance phase of gait cycle,
applied torque is critical not only to provide normal gait trajectory but also to secure the
amputee from risk of falling.
There are several existing models of stance phase done in musculoskeletal model
[7]. While this model is clinically accepted, the parameter involved in the model is very
high. EMG based modelling to estimate joint torque has also been investigated [8,9].
Modelling approach investigated in [8] used joint kinematics and EMGs data as the input
to the model. While the modelling method can estimate inverse dynamic joint moments
better than the previous model, the model needed to be calibrated to each subject based
on each person physiological parameters. Kwon et al. investigated EMGs and moment
Ref. code: 25615822043435QIF
8
relationship in cerebral palsy subjects using four estimation models based on the
contribution of different muscles. The experimental design on this research was using a
dynamometer and perform an isokinetic test with EMGs attached on the muscles [9].
There is also an investigation in determining ankle impedance. Ankle impedance as
investigated in [10] is used as the model to design a powered ankle prosthesis.
Swing phase control as proposed by Ekkachai and Nilkhamhang in [11] are used
as the comparison in this thesis. The control structure consists of a neural network
predictive control with particle swarm optimization, and also a non-parametric feed
forward neural network swing phase model. It utilizes knee angle data and voltage
command as the input to the controller. The performance of this controller is measured
by normalized root mean squared error with validated data come from an experiment.
The result show that this controller perform better than the user-adaptive control found
in [2].
Ref. code: 25615822043435QIF
9
Chapter 3
Heuristic data based approach: Study case in sit to stand movement
3.1 Sit to Stand Movement
Sit to stand (STS) movement is a general task that frequently performed in daily
activities. This movement is a coordinated movement to raise the body center of mass
with applying a significant torque at the knee joint and other related jo int. Performing
STS movement is also the prerequisite task to do other movement such as walking from
sitting position. There are some cases where people loss or decline this function such
as in elderly people [12,15], disability diseases such as Parkinson’s disease [13,16], and
in the amputee [14].
There are many techniques to analyze STS activity ranging from fuzzy
clustering techniques, time series analysis, and Gaussian mixture model as described in
Chapter 2. In this chapter, a heuristic rule based technique is developed to detect the
phase transition in STS movement using less parameters consists of knee angle and
ground reaction force (GRF) data. Design of this experiment is a single subject
experimental design performing STS movement. Data of STS movement were recorded
using a motion capture system and force plate. The obtained data were used to develop
a rule based algorithm to detect the STS transition.
Unlike any other existing method that relying on image processing technique or
in time series analysis using the force sensor data in the chair sitting, this chapter
focused on how to detect STS movement using less data, based on heuristic rule based
algorithm developed during the experiment for the purpose of prosthetic knee, as
performing basic STS movement may become a challenge to the amputee because
sometimes the prosthetic device can not support them performing this movement.
3.2 Experimental System
Experiment was conducted in a gait laboratory equipped with motion capture
system (eight cameras VICON MX-T series) sampled at 100 Hz and also three force
platform (ANTI force plate) sampled at 1000 Hz. For this STS experiment, only two
force platform were used. STS experiment done with a single able-bodied subject, male,
Ref. code: 25615822043435QIF
10
age 23 years, weight 69 kg, and height 172 cm at the time the experiment was taken.
The subject had no history of severe lower limb trauma.
A plug in full body gait model were used in this experiment [17]. Thirty eight
reflective passive markers with diameter of 14 mm were positioned on the body in
several position that can be classified to four categories, i.e. head markers, torso
markers, upper limb markers, and lower limb markers.
Head markers consists of symmetrical left-right front head markers placed on
the temple and back head markers placed roughly in a horizontal plane of the front head
markers. Torso markers were placed in the spinous process of C7 and T10 along with
asymmetry right back markers in the right scapula on the back of the subject’s body,
with clavicle and sternum markers on the front. Upper limb markers were placed
symmetrically on the acromioclavicular joint in the shoulder, between the shoulder and
elbow markers in the upper arm, elbow joint, fore arm, thumb side and pinkie side of
the wrist, and on the fingers just below the head of the second metacarpal. Lower limb
markers were also placed symmetrically on the pelvis, leg, and foot following the
standard clinical gait markers [17].
Figure 3.1 Experimental set up in a gait laboratory utilizing eight sets of Vicon MX-T
series and two ANTI force plates.
Ref. code: 25615822043435QIF
11
The subject were asked to sit in an armless chair and perform STS movement at
self-selected pace with arms positioned on the distal end of thigh segment. Subject were
given a ‘start’ verbal cue from the operator to do standing action and will stop after five
seconds from the cue. This procedure was repeated ten times. Experimental design is
shown in Figure 3.1. with coordinate system following this agreement, x-axis is the
medio-lateral (ML) direction, y-axis is the antero-posterior (AP) direction, and z-axis
is the vertical direction.
3.2.1 Data Collection and Processing
Data obtained by the experimental system were exported to Microsoft Excel and
manually selected to be processed in MATLAB. The difference in sampling rate
between the motion capture system and force platform were synchronized by performing
down sampling to 100Hz on the force platform data. The overall flowchart of data
processing is shown in Figure 3.2.
Figure 3.2 Flowchart of data processing used in this experiment.
All the data were filtered through the fourth order Butterworth filter at 5 Hz using
finite impulse response (FIR) technique to achieve a zero-phase lag signal. The result of
applied filter technique is shown in Figure 3.3.
Ref. code: 25615822043435QIF
12
Figure 3.3 Applied FIR 4th order low-pass Butterworth filter to the GRF data.
In this experiment we use two data that consist of knee angle and GRF to develop
the detection technique in STS movement. As opposed to other existing method in [5]
that use the GRF at the chair, here we use GRF at the foot of the subject to comply with
our objective for the prosthetic application. Two components of GRF, i.e.
anteroposterior GRF (GRF-AP) and vertical GRF (GRF-Ver) are used in forming the
rule based algorithm.
3.3 Rule Based Intent Recognizer
Event detection is classified by three events, start of STS event (E1), seat-off
event (E2), and end of STS event (E3). We determine the events by observing
parameters obtained from experiment starting from the given verbal cue. Based on the
performed experiment, we found that at least two parameters must be used with
multiple fold of validation to prevent the false detection. The detection events of this
STS movement are governed by the following rules:
E1 was defined as the point where the first differential of GRF-AP over time is
negative and the first differential of knee angle is also negative over a specified
time.
E2 was defined as the point where the GRF-Ver has overcome the body weight
and the first differential of the GRF-Ver maintained at positive trend over a
specified time.
E3 was defined as the point where the knee angle is extended below 20 degrees,
with the second differential of knee angle is approximately equal to zero, and
the GRF-Ver is within two percent of body weight over a specified time.
Ref. code: 25615822043435QIF
13
GRF-AP is used in determining the E1 as it was observed that there is a forward
– backward force in ten performed trial on the subject’s foot during the initiation of
standing after the verbal cue. When the subject intent to stand from the sitting position,
it was observed that the body sway forward and thus creating a backward force (force
on foot is acting on posterior direction). The forward body sway can be observed by the
trajectory of clavicle (CLAV) and sternum (STRN) markers as depicted in Figure 3.4.
Figure 3.4 Initiation of forward body sway represented by CLAV and STRN markers
in torso with its effect to GRF-AP on subject’s foot.
In the performed experiment, AP-coordinate is calibrated with the front leg of
the chair as the reference point. It can be seen from Figure 3.4. that initiation of forward
body sway is shown after the CLAV marker crossed the reference point and at the exact
time the GRF-AP shows a descending trend. Based on this observation, we used GRF-
AP as the rule to detect E1.
GRF-Ver is used in determining the E2 as when in sitting position it was
observed only around 20% of body weight is supported by the foot and the rest is
supported by the chair shown by the center of mass (CoM) in sitting position is within
the chair area. When the transition from sitting to standing position occur, the CoM is
moving to the anterior direction and is going to fall under the support of the foot at full
Ref. code: 25615822043435QIF
14
standing position while body weight is transferred from the chair to the foot. The CoM
trajectory resulted from the body weight transfer is depicted in Figure 3.5.
Figure 3.5 Body weight transfer from sit to stand position represented by CoM
coordinate that fall between the LTOE and LANK position.
It can be seen from Figure 3.5. that when GRF-Ver has crossed the body weight
line of the subject, CoM trajectory has completely leave the chair support polygon and
begin to fall in between the support polygon of the foot of the subject shown by the left
toe (LTOE) and left ankle (LANK) markers. Based on this observation, we used GRF-
Ver as the rule to detect E2.
Knee angle data is coupled with GRF-AP and GRF-Ver to confirm the intention
of standing, shown by the continuous extension of the knee until reaching a fully
extension position. The specified time on this experiment was set at five times of the
sampling time (10 ms), thus resulted in 50 ms. This five times validation rule is set as
a precaution to better recognize the true intent of movement. The rule based method of
detection events is depicted in Table 3.1.
Ref. code: 25615822043435QIF
15
Table 3.1 Rule based of detection events in STS movement
GRF-AP GRF-Ver Knee Angle
Start (E1) 𝑑𝐺𝑅𝐹𝐴𝑃
𝑑𝑡< 0
𝑑𝜃𝐾
𝑑𝑡< 0
Seat-off (E2)
𝐺𝑅𝐹𝑉 > 𝐵𝑊
𝑑𝐺𝑅𝐹𝑉
𝑑𝑡< 0
𝑑𝜃𝐾
𝑑𝑡< 0
End (E3) 𝐺𝑅𝐹𝑉 = 𝐵𝑊 ± 2% 𝐵𝑊
𝜃𝐾 < 20°
𝑑2𝜃𝐾
𝑑𝑡2≅ 0
3.4 Results and Analysis
Data from the ten performed STS trial were processed according to chapter
3.2.1., with additional step that each of the dataset is limited only 500 data points or a
total of five seconds trial time that contains the data of STS movement. Using knee
angle, GRF-AP, and GRF-Ver data governed by the rule based algorithm presented in
chapter 3.3., the result of event detection and overall STS time from the ten performed
trial is shown in Table 3.2.
In Table 3.2., T is an abbreviation of Trial. In the ten performed trial, the mean
and standard deviation for STS time are 1.34 s and 0.29 s respectively. The fastest
recorded STS time in this trial is 1.04 s at T7, while the slowest recorded STS time is
1.7 s at T4. At E1 event detection, GRF-AP were recorded with mean and standard
deviation of -0.29 N and 7.87 N respectively. Minus sign in GRF-AP means that the
force was acting in the posterior direction, giving a reaction force due to sudden change
of forward progression of our body, while positive sign means that the force is acting
in anterior direction to propel the body forward. Detection of this E1 event would be
better when the GRF-AP is still acting in posterior direction as this means that it can
recognize the sudden change. Using the rule based method, six out of ten trial were
detected with GRF-AP act in posterior direction. One of the reason why the rule based
sometimes failed to detect E1 event at negative value of GRF-AP is because the STS
movement were happen fast enough that the five-fold validation missed the event
detection as seen in T1 – T3 compared to T4 – T6 in Table 3.2.
Ref. code: 25615822043435QIF
16
Table 3.2 STS time with event detection and quantitative analysis of related parameter on ten performed trial
T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 mean (st.dev)
E1
Time (s) 1.90 1.63 1.69 2.07 2.35 1.94 2.12 1.61 1.56 3.12 -
Knee angle (degree) 83.44 86.35 81.74 78.56 80.42 78.98 79.07 77.9 77.92 76.85 80.12 (2.94)
GRF-AP (N) 1.63 17.4 0.35 -6.56 -11.1 -2.38 6.96 -4.87 -1.94 -2.41 -0.29 (7.87)
GRF-V (N) 119.80 230.58 126.51 101.15 98.8 122.2 211.67 120.85 121.63 100.56 135.37 (46.54)
E2
Time (s) 2.57 2.34 2.40 2.74 2.79 2.59 2.73 2.28 2.23 3.78 -
Knee angle (degree) 27.16 39.04 34.62 40.81 56.37 29.16 33.98 28.5 47.77 23.1 36.05 (10.21)
GRF-AP (N) 55.82 38.65 53.5 53.9 44.21 50.72 48.65 67.63 62.64 57.17 53.28 (8.43)
GRF-V (N) 814.49 788.41 788.05 794.14 800.57 811.65 829.05 843.2 800.34 824.33 809.42 (18.47)
E3
Time (s) 2.97 2.71 2.78 3.77 3.91 3.53 3.16 3.20 2.61 4.78 -
Knee angle (degree) -5.05 -4.2 -5.95 -4.56 -6.03 -5.12 -6.92 -0.11 -0.35 -6.26 -4.45 (2.37)
GRF-AP (N) 27.57 0.18 8.32 -20.12 -2.15 -12.69 -11.9 -8.66 8.76 14.97 0.42 (14.58)
GRF-V (N) 669.3 666.87 677.14 690.21 682.43 678.79 665.19 682.25 665.29 669.02 674.64 (8.70)
STS
Time Overall time (s) 1.07 1.08 1.09 1.70 1.56 1.59 1.04 1.59 1.05 1.66 1.34 (0.29)
Ref. code: 25615822043435QIF
17
Detection knee angle were recorded with mean and standard deviation of
80.12° and 2.94° respectively (range 76.85° – 86.35) at E1 event, 36.05° and 10.21°
respectively (range 23.1° – 56.37° ) at E2 event, -4.45° and 2.37° respectively (range -
6.92° – -0.11°) at E3 event. The 20° threshold of knee angle that was set a priori in the
rule based algorithm to detect E3 event is found to be in agreement with the result
presented, as this value is below the range of the E2.
Peak force of GRF-Ver is approximated very close to E2 event as this event
marks the full transfer of our body weight from the chair. It can be seen in Table II, the
recorded mean and standard deviation of GRF-Ver at E2 event are 809.42 N and 18.47
N respectively. In E3 event detection, mean and standard deviation were recorded
674.64 N and 8.7 N respectively, and the body weight of the subject measured
approximately at 676.65 N is within this range. The plot example of the T1 dataset
along with the detection events is depicted in Figure 3.6.
Figure 3.6 Dataset T1 used in this study consist of θK (top), GRF-AP (middle), and
GRF-Ver (bottom) with the phase transition event E1-E3.
Based on the event detection in Figure 3.6., STS movement can be divided into
four phase. Phase I is a quiet sitting phase where the subject is sitting uninterruptedly
and ended when the sensors detect E1 event. Phase II, the forward acceleration phase,
is the start of trunk flexion and knee extension which marked from the E1 event until
Ref. code: 25615822043435QIF
18
the seat-off (E2) event. Phase III, the upward acceleration phase, is the changing from
trunk flexion into trunk extension which marked from E2 event until E3 event. Phase
IV is the quiet standing phase where the knee is fully extent to a stance position and the
observed GRF-Ver is stable within two percent of the body weight of the subject.
Figure 3.7 Time detection comparison between the presented rule based method and
true seat off measurement from sit force plate.
It should be noted that there is a difference between seat off (E2) using the rule
based method and the true seat-off event measured from the sit force plate. It can be
seen at Figure 3.7., that in the ten performed STS trial, the rule based method detect
maximum one data point faster or 10 ms faster than the true seat-off measured by sit
force plate. It is found that three out of ten trial have the same seat-off time.
Ref. code: 25615822043435QIF
19
Chapter 4
Parametric and Non Parametric Approach : Study Case in Knee
Torque Estimation
4.1 Dynamics Modelling
Parametric approach employs a set of mathematical relation derived from the
model being evaluated. In this section, parametric approach is discussed by deriving
sets of mathematical relation from the dynamics model presented. Parametric approach
is evaluated by two different dynamic calculation approach, i.e. Newtonian calculation
approach and Lagrangian calculation approach. The simplest model presented is two
degree of freedom (DOF) located at knee and ankle joint evaluated with Newtonian
calculation approach. A higher DOF model is also presented and evaluated with
Lagrangian calculation approach, consist of four DOF located at hip, knee, ankle, and
foot.
4.1.1 Two Degree of Freedom Newtonian Calculation Approach
The simple model evaluated in this study is the two DOF Dynamics with
Newtonian calculation approach model consist of two rigid body, i.e. the shank and
foot. The free body diagram is depicted in Figure 4.1. The focus of the model is to
generate knee torque derived of Newton’s second law as shown in Equation 4.1.
- - -
- - -
- - -
- - -
( - )
( ) ( ) -
( - ) - (
-
)
-( )( -
-
-
)
K f f s s y y z z
ks y f f y s s y y
ks z f f z s s z z
as z af z z f f z
as y af y fy f y
M I I F d F d
d m a m a F
d m a m a F
d d F m a
d d F m a
g g
g
(4.1)
Ref. code: 25615822043435QIF
20
Figure 4.1 Free body diagram of two DOF model
where 𝑀𝐾 𝐼, and 𝐹 are the knee torque, moment of inertia at the center of mass, and
ground reaction force respectively. α, 𝑎, 𝑚, 𝑔 and 𝑑 are the angular acceleration at
center of mass, translational acceleration at the center of mass, mass of a segment,
gravitational acceleration, and distance from joints to the center of mass respectively.
Subscripts 𝑓, 𝑠, 𝑎, 𝑘, −𝑦, −𝑧 denote the foot segment, shank segment, ankle joint, knee
joint, antero-posterior direction, and vertical direction respectively.
4.1.2 Four Degree of Freedom Lagrangian Calculation Approach
This dynamics approach is used to derived a model with more complexity. The
stance phase model is a 4 links, 3 joints, also will be constrained to 4 DOF mechanism.
The diagram of 4 DOF model is presented in Figure 4.2. along with the generalized
parameter needed to simulate the model.
Ref. code: 25615822043435QIF
21
Figure 4.2 Four DOF model evaluated with Lagrangian approach
L T V (4.2)
i
i i
L LQ
t q q
(4.3)
The governing equation of Lagrangian dynamics are shown in Equation 4.2 to
4.3, where 𝐿 is the Lagrangian function defined by the difference between kinetic
energy, 𝑇, and potential energy, 𝑉, of the system. Equation 4.3 is used for the derivation
of equations of motion in each segment, where 𝑞𝑖 is the generalized coordinates and
𝑄𝑖 is the generalized forces acting on the system. 𝑇 and 𝑉 of the system in Figure 4.2.
are defined by Equation 4.4 to 4.5, where 𝑚, 𝑣, 𝐼, 𝑤, ℎ are the mass, linear velocity,
moment of inertia, angular velocity, and position of the segment in 𝑧-axis relative to
the ground, respectively. 𝑁 and 𝑔 are the total number of segments and gravitational
acceleration respectively.
2 2
1
1
2
N
i i i i
i
T m v I
(4.4)
1
N
i i
i
V m gh
(4.5)
Ref. code: 25615822043435QIF
22
i
i i i
T T VQ
t q q q
(4.6)
Since in mechanical system, 𝑉 is not a function of 𝑞�̇̇� and 𝑡, thus by substituting
Equation 4.2 to Equation 4.3, Equation 4.6 is obtained. In this case 𝑞𝑖 is defined by the
segment angle, 𝜃𝑖, while 𝑄𝑖 is defined by the acting moment on each segment. Equation
of motions now can be derived by substituting Equation 4.4 and Equation 4.5 to
Equation 4.6. Equation 4.7 is the simple matrix form of overall equation of motions
[18], where 𝑴 is the inertial matrix, �̈� is the generalized coordinate vectors which in
this case are the segment angles, 𝑪 is the vector of Coriolis and centrifugal forces, 𝑮 is
the gravitational forces vector, and 𝑸 is the generalized forces vector. The expansion
of Equation 4.7 is shown in Equation 4.8 with the respected 𝑸 in each segment. Variable
𝑎𝑖𝑗 , 𝑐𝑖𝑗 , and 𝑔𝑖 are explained further in Appendix A.
2M C G Q (4.7)
11 12 13 14 1 11
21 22 23 24 2 22
31 32 33 34 3 33
41 42 43 44 4 44
A GRF
K A
H K
H
a a a a c g M M
a a a a c g M M
a a a a c g M M
a a a a c g M
(4.8)
4.2 Neural Network Based Modelling
Non parametric approach modelled in this section is in the form of neural
network. Neural network is constructed following the feed-forward neural network
(FNN) structure with one hidden layer. Data used to model the network were obtained
from the gait experiment done in [19], which collect a multiple task of gait experiment
from twenty subjects with mean age of 43.1 ± 15.4 years, body mass 68.5 ± 15.8 kg,
and body height 1.71 ± 0.1 m.
Ref. code: 25615822043435QIF
23
4.2.1 Model Design and Data Preparation
The dataset is available in µ and µ±σ level and classified into 5 groups from the
walking speed (very slow, slow, natural, fast, very fast). In this study only dataset from
adult group is used in all µ±σ level of every walking speed. The dataset is prepared into
a p x q matrices where p is the number of input variable and q is the data points. The
available data points was in the unit of percent (%) gait cycle from 0% to 100% with
1% increment, thus for every variable there are 101 data points. This study presents a
double stage delayed input, thus only 98 data points is used for every input variable.
The prepared dataset is then randomized following the random permutation technique.
A half of this randomized dataset is saved and used as training dataset.
The system design is a double stage delayed system where input at time t, t-1,
and t-2 will be used to predict output at time t+1. A total of maximum 6 FNN Input
consist of hip angle, θH, knee angle, θK, ankle angle, θA, and GRF in antero-posterior
direction (GRFx), medio-lateral direction (GRFy), and vertical direction (GRFz) is
combined into 6 different cases as shown in Table 4.1.
Table 4.1 Case model presented with input variation
Case FNN Input
θH θK θA GRFx GRFy GRFz
1 v v v v v v
2 v v v v v
3 v v v v
4 v v v
5 v v
6 v v
*v=data used
Case 1 consider all the parameters needed in conventional knee torque
calculation and in this study act as basis of comparison in FNN model. In later cases,
we omitted θH based on the objective of this study, to find model candidate to design
sensors needed in prosthetic knee, as acquiring θH would be inconvenient for the
amputee because it would require a sensor device to be attached near the residual limb.
In case 3 we try to model the knee torque estimation without the availability of θA data,
Ref. code: 25615822043435QIF
24
while for Case 4 and 5 we try to only use vertical GRF data. We also try to model
without any kinetics data in Case 6 with only two kinematics data θK and θA.
4.2.2 Training Algorithm and Strategy
FNN is constructed as shown in Figure 4.3. Training dataset is randomly divided
into training data, validation data, and testing data with a 70:15:15 ratio. The FNN
consists of one hidden layer with an optimized number of hidden nodes, nopt. The
method to determine nopt is done by varying the number of nodes from 2 to 30 with an
increment of 2 nodes [11]. Every number of nodes is run 10 times with different initial
weight and random data division. The performance of each node is evaluated in the
form of MSE. The average MSE from 10 iterations on each simulated nodes is set to be
the criterion to choose nopt.
Figure 4.3 Structure of FNN used to estimate knee torque
The training algorithm is following the Levenberg-Marquardt backpropagation
algorithm, with hyperbolic tangent sigmoid transfer function as the activation function
in the hidden layer and linear transfer function in the output layer. Training run with a
predetermined MSE performance goal of 10-8. Training also constrained to maximum
of 3000 iterations and 10 times validation checks, which mean the training will stop if
Ref. code: 25615822043435QIF
25
and only if it reach the performance goal, or it reach 3000 iteration, or if within 10
iterations the performance is not improving significantly.
4.3 Simulation and Analysis
The computation in this study is performed using Intel® Core™ i7 3.5 GHz
processor with 8 GB RAM. Simulation result of the presented model is discussed in
this section. The first step in FNN simulation is to determine the nopt parameter. The
effect of hidden nodes size to MSE in each presented case is presented in Figure 4.4.
Figure 4.4 The effect of hidden nodes size to MSE on each presented cases.
Average MSE and the corresponding selected number of hidden nodes are
shown in Table. 4.2. There is a 0.003 or 2.85% relative average MSE difference between
Case 1 and Case 2, thus it can be concluded that θH is relatively insignificant in
predicting knee torque. There is also 0.032 relative average MSE difference between
Case 3 and Case 4, or in other words Case 4 perform 21.62% better in term of average
MSE than Case 3. Thus it can be concluded that kinematics data of θA is more important
than kinetics data of GRFx and GRFz in predicting knee torque.
Ref. code: 25615822043435QIF
26
Table 4.2. Selected optimum size of hidden nodes and its corresponding training MSE
for every presented case.
Case 1 Case 2 Case 3 Case 4 Case 5 Case 6
nopt 22 28 20 28 18 24
MSE 0.105 0.108 0.148 0.116 0.206 0.241
Natural walking at mean level is used as test data to evaluate the trained network
in Figure 4.5. Normalized root mean squared error (NRMSE) is used as the performance
parameter as defined in Equation 4.9, where n is the total data points, τk(val) is the knee
torque validation, and τk(pre) is the knee torque FNN prediction. It is shown in Figure
4.5. that the performance in term of NRMSE is relatively decreasing around 30% from
Case 1 to Case 4, and another 30% Case 4 to Case 6. The NRMSE from training FNN
of all cases with all walking speed is presented in Table 4.3.
2
( ) ( )1
( )max ( )min
1 n
k val k pret
k val k pre
nNRMSE
(4.9)
Table 4.3. NRMSE achieved by each case in every walking speed test data
Case
Walking Speed 𝑵𝑹𝑴𝑺𝑬̅̅ ̅̅ ̅̅ ̅̅ ̅̅ ̅
(%) Very Slow Slow Natural Medium Fast
1 0.0346b 0.0189a,b 0.0205b 0.0218 0.0240b 2.39
2 0.0443 0.0219 0.0213a 0.0213a,b 0.0338 2.37
3 0.0509 0.0227a 0.0238 0.0313 0.0333 2.69
4 0.0417 0.0276 0.0311 0.0250a 0.0350 2.67
5 0.0840 0.0386 0.0347a 0.0393 0.0479 4.07
6 0.0673 0.029a 0.0318 0.0447 0.0476 3.67
a = lowest NRMSE for each case
b = lowest NRMSE for each walking speed
Ref. code: 25615822043435QIF
27
Overall performance of each case is defined by 𝑁𝑅𝑀𝑆𝐸̅̅ ̅̅ ̅̅ ̅̅ ̅̅ parameter as shown in
Table. 4.3. It is shown that there is only 0.02% difference in Case 1 and Case 2, thus it
can be concluded that hip angle data is insignificant to estimate the knee torque.
Comparing from Case 6 and Case 4, we can conclude that adding one more input
variable will lead to 27.24% increasing of performance.
It can be observed that in Figure 4.5, Case 6 perform with maximum error at the
phase transition toe-off event. This event is closely related to the dynamics impact in
push-off stage of our foot. From this result, we can conclude the importance of GRF
data, more over in critical event such as toe-off and heel strike in estimating knee
torque.
(a) Case 1 (b) Case 2
(c) Case 3 (d) Case 4
Figure 4.5 Comparison between FNN prediction and validation data with absolute error
for (a) Case 1, (b) Case 2, (c) Case 4 and (d) Case 6. Vertical dash line indicate toe off
event and mark the phase transition.
Ref. code: 25615822043435QIF
28
Knee torque calculation typically requires an anthropometric measurement of
the body, kinematics data that can be captured by a 3D motion system and also kinetics
data that can be calculated by kinematics data and GRF data from force plate. Based on
these data, knee torque is calculated using inverse dynamics starting from the foot
segment to calculate the ankle torque.
This study has discussed the parametric approach to calculate the knee torque
using two dynamics modelling approach that are two DOF Dynamics with Newtonian
calculation approach and a more complex four DOF dynamics with Lagrangian
calculation approach as an example on how the 3D motion capture system calculate
other kinetics data particularly joints torque data. Both of the dynamics based approach
require a number of parameters including several estimated anthropometric
measurement of the investigated lower limb of the subject. In this matter, the Lagrange
method require less estimated anthropometric measurement than the Newton method,
thus the error caused by estimated parameters will be lower in Lagrange method. A full
comparison is explained further in Table 4.4.
It can be concluded from Table 4.4 that estimation method based on FNN have
the advantage of using only sensor based measurement, while both of Newton and
Lagrange methods require a number of anthropometric measurement which is different
for each person. Both Newton and Lagrange method are solved by using numerical –
iterative approach. While numerical – iterative can be time consuming, it will have no
problem to estimate the knee torque in any walking speed as long as we have the
complete data from sensor based measurement input and the constants from
anthropometric measurement. On the other hand, using FNN require a training that has
to be carefully planned to cover from the slowest speed the subject usually walk until
the fastest speed to make sure the FNN can estimate the knee torque needed.
This study has also investigated knee torque estimation method by only using
several combined kinematics and kinetics data. A total of six different combination of
inputs were evaluated using FNN. The result shows that hip angle data, θH, is
insignificant to estimate the knee torque showed by the 𝑁𝑅𝑀𝑆𝐸̅̅ ̅̅ ̅̅ ̅̅ ̅̅ parameter where it is
slightly higher on case 1. This also shows that FNN modelling agree with dynamics
modelling where there any of the hip parameter is not needed to calculate knee torque.
Ref. code: 25615822043435QIF
29
This study also shows that ankle angle data can replace the antero-posterior and medio-
lateral GRF data and still perform slightly better in term of 𝑁𝑅𝑀𝑆𝐸̅̅ ̅̅ ̅̅ ̅̅ ̅̅ parameter.
Table 4.4 Comparison between methods presented in estimating knee torque
Parameters
required
Newton Lagrange FNN
Anthropometric
measurement
(Estimated using
ratio [20] )
Moment of inertia (I) in
each segment
Moment of inertia (I)
in each segment
Not
required
Distance from distal end of
a segment to its Centre of
Mass (COM) (d)
Distance from distal
end of a segment to
its COM (d)
Distance from proximal
end of a segment to its
COM (d)
Segment length (l)
Segment mass (m) Segment mass (m)
Kinematics and
Kinetic
measurement
(Sensor based)
Rotational acceleration (α)
of each segment Segment angle (θ)
Relative
angle
(θK, θA)
GRF (F) GRF GRFz
Linear acceleration in x-
axis and y-axis (ax, ay) of
each segment
-
Knee
torque
(τK)
*for
training
Method of
Estimating Knee
Torque
Numerical / Iterative Numerical / Iterative Need
Training
Ref. code: 25615822043435QIF
30
Chapter 5
Machine learning approach : Reinforcement learning control in
swing phase
This chapter explore a machine learning approach as control structure in a
prosthetic device. Reinforcement q-learning control is evaluated as control strategy in
semi-active prosthetic knee device using magnetorheological (MR) damper as damping
mechanism. Control strategy discussed in this chapter is going to be limited to a specific
phase in walking, swing phase.
There are three sections of this chapter: system and environment model, q-
learning control, and simulation and results. The first section discussed a brief
description of MR damper as main system to be controlled in a simulated environment
modelled by double pendulum. In the second section, the underlying principle and
mathematical description of q-learning is discussed along with the proposed user design
reward and its effect to related parameters. The third section discussed the simulation
and results of the proposed system and control design.
5.1 System and Environment Model
System in this section defined as main actuator to be controlled, i.e. MR damper.
Meanwhile, environment is defined as the application where the system will be used,
in this case is the double pendulum model used as the simulated environment to perform
swing phase on a gait cycle. The following are the brief descriptions on the system and
environment used in this study.
5.1.1 System Description
MR damper is a semi-active device that generate passive force in the form of
damping. It is controlled by applying voltage to generate necessary damping in
prosthetic knee with aim to follow the trajectory of knee angle. In this study, MR
damper is modelled following the elementary hysteresis model (EHM)-based
feedforward neural network (FNN) model as described in [21]. The model consists of
two FNN, where one FNN coupled with EHM acted as hysteresis model and the output
Ref. code: 25615822043435QIF
31
of this network is fed to another FNN that acted as gain function [21]. The structure of
this model is depicted in Figure 5.1.
Figure 5.1 Structure of MR damper model [21]
MR damper system is then controlled using the structure depicted in Figure 5.2.
Voltage controller is introduced to calculate a suitable voltage input signal that will
produce the desired damping force. Using this control structure is proved to have
several advantages such as lower energy consumption, excellent accuracy, fast response
time, and does not need force feedback to be implemented.
Figure 5.2 Controller structure of MR damper [21]
The MR damper used is a cylinder type and is attached at a distance MRd away
from the knee joint. Based on this distance, torque generated at knee joint by the MR
damper can be calculated using Equation 5.1.
cosK MR KM d F (5.1)
Ref. code: 25615822043435QIF
32
5.1.2 Environment Model
Double pendulum model is proposed as the environment model for swing phase.
The model consists of two links, i.e. thigh and a lumped shank and foot segment as
depicted in Figure 5.3. There are two actuated joints with a total of four degrees of
freedom, where hip joint have one rotational degree of freedom on the z axis and two
translational degrees of freedom on x and y axis, and knee joint have one rotational
degree of freedom on the z axis.
Figure 5.3. Double pendulum as environment model to simulate swing phase
This model is then simulated in MATLAB Sim Mechanics environment.
Torque generated by each joint are governed by Equation 5.2 and 5.3, where KM and
HM are the torques at knee and hip respectively. , , ,m I d L are segment mass, moment
of inertia at segment’s center of mass, length measured from the proximal end of the
segment to the center of mass, and segment length respectively. Subscripts L and T
denote the leg segment and thigh segment, while hxa and hya are the linear acceleration
at hip joint along the x and y axes. , , , g are the angle, angular velocity, angular
acceleration, and gravitational constant at 9.8 m/s2 respectively. Knee angle, K , can
be calculated by K T L .
Ref. code: 25615822043435QIF
33
2( cos( ) sin( )
cos ( )sin )
K L L L L T T L T T T L T
hx L hy L
M I m d L L
a a g
(5.2)
2 2( ) ( cos( ) sin( ))
( )( cos ( )sin )
H K L T T T L L T L L T L L T
L T T T hx T hy T
M M m L I m d L
m L m d a a g
(5.3)
5.1.3 Data Used
Gait data used in this chapter are also normal gait data collected from [11] for
convinience in comparison study of the controller. A male subject with 83 kg of weight
and 1.75 m tall at the time of the experiment, were asked to walk on a treadmill at
various speed of 2.4, 3.0, 3.6, 4.2, 4.8, and 5.4 km/h. Markers were placed at hip, knee,
and ankle joints. A high speed camera were used to capture the joints coordinate and
later converted to relative joint angle. In this chapter, since only control in swing phase
is discussed, then the gait data used will be constrained into swing phase only. Knee
angle data at swing phase with various speed are depicted in Figure 5.4.
Figure 5.4 Knee angle data used at swing phase with various speed [11]
Ref. code: 25615822043435QIF
34
5.2 Q-Learning Control
In this section, the proposed q-learning control is discussed. Q-learning belongs
to tabular reinforcement learning group in machine learning algorithm. In general,
reinforcement learning learns the control policies within a specified environment where
the performance and training information are provided in terms of whether the applied
control policies is a success or a failure. Success or failure in this case can be determined
by a certain performance index depending on the system and environment involved.
5.2.1 Q-Learning
The general structure of reinforcement learning is depicted in Figure 5.5. As
stated before, q-learning is one of machine learning algorithm in reinforcement learning
group. In Figure 5.5, an agent gives an action to the system and environment. Based on
the given action, the system will react to another state and also giving a reward based
on the performance index calculated from the current state. In this study, the agent will
be the q-function with mathematical description as shown in Equation 5.4.
Figure 5.5 General reinforcement learning structure
( , ) ( , ) ( , ) ( 1, ) ( , )maxt t t t t t t t t ts a s a s a s a s a
a
Q Q R Q Q
(5.4)
where Q and R are the Q function and reward function respectively. , , ,s a are the
state, action, learning rate, and discounted rate respectively, while subscript t denote
the time. Learning rate and discounted rate is a dimensionless variable between 0 and
1. Higher learning rate ( set closer to 1) means that the Q function will be updated
Ref. code: 25615822043435QIF
35
quickly per iteration, while if it set equal to 0 means that Q function will never be
updated. Discounted factor is a variable that will determine how the Q function acts
toward the reward. If it is set closer to 0 means it will only consider the instantaneous
reward (current reward), while if it is set closer to 1 means it will strive more into the
long term higher rewards.
In this study, q-learning is proposed to be used as a controller of a dynamics
system of an MR damper in prosthetic knee in a double pendulum simulated
environment. State is the parameter that can be extracted from the environment that
contains necessary information from the environment to be used to evaluate the control
policies. In most cases, q-function with multi state is used to better learn the
environment. In this study, knee angle ( K ), and the derivative of knee angle ( K ), are
used as the state, while the command voltage ( v ) is used as the action. Thus the update
rule of q-function can be written as Equation 5.5.
( , , ) ( , , ) ( 1, 1, ) ( , , )
maxk k k k k k k k
taa a a a
Q Q R Q Q
(5.5)
5.2.2 User designed reward function
The structure of reward mechanism in q-learning algorithm used in this study is
modified into a rationed multiple reward as a function of time. This structure enables
the learning process to give more reward to latter horizon events due to the response
time needed by MR damper to generate the necessary damping force. Mathematical
descriptions of this multiple reward mechanism are shown in Equation 5.5 to Equation
5.7.
( , , ) ( , , ) ( 1, 1, ) ( , , )1
maxk k k k k k k k
n
t taa a a a
t
Q Q R Q Q
(5.5)
2
t ct (5.6)
Ref. code: 25615822043435QIF
36
1
1n
t
t
(5.7)
where t are the specific designed ratio of reward priority, n is the number of
prediction horizon, and c is a constant that depends on n . In this study, n is set to 4,
thus 0.033c , to be conveniently compared to neural network predictive controller
(NNPC) algorithm studied in [11] that set the prediction horizon to 4. The reward
priority given at specified prediction horizon is an exponential function as depicted in
Figure 5.6.
Figure 5.6 t as an exponential function with n=4.
As the aim of the controller is to mimic the biological knee trajectory in swing
phase, reward will be given according to whether the prosthetic knee can follow the
biological knee trajectory. In this study, reward is designed as a function of a
performance index ( PI ). A simple absolute error, te , is chosen as perfomance index
and evaluated per interval time. Reward function is also designed to have a continuous
value over a specified boundary and following a decayed function. Mathematical
descriptions of the proposed novel reward function are depicted in Equation 5.8 to
Equation 5.10.
Ref. code: 25615822043435QIF
37
( )tR f PI (5.8)
( )
( )
K K val
t
K val
PI e
(5.9)
max
min
min
; 0
;
;
t
l t
E
t u
L E
t u t l
t l
R E L
R R L E L
R E L
(5.10)
where ( )K val is the validation of knee angle at time t, maxR and minR are the
maximum reward and minimum reward set to 1 and -1 respectively. tE is the percentage
of te , which can be written as 100t tE e . , ,u lL L are the reward constant set arbitrarily
to 0.01, performance limit to get the positive reward, and performance limit to get the
lowest reward. In this study, PI is aimed to be within 0.01 which means the error
should be under 1%, thus uL is set to be 1 and lL could be set to any number above uL
to give a variable negative reward. In this case lL is set to be twice the value of uL .
The graphical description of this reward design is depicted in Figure 5.7. To be noted,
, ,u lL L , minR , and maxR can be defined accordingly for other application depending on
the system being evaluated. The designed reward function is preferred to follow an
exponential function rather than linear function in order to better train the q-function to
reach the state with biggest reward value that can lead to faster convergence.
Ref. code: 25615822043435QIF
38
Figure 5.7. The proposed user designed reward function as a function of tE .
5.3 Simulation and Results
In this section, a simulation of swing phase control using the proposed controller
is discussed along with a comparison study. The simulation was computed using Intel®
Core™ i7 3.5 GHz processor with 8 GB RAM. The block diagram of the proposed q-
learning control is depicted in Figure 5.8., consisting of a block of q-function with input
of multi-state fed from the memory block, and updated by the reward function. Input
of the reward function are the actual knee angle ( )K t and the desired knee angle
_ ( )K desired t from experimental data.
Ref. code: 25615822043435QIF
39
Figure 5.8. Block diagram of the proposed q-learning control.
There are several parameters that need to be defined and optimized. First, as this
control approach is a tabular reinforcement learning using q-learning method, q-
function value are stored in a Q-matrix. The sizes of Q-matrix depends on the number
of state and action. In this simulation, the structure of Q-matrix is a 3 dimensional
matrix consists of l row of state K , m column of state K , and n layer of action v .
Q-matrix must cover all the state and action available on the system. Based on the data
used, state K is defined within the range of 0 and 70 degrees and step size of 0.5
degrees, thus resulting with 141 rows. State K set within the range of -7 to 7 degrees
per unit of time with 0.05 step size, thus resulting with 281 columns. Range of
command voltage are set from 0 to 5 volt with 0.1 step size, thus resulting with 51 layers
of action v .
Secondly, learning rate is need to be defined. In this simulation, several value
of learning rate are simulated to see its effect toward the number of iteration needed to
achieve best performance. The performance index used to evaluate this simulation is
normalized root mean squared error (NRMSE) as shown in Equation 5.11., where sn
is the number of samples in dataset.
Ref. code: 25615822043435QIF
40
2
_ ( ) ( )
1
_ (max) _ (min)
1 sn
K desired t K t
ts
K desired K desired
nNRMSE
(5.9)
Several value of learning rate 0.001 0.01 0.1 0.5 0.9 are picked a
priori to be simulated with a maximum 3000 iteration in a single speed simulation. The
effect of this learning rate to NRMSE is shown in Figure 5.9. It can be concluded that
lowest learning rate simulated with a constrained iteration of 3000 perform the worst
among the learning rate sample. It is also shown that higher learning rate does not
guarantee better performance, as inspected from simulation number 4 0.5 , with
% 1.6%NRMSE compared to simulation number 3 0.1 , with
% 1.04%NRMSE . Based on this finding, further simulation is going to use
0.1 ... 0.9 .
Figure 5.9. Effect of learning rate with constrained iteration to NRMSE
Ref. code: 25615822043435QIF
41
Figure 5.10. Effect of learning rate to convergence and NRMSE.
A further simulation using 0.1 0.3 0.5 0.7 0.9 was done as shown
in Figure 5.10 to see the effect of learning rate to the convergence of the performance.
It can be concluded that higher learning rate leads to faster convergence. In further
simulation 0.5 and 0.8 will be used.
There are many approach to train the q-function in this study. Training one q-
function for a specific case of one walking speed is the simplest one, while training
multispeed at once under one q-function is challenging. In this simulation, training
multispeed under one control policy is proposed. Slowest walking speed of 2.4 km/h,
fastest walking speed 5.4 km/h, and one walking speed between that range, 3.6 km/h,
are used to be trained.
In this simulation time interval is set to be 20 ms, thus the action or command
voltage to prosthetic knee will be updated every 20 ms. The dataset of 2.4, 5.4, and 3.6
km/h are chosen randomly for every iteration of the simulation. There are two
conditions for the simulation to stop. First is if all the NRMSE of all trained speed fall
under 1%, and second is if all the trained speed converge into one final value of NRMSE
Ref. code: 25615822043435QIF
42
for at least 10 further iterations. The training process of this simulation is depicted in
Figure 5.11.
Figure 5.11. Training process of multispeed under one control policy simulation.
It can be seen in Figure 5.11 that the fastest convergence is achieved by the
fastest walking speed, which converge at around 3300 iterations, followed by walking
speed of 3.6 km/h, which converge around 6700 iterations and the latest is the slowest
walking speed, which converge around 6900 iterations. This occurrence happens
because faster walking speed means lesser time in gait cycle in general, which means
lesser time in swing phase. Lesser time in swing phase with a fixed control interval of
20 ms means that the q-function will calculate less action than the slower walking
speed.
The proposed controller is then compared to the open loop user-adaptive
controller and NNPC found in [11]. The comparison of 2.4, 3.6, and 5.4 km/h walking
speed are depicted in Figure 5.12 and Table 5.1.
Ref. code: 25615822043435QIF
43
(a) 2.4 km/h
(b) 3.6 km/h
Ref. code: 25615822043435QIF
44
(c) 5.4 km/h
Figure 5.12 Comparison between the proposed q-learning control (black line),
user-adaptive control (green dashed line), and NNPC (red line).
Table 5.1. Comparison between user adaptive, NNPC, and q-learning control
Walking speed (km/h) NRMSE (%)
User-adaptive NNPC Q-learning
2.4 2.70 0.81 0.78
3.6 3.65 0.61 0.88
5.4 3.46 2.42 0.52
*green shade indicate best performance
It can be seen from Table 5.1 that the q-learning method perform within 1% of
NRMSE which following the designed reward function. There has not been a detailed
study about the acceptable criterion in term of NRMSE performance index of knee
trajectory in a prosthetic knee. Knee trajectory is only one of the parameters to be
optimized among other correlated system such as ankle and foot prostheses to achieve
Ref. code: 25615822043435QIF
45
better gait symmetry and reduce metabolic costs. In this study, it is aimed to mimic the
biological knee trajectory which shown by the NRMSE performance index.
Using the computational hardware stated in section 5.3 and source code
implemented in MATLAB, the overall calculation and online update q-function process
took approximately 40.4 ms, while each evaluation of NNPC with pre-trained swing
phase model took approximately 13.2 ms [11]. Changing the source code
implementation in C language and using a dedicated processing hardware could shorten
the calculation time to be within the proposed control interval of 20 ms as studied in
[11].
Ref. code: 25615822043435QIF
46
Chapter 6
Conclusions and Recommendations
6.1 Conclusions
The following conclusions are drawn based on the study conducted within this
thesis.
Heuristic experimental based approach
This study describes a rule based approach to detect the phase transition
in STS movement using less data. Two components of GRF data, i.e. GRF-AP
GRF-Ver, along with knee angle data were used in this study to construct the
algorithm. These data were obtained by a single subject experimental design
done in a gait laboratory utilizing motion capture systems and force plates.
On the ten performed trial, using the rule based method developed in this
study the mean and standard deviation of STS time are 1.34 s and 0.29 s
respectively. For E1 event detection, four out of ten trial were detected with
GRF-AP act in antero direction due to faster STS movement compared to other
trial. For E2 event detection, seven out of ten trial were detected 10 ms early
compared to the true seat-off measured from the sit force plate. The rule based
algorithm presented in this study provide a groundwork to develop an intent
recognizer system in a powered prosthetic device with a limited measureable
parameter for better control strategy assisting this movement.
Non-parametric approach in estimating knee torque
The proposed non-parametric approach in estimating knee torque
provide an alternative method as indirect measurement of knee torque. There
are six presented study case of input variation to the FNN in this study. The
investigation in this study provides a more simple method involving less
parameters in estimating knee torque that can be achieved by using only two
kinematics data that are knee angle and ankle angle with a vertical GRF data,
resulting in 2.67% of 𝑁𝑅𝑀𝑆𝐸̅̅ ̅̅ ̅̅ ̅̅ ̅̅ .
Ref. code: 25615822043435QIF
47
In the simulation, result shows that hip angle is considered insignificant
to estimate the knee torque as showed by the NRMSE parameter. It is also found
that ankle angle can replace the antero-posterior and medio-lateral GRF data
and can perform slightly better. This method can be used to design the minimal
sensors required, i.e. two sets of angle sensors to measure knee and ankle angle
with one set of force sensor on foot to measure the ground reaction force, for a
prosthetic knee with impedance based control in stance phase to mimic the
biological knee torque profile to support the amputee during the stance phase.
Reinforcement q-learning control on swing phase The proposed controller in this study is following the structure of a
tabular reinforcement learning algorithm in machine learning. Q-learning
control consists of a q-function that store its value in a q-matrix, and a reward
function following the novel user designed reward proposed in this study. The
advantages of using this control structure are it can be trained online and also it
is a model-free control algorithm that does not require prior knowledge of the
system to be controller.
A continuous user designer reward as a function of performance index
and following a decayed function is proposed as reward function in this study
that can lead to a better reward mechanism. This control structure also shows
adaptibility to various walking speed. Performance index shows that this control
structure perform better than the user-adaptive control. In some of the walking
speed, this control structure perform better than the NNPC.
6.2 Recommendations
Based on the study conducted in this thesis, there are several recommendations
for future works as the following:
In heuristic experimental based of intent recognizer, more experiments with
different subject is needed along with testing the rule based algorithm proposed
in this thesis in a multiple mixed movements.
A non-parametric approach conclude that it is possible to use less data that
only consists of knee angle, knee ankle and GRF, to estimate the knee torque
Ref. code: 25615822043435QIF
48
that can be used as impedance based control in prosthetic knee. In a perspective
of performance index, another structure of neural network could be proposed
to better predict the knee torque. In a perspective of practical issue, this model
could be examine further in a real system capturing the amputee gait cycle.
The advantage of reinforcement q-learning control is the model-free and
adaptive control that need no prior knowledge of the system. However the
training process per iteration per cycle using the computation system in this
thesis is still slow. Another training strategy could be explored further. On the
other hand, this thesis proposed a tabular-discretized q-function that stored in
a Q-matrix. A continuous q-function could be explored for further work to
better cover all the state and action.
Ref. code: 25615822043435QIF
49
References
1. Kalanovic, V. D., Popovic, D., & Skaug, N. T. (2000). Feedback error learning
neural network for trans-femoral prosthesis. IEEE Transactions on Rehabilitation
Engineering, 8(1), 71–80. https://doi.org/10.1109/86.830951
2. Herr, H., & Wilkenfeld, A. (2003). Feature User-adaptive control of a
magnetorheological prosthetic knee, 30(1), 42–55.
https://doi.org/10.1108/01439910310457706
3. Xie, H. L., Liang, Z. Z., Li, F., & Guo, L. X. (2010). The Knee Joint Design and
Control of Above-knee Intelligent Bionic Leg Based on Magneto-rheological Damper.
International Journal of Automation and Computing, 7(3), 277–282.
https://doi.org/10.1007/s11633-010-0503
4. Garcia, E., Arevalo, J. C., Muoz, G., & Gonzalez-De-Santos, P. (2011).
Combining series elastic actuation and magneto-rheological damping for the control of
agile locomotion. Robotics and Autonomous Systems, 59(10), 827–839.
https://doi.org/10.1016/j.robot.2011.06.006
5. Bulea, T. C., Kobetic, R., To, C. S., Audu, M. L., Schnellenberger, J. R., &
Triolo, R. J. (2012). A variable impedance knee mechanism for controlled stance
flexion during pathological gait. IEEE/ASME Transactions on Mechatronics, 17(5),
822–832. https://doi.org/10.1109/TMECH.2011.2131148
6. Wentink, E. C., Koopman, H. F. J. M., Stramigioli, S., Rietman, J. S., & Veltink,
P. H. (2013). Variable stiffness actuated prosthetic knee to restore knee buckling during
stance: A modeling study. Medical Engineering and Physics, 35(6), 838–845.
https://doi.org/10.1016/j.medengphy.2012.08.016
7. Dabiri, Y., Najarian, S., Eslami, M. R., Zahedi, S., & Moser, D. (2013). A
powered prosthetic knee joint inspired from musculoskeletal system. Biocybernetics
and Biomedical Engineering, 33(2), 118–124
Ref. code: 25615822043435QIF
50
8. Lloyd, D. G., & Besier, T. F. (2003). An EMG-driven musculoskeletal model
to estimate muscle forces and knee joint moments in vivo. Journal of Biomechanics,
36(6), 765–776.
9. Kwon, Suncheol, et al. "A practical strategy for sEMG-based knee joint moment
estimation during gait and its validation in individuals with cerebral palsy." IEEE
Transactions on Biomedical Engineering 59.5 (2012): 1480-1487.
10. Rouse, E. J., Hargrove, L. J., Perreault, E. J., & Kuiken, T. A. (2014). Estimation
of human ankle impedance during the stance phase of walking. IEEE Transactions on
Neural Systems and Rehabilitation Engineering, 22(4),870–878.
11. Ekkachai, K., & Nilkhamhang, I. (2016). Swing Phase Control of Semi-Active
Prosthetic Knee Using Neural Network Predictive Control With Particle Swarm
Optimization. IEEE Transactions on Neural Systems and Rehabilitation Engineering,
24(11), 1169–1178. https://doi.org/10.1109/TNSRE.2016.2521686
12. Banerjee, T., Keller, J., Skubic, M., & Abbott, C. 2010. Sit-To-Stand Detection
Using Fuzzy Clustering Techniques. Fuzzy Systems (FUZZ), 2010 IEEE International
Conference on, 1–8.
13. Zijlstra, A., Mancini, M., Lindemann, U., Chiari, L., & Zijlstra, W. 2012. Sit-
stand and stand-sit transitions in older adults and patients with Parkinson’s disease:
event detection based on motion sensors versus force plates. Journal of
NeuroEngineering and Rehabilitation, 9(1), 75.
14. Varol, H. A., Sup, F., & Goldfarb, M. 2009. Powered sit-to-stand and assistive
stand-to-sit framework for a powered transfemoral prosthesis. In 2009 IEEE
International Conference on Rehabilitation Robotics (pp. 645–651). IEEE.
15. Najafi, B., Aminian, K., Loew, F., Blanc, Y., & Robert, P. A. 2002.
Measurement of stand-sit and sit-stand transitions using a miniature gyroscope and its
application in fall risk evaluation in the elderly. IEEE Transactions on Biomedical
Engineering, 49(8), 843–851.
Ref. code: 25615822043435QIF
51
16. Mak, M. K. Y., Levin, O., Mizrahi, J., & Hui-Chan, C. W. Y. 2003. Joint torques
during sit-to-stand in healthy subjects and people with Parkinson’s disease. Clinical
Biomechanics, 18(3), 197–206.
17. VICON. Plug-in Gait guide. Retrieved June 7, 2017 from
https://www.vicon.com/downloads/documentation/plug-in-gait-product-guide
18. Acar, M., & Bayraktaroglu, Z. Y. (2011). Modeling and control of a one-legged
hopping mechanism. 2011 IEEE International Conference on Mechatronics, ICM 2011
- Proceedings, 839–844.
19. Bovi, G., Rabuffetti, M., Mazzoleni, P., & Ferrarin, M. (2011). A multiple-task
gait analysis approach: kinematic, kinetic and EMG reference data for healthy young
and adult subjects. Gait & Posture, 33(1), 6–13.
20. Winter, D. A. (2009). Biomechanics and motor control of human movement.
John Wiley & Sons.
21. Ekkachai, K., Tungpimolrut, K., & Nilkhamhang, I. (2013). Force control of a
magnetorheological damper using an elementary hysteresis model-based feedforward
neural network. Smart Materials and Structures, 22(11). https://doi.org/10.1088/0964-
1726/22/11/115030
Ref. code: 25615822043435QIF
52
Appendix
Ref. code: 25615822043435QIF
53
Appendix A
Variables of Lagrange Dynamics Approach
2 2
11 1 1 1 1 2 3 4
12 2 1 2 1 2 3 1 2 4 1 2
13 3 1 3 1 3 4 1 3
14 4 1 4 1 4
21 2 1 2 1 2 3 1 2 4 1 2
2 2
22 2 2 2 2 3 4
23 3 2 3 2 3 4 2 3
24
( )
sin( )
sin( )
sin( )
sin( )
( )
cos( )
a I d m l m m m
a d l m l l m l l m
a d l m l l m
a d l m
a d l m l l m l l m
a I d m l m m
a d l m l l m
a d
4 2 4 2 4
31 3 1 3 1 3 4 1 3
32 3 2 3 2 3 4 2 3
2 2
33 3 3 3 3 4
34 4 3 4 4 3
41 4 1 4 1 4
42 4 2 4 2 4
43 4 3 4 4 3
2
44 4 4 4
11
12 2 1 2 1
cos( )
sin( )
cos( )
cos( )
sin( )
cos( )
cos( )
0
l m
a d l m l l m
a d l m l l m
a I d m l m
a d l m
a d l m
a d l m
a d l m
a I d m
c
c d l m l
2 3 1 2 4 1 2cos( )l m l l m
13 3 1 3 1 3 4 1 3
14 4 1 4 1 4
21 12
22
23 3 2 3 2 3 4 2 3
24 4 2 4 2 4
31 13
32 23
33
34 4 3 4 4 3
41 14
42 24
43 34
44
1 1 1 1 2 1 3 1 4
cos( )
cos( )
0
sin( )
sin( )
0
sin( )
0
(
c d l m l l m
c d l m
c c
c
c d l m l l m
c d l m
c c
c c
c
c d l m
c c
c c
c c
c
g d m l m l m l m
1
2 2 2 2 3 2 4 2
3 3 3 3 4 3
4 4 4 4 4 3 4 3 4 3 4
4 2 4 2 4 2 4
) cos
( ) sin
( ) sin
sin 2 cos( )sin( )
sin( )
g
g d m l m l m g
g d m l m g
g d m g d l m
d l m
Ref. code: 25615822043435QIF
![Data processing provisions—DPA 1998 and GDPR compliant—pro-controller · 2019. 7. 2. · Controller within three [3] Business Days of receipt of the request; 5.1.2 provide such](https://img.pdfslide.net/doc/110x75/604617e9bc942968cf363747/data-processing-provisionsadpa-1998-and-gdpr-compliantapro-controller-2019.jpg)
