Upper-Limb EMG-Based Robot Motion Governing Using Empirical Mode Decomposition and Adaptive Neural Fuzzy Inference System

DOI 10.1007/s10846-012-9677-6

Upper-Limb EMG-Based Robot Motion Governing

Using Empirical Mode Decomposition and Adaptive

Neural Fuzzy Inference System

Hsiu-Jen Liu· Kuu-Young Young

Received: 31 March 2011 / Accepted: 30 April 2012 / Published online: 22 May 2012 © Springer Science+Business Media B.V. 2012

Abstract To improve the quality of life for the

dis-abled and elderly, this paper develops an upper-limb, EMG-based robot control system to pro-vide natural, intuitive manipulation for robot arm motions. Considering the stationary and non-linear characteristics of the Electromyography (EMG) signals, especially when multi-DOF move-ments are involved, an empirical mode decom-position method is introduced to break down the EMG signals into a set of intrinsic mode functions, each of which represents different physical charac-teristics of muscular movement. We then integrate this new system with an initial point detection method previously proposed to establish the map-ping between the EMG signals and corresponding robot arm movements in real-time. Meanwhile, as the selection of critical values in the initial point detection method is user-dependent, we em-ploy the adaptive neuro-fuzzy inference system to find proper parameters that are better suited

K.-Y. Young (

_B

Department of Electrical Engineering, National Chiao Tung University,

1001 University Road, Hsinchu 300, Taiwan e-mail: [email protected]

H.-J. Liu

National Space Organization,

8F, 9 Prosperity 1st Road, Hsinchu Science Park, Hsinchu 30078, Taiwan

e-mail: [email protected]

for individual users. Experiments are performed to demonstrate the effectiveness of the proposed upper-limb EMG-based robot control system.

Keywords Electromyography (EMG)·

Human-assisting robot· Upper-limb motion classification· Empirical mode decomposition (EMD)· Adaptive neuro-fuzzy inference system (ANFIS)

1 Introduction

Electromyography (EMG) signals, generated dur-ing muscle contraction, in some sense reflect hu-man intentions. Therefore, some research has focused on the study of using EMG signals to control rehabilitation devices [1–4] and human-assisting manipulators [5–10], so that the physi-cally handicapped as well as the elderly may use them to improve their mobility and quality of life. However, since nonlinear, non-stationary charac-teristics and high variations are inherently present in EMG signals, these disturbances make it hard to analyze and discriminate among EMG signals. Consequently, figuring out how to achieve a high discrimination rate for EMG pattern recognition still remains a challenge. Several methods for recognizing the intended movements from EMG signals have been proposed. In the time domain, there are mean absolute value, variance, bias

zero-crossing, Willison amplitude, AR-model [1,

2,11], Euclidean distance and standard deviation [12], and hidden Markov model [13], etc. In the frequency domain, there are Fourier transform [14] and wavelet analysis [15, 16], etc. Unfortu-nately, the time-domain approaches usually in-clude high computational complexity [2]. As for approaches in frequency domain, the data must be linear and strictly stationary for Fourier transform [17] to work; however, EMG signals are non-linear and non-stationary signals, especially for contraction levels higher than 50% of maximum voluntary contraction [18]. Meanwhile, a mother wavelet has to be defined a priori for wavelet analysis [19]. Unsuitable mother wavelets may lead to unsatisfactory results.

In contrast, Hilbert–Huang transform (HHT) is a time-frequency method. Based on the local time scale of the data, HHT breaks down a signal into several intrinsic mode functions (IMFs) via empir-ical mode decomposition (EMD), and then calcu-lates the instantaneous frequency of each IMF at any point in time via Hilbert transform. Hence, it is suitable for nonlinear and non-stationary data analysis. HHT has been broadly applied in numerous scientific disciplines and investigations, e.g., analysis on the bioelectricity signal, failure testing, and earthquake signals, etc [20]. Several researchers have applied HHT to EMG related studies. Xie et al. [18] and Peng et al. [21] adopted HHT to find the features of muscular fatigue. Ma and Luo [20] proposed using HHT and AR-model to extract the surface EMG feature to recog-nize hand-motions. Wang et al. [22] presented a feature extraction technique based on EMD to classify the walking activities from accelerometry data. Chen et al. [23] employed HHT to extract the frequency features of the stump to control transfemoral prosthesis. Zong and Chetouani [17] presented a feature extraction technique based on HHT for emotion recognition from physio-logical signals. In our previous research [24], we employed the EMD method to extract the upper-limb EMG signals for governing 1-DOF robot arms in real time.

This paper proposes developing an upper-limb EMG-based robot control system that can govern human assisting robot manipulators in natural and intuitive manner. The developed system is not

intended to serve as a classifier that accurately identifies human intention from EMG signals. The development of such a classifier needs to well consider the influence from the muscle type, strength of muscle contraction, fatigue level, strat-egy in performing the task, and others. Instead, we attempt to develop a system for effective ro-bot motion governing in real-time. At the current stage, we aim to govern the robot manipulator of 2 DOFs via the EMG signals measured from four surface electrodes placed on Biceps Brachii, Triceps Brachii, Pectoralis Major, and Teres Mi-nor of the human operator. To tackle the non-stationary and nonlinear EMG signals generated during multi-joint movements, the EMD method is employed to break down the EMG signals into a set of IMFs, representing different physical characteristics for muscular movement recogni-tion. We then utilize the initial point detection method previously proposed [10] to establish the relationship between the upper-limb EMG signals and corresponding robot arm movements in real time. As the selection of critical values in the initial point detection method is user-dependent, we need to find proper system parameters that are better suited for individual users. For such sys-tem parameter search, the adaptive neuro-fuzzy inference system (ANFIS) [10,25] is employed to realize the fuzzy system, due to high complexity exhibited by the 2-DOF movements. A series of experiments are performed to demonstrate the effectiveness of the proposed system. Some other learning schemes have been proposed for related applications. Chan et al. [26] proposed a fuzzy approach to classify single-site EMG signals for prosthesis control based on the time-segmented features, which uses the Basic Isodata algorithm to cluster the features without supervision and the back-propagation algorithm to train the fuzzy rules. Vachkov and Fukuda [27] proposed struc-tured learning and decomposition of fuzzy models for robotic control. The random walk algorithm with variable step size was used to tune the an-tecedent parameters of the membership functions and a local learning algorithm to tune the conse-quent parameters of the singletons.

The rest of the paper is organized as follows. In Section2, the upper-limb EMG-based robot con-trol system is described, including the modules for

EMG signal measurement and processing, EMD feature extraction, and motion classification with adaptation. Section 3 presents the experimental results and discussions. The conclusion is given in Section4.

2 Proposed Upper-Limb EMG-Based Robot Control System

Figure 1 shows the system diagram of the pro-posed upper-limb, EMG-based robot control sys-tem for governing a 2-DOF robot manipulator, which consists of three main modules: signal mea-surement and processing, EMD feature extrac-tion, and motion classification with adaptation. The signal measurement and processing module measures the raw EMG signals and also filters out noise. The filtered EMG signals are then sent to the EMD feature extraction module to break down the signals into a set of IMFs. With the IMFs, the motion classification with adaptation module then determines the arm movement of the operator and generates the commands to drive the human-assisting robot. From the resultant robot motion, the operator evaluates the performance and determines the next movement. These three modules are described below.

2.1 Signal Measurement and Processing

For this upper-limb, EMG-based robot control system, we extract the EMG signals of Biceps Brachii, Triceps Brachii, Pectoralis Major, and Teres Minor. To obtain more precise EMG

sig-nals, the electrodes are placed on the belly of the muscle. The recommended inter-electrode distance (from one differential electrode to the other) is about 1∼2 cm [28, 29]. Several types of noises may affect the measurement of the EMG signals, such as ECG crosstalk, electro-magnetic induction from power lines, and arm and cable movements. The ECG crosstalk can be suppressed by measuring signals from those muscles away from the heart. The frequency of the electromagnetic noise is around 60 Hz. While a notch filter at that frequency can be an option, it should be avoided, because EMG generates many extra signals at these and neighboring frequen-cies (the primary frequency of the EMG signal is 50∼150 Hz). We thus let the proposed approach tackle its influence as the disturbance. Meanwhile, the frequency distribution for the arm and cable movements is around 0–20 Hz, which can be han-dled using a high-pass filter.

2.2 Empirical Mode Decomposition Feature Extraction

Hilbert–Huang transform (HHT) [21,23,30] is an adaptive signal processing technique based on the local characteristic time scale of the data. The key part of the HHT is the EMD process, which uses the sifting process to break down a complicated signal without a basis function, such as sine or wavelet functions, into several IMFs that are em-bedded in the complicated signal [31]. Each IMF, linear or nonlinear, represents a simple oscillation, which has the same number of extremes and zero-crossings. Figure 2 shows the flow chart of the

Fig. 1 Proposed upper-arm EMG-based robot control system

Robot motion Filtered EMG Upper-limb motion Signal Measurement & Processing IMFs Motion command EMD Feature Extraction Motion Classification with Adaptation Human- Assisting Robot Operator

Fig. 2 Flow chart for empirical mode decomposition No Filtered EMG Signals Sifting Process Intrinsic Mode Functions Trend or Constant Yes

Empirical Mode Decomposition

x(t) h (t)i r (t)i

Stop

EMD process, which breaks down the complete signal into a set of IMFs in eight steps:

Step 1 Obtain the upper envelop U(t) of the filtered EMG data x(t) by using a cubic spline curve to interpolate between those maximum points.

Step 2 Apply the same actions in Step 1 to obtain the lower envelope L(t).

Step 3 Compute the mean value of the upper and lower envelope m1(t):

m1(t) =

U(t) − L(t)

2 (1)

Step 4 Subtract the running mean value m1(t) from the original data x(t) to obtain the first component h1(t):

h1(t) = x(t) − m1(t) (2)

Step 5 Iterate Steps 1–4 on h1(t) for k times

h11(t) = h1(t) − m11(t)

• •

h1k(t) = h1(k−1)(t) − m1(k−1)(t)

(3)

until the resulting component h1k(t) sat-isfies two conditions: (a) the difference between the number of local extremes and that of zero-crossings is zero or one and (b) the running mean value is zero. In Eq.3, m1_(k−1)(t) is the mean value of the upper and lower envelope of h_1(k−1)(t). Step 6 Designate c1(t) = h1k(t) if h1k meets the

two requirements mentioned above.

Step 7 Subtract the first IMF c1(t) from the orig-inal data to obtain the residual r1(t):

r1(t) = x(t) − c1(t) (4)

Step 8 Treat r1(t) as the new data and repeat Steps 1–7 on r1(t) to obtain all the subsequent i.e., r2(t) = r1(t) − c2(t) , . . .,

rn(t) = rn−1(t) − cn(t) until the final residual rn(t) meets the predefined stopping criteria as a monotonic function, considered as the trend.

Based on the procedure above, the original data

x(t) can be exactly reconstructed by a linear

su-perposition: x(t) = n  i=1 ci(t) + rn(t) (5)

where n is the number of IMFs.

The number of IMFs depends on the charac-teristics of the data. Complex data can be de-composed into more IMFs, which increases the computational load in EMD. As EMG signals are nonlinear, non-stationary, and varying, using a high data sampling window would lead to a situation where the mapping between the upper-limb EMG signals and corresponding robot arm movements cannot be established within the ex-pected time interval. After several experiments, it was found that combining a sixth-order band-pass Butterworth filter (a type of filter designed to have as flat a frequency response as possible in the passband) with the cut-off frequencies at 20 and 400 Hz with a window containing 20 samples

per second could reduce the computational load and noises, and this was used afterward. Figure3

shows a typical example of the EMD process when a muscle relaxes and then flexes. Each of example includes the empirical EMG signal x(t), three IMFs (c1(t), c2(t), c3(t)), and residue (r3(t)).

Judging from Fig. 2a and b, c1(t) should be the background noise induced by skin impedance and temperature, cable movement, etc., since its mag-nitude did not vary with arm movement evidently. In contrast, those of c2(t) and c3(t) did vary. We found that the variation of the magnitude of c2(t)

Fig. 3 A typical example of an empirical EMG signal and corresponding empirical mode

decomposition components, including 3 IMFs and 1 residue (trend): muscle in a relaxation and b flexion

0 2 4 6 8 10 12 14 16 18 20 -20 0 20 x( t) 0 2 4 6 8 10 12 14 16 18 20 -10 0 10 0 2 4 6 8 10 12 14 16 18 20 -5 0 5 0 2 4 6 8 10 12 14 16 18 20 -5 0 5 0 2 4 6 8 10 12 14 16 18 20 -5 0 5 Time (Samples)

(a) Muscle in relaxation

0 2 4 6 8 10 12 14 16 18 20 -20 0 20 0 2 4 6 8 10 12 14 16 18 20 0 0 0 0 2 4 6 8 10 12 14 16 18 20 0 0 0 0 2 4 6 8 10 12 14 16 18 20 .50 5x 10 -3 0 2 4 6 8 10 12 14 16 18 20 .50 5 Time (Samples) (b) Muscle in flexion c1 (t) c2 (t) c3 (t) c1 (t) c2 (t) c3 (t) r3 (t) r3 (t) x( t)

is incremental, which basically followed that of the muscle activities, while that of c3(t) is decremen-tal. Therefore, c2(t) was identified to represent the limb movement.

2.3 Motion Classification with Adaptation To achieve real-time motion classification, we adopt the initial critical value detection method previously proposed. Note that this method is not intended to provide a classifier that accurately identifies human intention from EMG signals, but to efficiently establish the relationship between the upper-limb EMG signal and corresponding movements. Its feasibility and effectiveness have been verified via experiments based on one-joint upper-limb movement [10, 24]. Details of the method can be referred to [10,24], and are briefly described below.

The initial point detection method determines the onset of the upper limb motion by detecting the instant when the magnitude of the extracted feature reaches the critical value, as illustrated in Fig. 4. In Fig. 4, the state of the muscle, MS, is determined to be active when the value for the extracted feature Fk, calculated by the root mean square (RMS) method, is larger than the predefined upper critical value CVu, and MS is inactive when Fk is smaller than the predefined lower critical value CVl. Furthermore, an active

MS corresponds to an “ON” robot command and

Fk u CV State ON OFF State Robot Command OFF l CV

Fig. 4 Conceptual diagram for initial value detection

an inactive one to “OFF.” This relationship can be formulated as: MS=  1, if Fk> CVu 0, if Fk< CVl (6)

where Fk= RMS(c2(t)), 0 ≤ t ≤ 20 samples, and

c2(t) is the 2nd IMF.

The proper selection of CVu and CVl is criti-cal for the performance of the motion classifier. And, their selection is user-dependent. Finding the proper CVu and CVl for individual users is very challenging, especially when limb move-ments of multi-DOF are involved. In the search of proper CVuand CVl, we employ the adaptive neuro-fuzzy inference system (ANFIS) [25] to uti-lize the fuzzy system due to its excellence at adap-tation. Figure 5shows the conceptual diagram of the proposed ANFIS for CVuand CVl determina-tion, which consists of fuzzy rule, fuzzifier, fuzzy inference engine, and defuzzifier. We utilize the empirical knowledge to generate the fuzzy rules, listed in Table 1, where INa, INb, INc, and INd

stand for the EMG signals of BB, TB, PM, and TM, respectively, OUT for CVu, and W, M, S, VL, L, H, and VH for weak, middle, strong, very low, low, high, and very high. Fuzzifier transforms the measured EMG signals of Biceps Brachii (BB), Triceps Brachii (TB), Pectoralis Major (PM), and Teres Minor (TM) into linguistic variables. In addition, a one-degree Sugeno-type inference sys-tem is employed to depict the fuzzy rules in the fuzzy inference engine. The fuzzy rules are formu-lated as:

Ri: IF BB is Aiand TB is Biand PM is Ci

and TM is DiTHEN CVu (CVl) = pi× BB + qi× TB + ri× PM + si

× TM, i ∈ {1, 2, . . . , 54} (7) where BB, TB, PM, and TM are the input variables, A, B, C, D = {W, M, S} the linguis-tic variables, CVu(CVl) the output variable, and [ piqirisi] the consequent parameter set, which can be determined by the least-squares method. Defuzzifier transforms the fuzzy results of the inference into a real CVu(CVl) using the weighted averaged method.

Fig. 5 Conceptual diagram of the proposed ANFIS to determine CVu and CVl Fuzzy Inference Engine Fuzzy Rule Base ANFIS Biceps Brachii Fuzzifier Defuzzifier Training Data Fuzzy System Triceps Brachii Teres Minor CVl,BB,TB,PM,TM (CVu,BB,TB,PM,TM) Pectoralis Major

Via extensive experiments, the values of the W, M, and S in INa ∼INd and those of VL, L,

M, H and VH in OUT are empirically set, as shown in Table 2, and CVl is set to be 0.7 times the value of CVu. Since some conditions, such as skin impedance and temperature, etc. may be different from those for training, the values afore-mentioned can be adjusted via a compensating

factorλ, ranging from 0.8 to 1.3. The procedure for determining CVuandCVlis described as follows: Step 1 Set the compensating parameterλ to be

0.8 in the proposed ANFIS to obtain the first set of CVuand CVl.

Step 2 Ask the operator to perform the motion of flexion, extension, internal rotation,

Table 1 Fuzzy rule base _{Rule No. IN}

a INb INc INd OUT Rule No. INa INb INc INd OUT

1 W W W W VL 28 M W W W L 2 W W W M VL 29 M W W M L 3 W W W S L 30 M W W S M 4 W W M W VL 31 M W M W L 5 W W M M L 32 M W M M M 6 W W M S L 33 M W M S M 7 W W S W VL 34 M W S W L 8 W W S M L 35 M W S M M 9 W W S S L 36 M W S S M 10 W M W W L 37 M M W W M 11 W M W M L 38 M M W M M 12 W M W S L 39 M M W S M 13 W M M W M 40 M M M W H 14 W M M M M 41 M M M M H 15 W M M S M 42 M M M S H 16 W M S W M 43 M M S W H 17 W M S M M 44 M M S M H 18 W M S S M 45 M M S S H 19 W S W W H 46 M S W W VH 20 W S W M H 47 M S W M VH 21 W S W S H 48 M S W S VH 22 W S M W H 49 M S M W VH 23 W S M M H 50 M S M M VH 24 W S M S H 51 M S M S VH 25 W S S W VH 52 M S S W VH 26 W S S M VH 53 M S S M VH 27 W S S S VH 54 M S S S VH

Table 2 Values of the linguistic variables in the fuzzy rule base

INa∼ INd OUT

W M S VL L M H VH

1/3 2/3 1 1/3 2/3 1 4/3 5/3

and external rotation two times consecutively.

Step 3 If the successful discrimination rate (SDR), defined in Eq. 8below, is lower

than 80%, add 0.1 toλ, and iterate Steps 2 and 3 until the SDR is equal to or more than 80%.

SDR= Number of successful motions following Total number of classif ications

× 100% (8)

The proposed ANFIS consists of five layers, as shown in Fig. 6. Layer 1 is the input layer. Each

Fig. 6 Structure of the ANFIS for the proposed system A1 A2 B1 B2 Π A3 B3 Π Π Π Π Π N N N N N N Σ BB TB

Layer1 Layer2 Layer3 Layer4 Layer5

) ( _l u CV CV 1 w C1 C2 D1 D2 C3 D3 TM PM 1 w CVu1 (CVl1 ) 2 w CVu2 (CVl2) 3 w CVu3 (CVl3) 4 w CVu4 (CVl4) 5 w CVu5 (CVl5) 6 w CVu6 (CVl6) Π Π Π Π Π N N N N N Π N 49 w CVu49 (CVl49) 50 w CVu50 (CVl50) 51 w CVu51 (CVl51) 52 w CVu52 (CVl52 ) 53 w CVu53 (CVl53) 54 w CVu54 (CVl54 )

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

node in this layer represents an input variable of the model with the membership function:

O1_i = μAi(BB) , O 1 i+3= μBi(T B) , O1_I+6 = μCi(PM) , O 1 i+9= μDi(T M) , i = 1, 2, 3 (9) The bell-shaped membership function is em-ployed, shown in Fig.7, and is expressed as:

μXi(Y) =

1 1+(Y − ci) /ai

2bi, i = 1, 2, 3 (10)

where [X, Y] ∈ {[A, BB], [B, T B], [C, PM], [D,

T M]}, [aibici] represent the premise parameter set, which can be determined by the backpropa-gation gradient descent method. Layer 2 is the in-ference layer. Each node in this layer is multiplied by the input signal to becomewi:

O2_i = wi= μAi(BB) × μBi(T B) × μCi(PM) × μDi(T M) , i = 1, 2, · · ·, 54 (11)

wistands for the firing strength of the rule. Layer 3 is the normalization layer that normalizes the firing strength by calculating the ratio of ithfiring strength to their sum:

O3_i = wi= _54wi

j=1wj

, i = 1, 2, · · ·, 54 (12)

Layer 4 is the output layer. Each node multiplies the normalized firing strength by the consequent function to generate the qualified consequent of each rule. The output of the node is computed as:

O4_i = wiCVu(CVl)i

= wi(piBB+ qiT B+ riPM+ siT M) ,

i= 1, 2, · · ·, 54 (13)

Layer 5 is the defuzzification layer, which com-putes the weighted average of the output signals from the output layer:

O5 i = 54 i=1wiCVu(CVl)i = 54 i=1wiCVu(CVl)i 54 i=1wi (14) 3 Experiments

We performed a series of experiments to evaluate the performance of the proposed system. Figure8

shows the implementation of the proposed system for experiments. In Fig. 8, the measured EMG signals are first amplified using the ETH-256 phys-iological signal amplifier (manufactured by iWorx Systems, USA), and the amplified analog signals (voltages) then transformed into digital signals Fig. 7 Bell-shaped

membership functions for input variables 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 W M S 0.4 0.5 0.6 0.7 0.8 0.9 1 W M S Input variable: BB (PM) Input variable: TB(TM) Membership Function ( µ )

Fig. 8 System implementation of the proposed scheme LabView Development System EMG signal Analog signal Digital signal EMG Amplifier Data Acquisition Device Robot motion Motion command Butterworth Filter EMD Feature Extractor Motion Classifier (Initial Point Detection & ANFIS)

via a National Instrument USB-6009 A/D data acquisition device with a 1 KHz sampling rate. The digital signals are further forwarded to the LabVIEW development system, which includes a sixth-order band-pass Butterworth filter with the cut-off frequencies at 20 and 400 Hz, respectively, a 20-sampling-data-window EMD feature extrac-tor; and a motion classifier with the initial point detector and ANFIS. By using this processing method, robot motion commands can be deter-mined, and then sent to a 6-DOF Mitsubishi RV-2A robot manipulator for execution (with only J1 and J2 manipulated). The entire experimental setup is shown in Fig.9.

Four sets of electrodes were placed on BB, TB, PM, and TM, as shown in Fig. 10. The clas-sifier is designed to let the feature extracted from BB, TB, PM, and TM correspond to upper limb

ETH-256 RV-2A

LabVIEW Develop System USB-6009 A/D

Fig. 9 Experimental setup

flexion, extension, internal rotation, and external rotation, respectively, and those for both BB and PM and both TB and TM together for flexion-internal rotation and extension-external rotation, respectively. Their muscle states will determine whether it is an up, down, turn-left, turn-right, up-left, or down-right movement. Due to some mus-cle crosstalk or imprecise feature identification, there may be some conflict movement decisions between the two muscles. Under such circum-stances, the classifier will send out an error signal. Totally, there are eight outputs for the classifier: STOP, UP, DOWN, LEFT, RIGHT, UP-LEFT, DOWN-RIGHT, and ERROR. Table 3 summa-rizes the mapping from EMG to robot movement, and Fig.11illustrates the classification outputs of 1∼6 corresponding to the robot arm movements.

Two male and two female subjects (with their physical data listed in Table 4) were asked to perform the following two experiments: (1) the motions of flexion, extension, internal rotation, and external rotation for four times consecutively, and (2) the motions of flexion plus internal ro-tation and extension plus external roro-tation for three times consecutively. The first experiment was used to evaluate the capability of the pro-posed system to recognize those motions related to basically one set of muscles (BB and TB or PM and TM), while the second experiment was used to recognize the motions involving the interaction between muscles, which incurred larger mutual interferences.

Figure12shows the results for the first exper-iment, which includes the subjects’ filtered raw

Fig. 10 Electrode locations: a Biceps Brachii and Pectoralis Major and b Triceps Brachii and Teres Minor

(a) (b) Biceps Brachii (CH1) Pectoralis Major (CH3) Triceps Brachii (CH2) Teres Minor (CH4)

Table 3 Mapping from EMG to robot movement

EMG Upper limb status Classifier Robot arm

BB TB PM TM output

(CH1) (CH2) (CH3) (CH4)

OFF OFF OFF OFF Relaxation 0 STOP

ON OFF OFF OFF Flexion 1 J2 axis UP

OFF ON OFF OFF Extension 2 J2 axis DOWN

OFF OFF ON OFF Internal Rotation 3 J1 axis TURN LEFT

OFF OFF OFF ON External Rotation 4 J1 axis TURN RIGHT

ON OFF ON OFF Flexion-Internal Rotation 5 J1 axis TURN LEFT and J2 axis UP OFF ON OFF ON Extension-External Rotation 6 J1 axis TURN RIGHT and J2 axis DOWN

All others Error 7 STOP

Fig. 11 Illustrations of classification outputs corresponding to the robot arm movements

6 5 4 3 2 1 0 Classifier Output

Table 4 Physical data of the subjects

Subject Height Weight Gender Body

(cm) (kg) type

A 166 60 Male Slender

B 164 82 Male Overweight

C 155 52 Female Normal

D 150 50 Female Normal

EMG signals, muscle states, and classification out-puts. The muscle states reveal that subjects C and D exhibited certain muscle mutual interferences

during movements, especially for subject D. The SDR for the subjects is 97%, 99%, 87.9%, and 81.8%, respectively. For these four kinds of mo-tions dominated by basically one set of muscles, the proposed system achieved quite high a suc-cessful discrimination rate, even for subject D.

Figure 13 shows the results of the second ex-periment. Different from the motions executed in the first experiment, these motions involved the onsets of two muscles simultaneously, leading to larger mutual interferences and couplings be-tween muscles. In Fig.13, some muscle states of

0 200 400 600 800 1000 1200 1400 1600 1800 2000 2200 -20 0 20 CH 1 Ra w E M G 0 10 20 30 40 50 60 70 80 90 100 110 0 2 4 CH1 M S 0 200 400 600 800 1000 1200 1400 1600 1800 2000 2200 -20 0 20 CH 2 Ra w E M G 0 10 20 30 40 50 60 70 80 90 100 110 0 5 10 CH2 M S 0 200 400 600 800 1000 1200 1400 1600 1800 2000 2200 -20 0 20 CH 3 Ra w E M G 0 10 20 30 40 50 60 70 80 90 100 110 0 5 10 CH 3 M S 0 200 400 600 800 1000 1200 1400 1600 1800 2000 2200 -20 0 20 CH4 Ra w E M G 0 10 20 30 40 50 60 70 80 90 100 110 0 2 4 CH 4 M S 0 10 20 30 40 50 60 70 80 90 100 110 0 2 4 6 C la s s ifi ca ti o n O u tp u t Time (Samples) (a) Subject A 0 500 1000 1500 2000 2500 3000 -20 0 20 CH1 Raw E M G 0 50 100 150 0 2 4 CH1 M S 0 500 1000 1500 2000 2500 3000 -20 0 20 CH 2 Raw E M G 0 50 100 150 0 5 CH2 M S 0 500 1000 1500 2000 2500 3000 -20 0 20 CH 3 Raw E M G 0 50 100 150 0 2 4 CH 3 M S 0 500 1000 1500 2000 2500 3000 -20 0 20 CH4 Ra w E M G 0 50 100 150 0 2 4 CH4 M S 0 50 100 150 0 2 4 6 C la s s ifi c a ti o n O u tp u t Time (Samples) (b) Subject B

Fig. 12 Experimental results for the motions of flexion, extension, internal rotation, and external rotation for four times consecutively

0 500 1000 1500 2000 2500 -20 0 20 CH 1 Ra w E M G 0 20 40 60 80 100 120 140 0 5 CH1 M S 0 500 1000 1500 2000 2500 -20 0 20 CH 2 Ra w E M G 0 20 40 60 80 100 120 140 0 5 10 CH2 M S 0 500 1000 1500 2000 2500 -20 0 20 CH 3 Ra w E M G 0 20 40 60 80 100 120 140 0 5 10 CH 3 M S 0 500 1000 1500 2000 2500 -20 0 20 C H 4 Ra w E M G 0 20 40 60 80 100 120 140 0 5 CH 4 M S 0 20 40 60 80 100 120 140 0 2 4 6 8 C las s if ic a ti on O u tp ut Time (Samples) (c) Subject C (d) Subject D 0 500 1000 1500 2000 2500 3000 3500 4000 4500 -20 0 20 C H 1 Ra w E M G 0 50 100 150 200 0 2 4 CH 1 M S 0 500 1000 1500 2000 2500 3000 3500 4000 4500 -20 0 20 CH 2 R a w E M G 0 50 100 150 200 0 5 10 CH 2 M S 0 500 1000 1500 2000 2500 3000 3500 4000 4500 -20 0 20 CH 3 Raw E M G 0 50 100 150 200 0 2 4 CH 3 M S 0 500 1000 1500 2000 2500 3000 3500 4000 4500 -20 0 20 C H 4 Ra w E M G 0 50 100 150 200 0 5 10 CH 4 M S 0 50 100 150 200 0 2 4 6 8 C la s s ifi c a ti o n O u tp u t Time (Samples) Fig. 12 (continued).

0 100 200 300 400 500 600 700 800 900 1000 -20 0 20 CH 1 Ra w E M G 0 5 10 15 20 25 30 35 40 45 50 0 5 CH 1 M S 0 100 200 300 400 500 600 700 800 900 1000 -20 0 20 CH 2 Ra w E M G 0 5 10 15 20 25 30 35 40 45 50 0 5 10 CH 2 M S 0 100 200 300 400 500 600 700 800 900 1000 -20 0 20 CH 3 Ra w E M G 0 5 10 15 20 25 30 35 40 45 50 0 2 4 CH 3 M S 0 100 200 300 400 500 600 700 800 900 1000 -20 0 20 CH 4 Ra w E M G 0 5 10 15 20 25 30 35 40 45 50 0 5 CH 4 M S 0 5 10 15 20 25 30 35 40 45 50 0 2 4 6 8 C la s s if ic a ti o n O u tp u t Time (Samples) (a) Subject A 0 200 400 600 800 1000 1200 -20 0 20 CH 1 R a w E M G 0 10 20 30 40 50 60 0 5 10 CH 1 M S 0 200 400 600 800 1000 1200 -20 0 20 CH2 Raw E M G 0 10 20 30 40 50 60 0 5 CH 2 M S 0 200 400 600 800 1000 1200 -20 0 20 CH 3 Ra w E M G 0 10 20 30 40 50 60 0 5 CH 3 M S 0 200 400 600 800 1000 1200 -20 0 20 CH4 R a w E M G 0 10 20 30 40 50 60 0 5 10 CH4 M S 0 10 20 30 40 50 60 0 2 4 6 8 C la s si fi c a ti o n O u tp u t Time (Samples) (b) Subject B

Fig. 13 Experimental results for the motions of flexion plus internal rotation and extension plus external rotation for three times consecutively

0 200 400 600 800 1000 -20 0 20 CH 1 Ra w E M G 0 10 20 30 40 50 0 2 4 CH1 M S 0 200 400 600 800 1000 -20 0 20 C H 2 Ra w E M G 0 10 20 30 40 50 0 5 CH 2 M S 0 200 400 600 800 1000 -20 0 20 CH 3 Ra w E M G 0 10 20 30 40 50 0 5 10 CH 3 M S 0 200 400 600 800 1000 -20 0 20 C H 4 Ra w E M G 0 10 20 30 40 50 0 5 10 CH 4 M S 0 10 20 30 40 50 0 2 4 6 8 C la s s ifi c a ti o n O u tp u t Time (Samples) (c) Subject C 0 100 200 300 400 500 600 700 800 900 1000 -20 0 20 C H 1 Raw E M G 0 5 10 15 20 25 30 35 40 45 50 0 5 10 CH 1 M S 0 100 200 300 400 500 600 700 800 900 1000 -20 0 20 CH 2 R a w E M G 0 5 10 15 20 25 30 35 40 45 50 0 5 10 CH 2 M S 0 100 200 300 400 500 600 700 800 900 1000 -20 0 20 CH 3 Raw E M G 0 5 10 15 20 25 30 35 40 45 50 0 5 10 CH 3 M S 0 100 200 300 400 500 600 700 800 900 1000 -20 0 20 CH4 Ra w E M G 0 5 10 15 20 25 30 35 40 45 50 0 5 10 CH 4 M S 0 5 10 15 20 25 30 35 40 45 50 0 2 4 6 8 C la s s ifi c a ti o n O u tp u t Time (Samples) (d) Subject D Fig. 13 (continued).

BB and PM (also TB and TM) did not simulta-neously reach the individual critical upper value. However, the SDR for the subjects was found to be 84.6%, 92.3%, 76.9%, and 99%, respectively, indicating that the proposed system still managed these complex motions well. Videos for these ex-periments can be located via the link connected to our laboratory Web page (http://140.113.149.114).

4 Conclusion

This paper presents an upper-limb EMG-based robot control system for achieving natural and in-tuitive robot manipulation. By integrating the ini-tial point detection method previously proposed with the EMD approach and the ANFIS, the map-ping between the upper limb EMG signals and corresponding robot arm movements has been established in real-time. Experiments have been performed to demonstrate its effectiveness. For the potential user to apply the proposed system for movements involving upper or other limbs, or involving more DOFs, they can follow the pro-posed procedure, possibly at the expense of more sophisticated learning schemes to deal with the incurring complexity. In future works, we plan to investigate the possibility of using varying CVs, because fixed CVs lead to consistent classification for about 5∼10 min, depending on the status of muscle fatigue. Our intention is to let the system maintain a high successful discrimination rate for a longer time. We also plan to apply the proposed system for full limb movement governing.

Acknowledgements This work was supported in part by the National Science Council under grant NSC 99-2221-E-009-157, and also Department of Industrial Technology under grants 97-EC-17-A-02-S1-032.

References

1. Kermani, M.Z., Wheeler, B.C., Badie, K., Hashemi, R.M.: EMG feature evaluation for movement control of upper extremity prostheses. IEEE Trans. Rehabil. Eng. 3(4), 324–333 (1995)

2. Huang, H.P., Chen, C.Y.: Development of a myoelec-tric discrimination system for a multi-degree prosthetic hand. In: IEEE International Conference on Robotics and Automation (1999)

3. Chen, L., Yang, P., Zu, L., Guo, X.: Movement recog-nition by electromyography signal for transfemoral prosthesis control. In: IEEE Conference on Industrial Electronics and Applications (2009)

4. Harada, A., Nakakuki, T., Hikita, M., Ishii, C.: Robot finger design for myoelectric hand and recognition of finger motions via surface EMG. In: IEEE Conference on Automation and Logistics (2010)

5. Fukuda, O., Tsuji, T., Kaneko, M., Otsuka, A.: A human-assisting manipulator teleoperated by EMG signals and arm motions. EEE Trans. Robot. Autom. 19(2), 210–222 (2003)

6. Artemiadis, P.K., Kyriakopoulos, K.J.: EMG-based po-sition and force control of a robot arm: application to teleoperation and orthosis. In: IEEE/ASME Interna-tional Conference on Advanced Intelligent Mechatron-ics (2007)

7. Ferreira, A., Celeste, W.C., Cheein, F.A., Bastos-Filho, T.F., Sarcinelli-Filho, M., Carelli, R.: Human-machine interfaces based on EMG and EEG applied to robotic systems. J. Neuroeng. Rehabil. 5(10), 1–15 (2008) 8. Gopura, R.A.R.C., Kiguchi, K.: EMG-based control

of an exoskeleton robot for human forearm and wrist motion assist. In: IEEE International Conference on Robotics and Automation (2008)

9. Bu, N., Okamoto, M., Tsuji, T.: A hybrid motion clas-sification approach for EMG-based human–robot in-terfaces using Bayesian and neural networks. EEE Trans. Robot. 25(3), 502–511 (2009)

10. Liu, H.J., Young, K.Y.: An adaptive upper-arm EMG-based robot control system. Int. J. Fuzzy Syst. 12(3), 181–189 (2010)

11. Li, Y., Tian, Y., Chen, W.: Multi-pattern recognition of sEMG based on improved BP neural network algo-rithm. In: Chinese Control Conference (2010) 12. Phinyomark, A., Hirunviriya, S., Limsakul, C.,

Phukpattaranont, P.: Evaluation of EMG feature extraction for hand movement recognition based on Euclidean distance and standard deviation. In: International Conference on Electrical Engineer-ing/Electronics Computer Telecommunications and Information Technology (2010)

13. Meng, M., She, Q., Gao, Y., Luo, Z.: EMG signals based gait phases recognition using hidden Markov models. In: IEEE International Conference on Infor-mation and AutoInfor-mation (2010)

14. Gora, J., Szecowka, P.M., Wolczowski, A.R.: Control of dexterous hand—algorithm implementation issues. In: International Conference on Information Technol-ogy and Applications in Biomedicine (2009)

15. Karlsson, S., Yu, J., Akay, M.: Enhancement of spec-tral analysis of myoelectric signals during static con-tractions using wavelet methods. IEEE Trans. Biomed. Eng. 46, 670–684 (1999)

16. Mahaphonchaikul, K., Sueaseenak, D., Pintavirooj, C., Sangworasil, M., Tungjitkusolmun, S.: EMG sig-nal feature extraction based on wavelet transform. In: International Conference on Electrical Engineer-ing/Electronics Computer Telecommunications and In-formation Technology (2010)

17. Zong, C., Chetouani, M.: Hilbert–Huang transform based physiological signals analysis for emotion recog-nition. In: IEEE International Symposium on Signal Processing and Information Technology (2009) 18. Xie, H., Wang, Z.: Mean frequency derived via

Hilbert–Huang transform with application to fatigue EMG signal analysis. Comput. Methods Programs Bio-med. 82, 114–120 (2006)

19. Andrade, A.O., Nasuto, S., Kyberd, P., Sweeney-Reed, C.M., Van Kanijn, F.R.: EMG signal filtering based on empirical mode decomposition. Biomed. Signal Process Contr. 1, 44–45 (2006)

20. Ma, W., Luo, Z.: Hand-motion pattern recognition of sEMG based on Hilbert–Huang transformation and AR-model. In: IEEE International Conference on Mechatronics and Automation (2007)

21. Peng, B., Jin, X., Min, Y., Su, X.: The study on the sEMG signal characteristics of muscular fatigue based on the Hilbert–Huang transform. In: Interna-tional Conference on ComputaInterna-tional Science (2006) 22. Wang, N., Ambikairajah, E., Celler, B.G., Lovell, N.H.:

Accelerometry based classification of gait patterns us-ing empirical mode decomposition. In: IEEE Interna-tional Conference on Acoustics, Speech, and Signal Processing (2008)

23. Chen, L., Yang, P., Zu, L., Guo, X.: Movement recog-nition by electromyography signal for transfemoral prosthesis control. In: IEEE Conference on Industrial Electronics and Applications (2009)

24. Liu, H.J., Young, K.Y.: Robot motion governing us-ing upper limb EMG signal based on empirical mode decomposition. In: IEEE International Conference on Systems, Man, and Cybernetics (2010)

25. Jang, J.S.R.: ANFIS: adaptive network based fuzzy in-ference system. IEEE Trans. Syst. Man Cybern. 23(3), 665–683 (1993)

26. Chan, Francis H.Y., Yang, Y.S., Lam, F.K., Zhang, Y.T., Parker, P.A.: Fuzzy EMG classification for pros-thesis control. In: IEEE Trans. Rehabil. Eng. 8(3), 305– 311 (2000)

27. Vachkov, G., Fukuda, T.: Structured learning and de-composition of fuzzy models for robotic control appli-cations. J. Intell. Robot. Syst. 32(1), 1–21 (2001) 28. Luca, C.J.D.: The use of surface electromyography in

biomechanics. J. Appl. Biomech. 13(2), 135–163 (1997) 29. Henryk, K., Grzegorz, S., Anton, N.: Effect of elec-trode position on EMG recording in pectoralis major. J. Hum. Kinet. 17, 105–112 (2007)

30. Huang, N.E., Zhen, S., Long, S.R., Wu, M.C., Shin, H.H., Zheng, Q., Yen, N.C., Tung, C.C., Liu, H.H: The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time se-ries analysis. Proc. R. Soc. Lond. 454(A), 903–995 (1998)

31. Li, X., Li, D., Liang, Z., Voss, Z.J., Sleigh, J.W.: Analy-sis of depth of anesthesia with Hilbert–Huang spec-tral entropy. Clin. Neurophysiol. 119(11), 2465–2475 (2008)