Virtual Control of Prosthetic Hand Based on Grasping Patterns and Estimated Force from Semg

Myoelectric prosthetic hands aim to serve upper limb amputees. The myoelectric control of the hand grasp action is a kind of real-time or online method. Thus it is of great necessity to carry on a study of online prosthetic hand electrical control. In this paper, the strategy of simultaneous EMG decoding of grasping patterns and grasping force was realized by controlling a virtual multi-degree-freedom prosthetic hand and a real one-degree-freedom prosthetic hand simultaneously. The former realized the grasping patterns from the recognition of the sEMG pattern. The other implemented the grasping force from sEMG force decoding. The results show that the control method is effective and feasible.


Introduction
Myoelectric prosthetic hand aims to serve people with upper limb amputation, whose myoelectric controlling process is a kind of real-time or online control.Thus it is of great necessity to carry on a study of online prosthetic hand electrical control, to verify the validity and feasibility of the EMG decoding algorithm.
Generally, only after being trained, can the myoelectric controlling hand, basing on pattern recognition, properly managing its movement or the power of gripping.During the training course, the disable is asked to take advantages of imagination or phantom limb sensation to carry out his willing action, which is a very complex process in which artificial limb user could easily have psychological pressure by excessive times of training or training time [1,2].The emergence of virtual reality technology makes it possible for the disables easier to focus on training.The disables can interact with the virtual prosthetic hand via the EMG data, pattern recognition algorithms and 3D visual effect and enjoy the feeling of being in real environment.In addition, virtual reality(VR) technology can be used in muscle checking, rehabilitation training, and the man-machine interface performance validation [3][4][5].
In this paper, the recognition algorithm of grasping movement patterns in experiments will be verified constantly through the virtual multi-freedom prosthetic hand controlled by the surface EMG signal obtained from a normal person.movement patterns and synchronizing decoding of grasping force combine with virtual multifreedom prosthetic hand and real unidirection prosthetic hand, will be online tested.Because the unidirection prosthetic hand has one limited action, different grasping patterns will be carried out by the virtual multi-freedom prosthetic hand; while the corresponding value of the gripping force can be synchronously realized by the unidirection prosthetic hand.

Building the virtual multi-freedom prosthetic hand system
VR technology is widely used in training disabled patients without limbs with great training improvement..As shown in Figure1, virtual prosthetic hand system of electro myographic control, integrated with electro myographic signal acquisition system, virtual prosthetic hand model, electrical control algorithm, computer and user, is very convenient to verify and evaluate the EMG decoding algorithm.This paper, based on Virtual Reality Modeling Language(VRML) [6,7] and Simulink 3D Animation, establishes a virtual environment for multifreedom prosthetic hand.

Figure.1 Soft and Hard Configuration Of Emg Control System Of Virtual Prosthetic Hand
Experimental platform of virtual prosthetic hands is shown in Figure 1.It can be seen from the picture that 8 wireless channels of the myoelectricity limb band collect EMG signal from the hand under the sampling frequency 1000HZ.The data then can be transmitted to a computer via Bluetooth in real-time, which uses the software developed by MS Visual Studio 2010 to achieve serial port communication with the Bluetooth device of the EMG control system,subsequently, the present sEMG data can be acquired by the serial communication protocol.At the same time, MATLAB calls the sEMG data in VS2010 and get the current pattern of movement according to sEMG data segmentation method and EMG decoding algorithm.The pattern is executed by virtual prosthetic hand in MATLAB/Simulink environment.This paper researches on the recognition of EMG signal patterns and the transmission of recognized signal of virtual prosthetic hand and mainly focuses on the 8 patterns of movements.(Natural state, spherical gripping, cylindrical gripping, tripod pinching, two fingers pinching, side pinching, hooking grasping, and cellphone fetching) According to the action code, the hand can perform the corresponding action patterns which include already made planning paths.In sum, VS collects the online sEMG data, shares the data with Matlab internal variables, and carry out online control of virtual prosthetic hand at the same time.

Experiment platform for EMG control of myoelectric prosthetic hand based on gripping pattern and synchronous force decoding
Limited by the experimental conditions, it is impossible to achieve the situation--applying the operation mode of electromyographic signals and the synchronous force decoding method to myoelectric control of artificial limb.The method adopted is to fulfill the myoelectric control of the virtual multi-DOF prosthetic hand and the single degree of freedom prosthetic hand in the meantime, so that virtual multi-DOF prosthetic hand is able to achieve various operation modes, while the single degree of freedom prosthetic hand attains the needed grasping force after EMG force decoding.As shown in Figure 4, the thumb of single degree of freedom prosthetic hand has installed Force Sensing Resistor (FSR), which can gain pressure information by measuring the resistance value variation when under pressure.FSR, constant resistances and the constant DC electrical source are installed in series to measure the resistance value at both ends of FSR when single degree of freedom prosthetic hand grasps objects.As a result, the resistance value and the pressure match.5th-order polynomial has been applied to correct the FSR pressure.

Gripping pattern and synchronous force decoding for EMG control of myoelectric prosthetic hand
In this paper, the method separates the myoelectric control of operation mode and grasping forces but operating both in the meantime to verify the synchronization myoelectric decoding algorithm of operation mode and grasping forces.As is shown in Figure 5, the operation mode of electromyogram signal decoded and the trainer control the grasping mode of virtual multi-DOF prosthetic hand.Meanwhile, grasping level after sEMG force decoding is regarded as the force for reference of single degree of freedom prosthetic hand and the grip can be handled by bottom force controller.
In Figure 5, after received the movement pattern by decoding the EMG data, virtual prosthetic hand will follow the already made path to do gripping, according to the corresponding pattern; The reference force of the unidirection prosthetic hand comes from the decoding of surface EMG data; when gripping, FSR is under pressure, and the actual gripping force will feedback to NI realtime control system; The deviation of reference force and actual force will be input into the force controller whose output data will be then transmit through driver to the DC motor of the unidirection prosthetic hand to adjust the gripping force.The trainer can control the virtual prosthetic hand and the unidirection prosthetic hand at the same time by EMG data.
Based on the feasibility of Linear Discriminant Analysis(LDA), Mel Frequency Cepstrum Coefficient, and ARC+SSC [8][9][10], the experiment asks the trainer to control the movement pattern of the virtual prosthetic hand and the actual gripping force of the unidirection prosthetic hand at the same time.The method of decoding EMG data is the judgement of threshold; the underlayer force controller of the unidirection prosthetic hand is PID controller; The trainer will grip under a comfortable and natural condition among all of the patterns; Other training samples of gripping are only for the three level of cylindrical gripping, in which the trainer act 10 times and the training time is 50s.

Verification and analysis of experiments
As shown in Figure 6, a trainer performed two kind of online EMG control experiment with different levels of gripping force----cylindrical gripping and spherical gripping.The experiment includes 8 stages: 10s for natural stage; 10s for small force gripping; 10s for moderate force gripping; 10s for large force gripping; another 10s for natural stage; 10s for small force gripping; 10s for moderate force gripping; and 10s for large force gripping.According to the training data set, the threshold of EMG signal decoding set to 1 10   , 2 20   and 3 40   .It is shown in Figure 7 and Figure 8 the result of EMG force control of unidirection prosthetic hand.
By analyzing the result of the experiment of gripping pattern and EMG force control, which are shown in Figure .7 and Figure .8, it is demonstrated that as soon as the virtual prosthetic hand acted its movement pattern, the EMG force control of unidirection prosthetic hand would be carried out.However, there still shows poor stability of decoding of gripping force: cylindrical gripping with large force was frequently misjudged to moderate one; decoding of spherical gripping with moderate force contained level of small force.As shown in Figure 9, a trainer has successively performed 7 gripping patters: spherical gripping, cylindrical gripping, tripod pinching, two fingers pinching, side pinching, hooking grasping, and cellphone fetching.Every pattern has 140s in sum, including 4 cyclic stages: 5s for natural state; 5s for small force gripping; 5s for moderate force gripping; 5s for large force gripping.In the same way, the threshold of EMG signal decoding set to , and .Decoding algorithm of gripping pattern is the threshold method; underlayer force control of the unidirection prosthetic hand employs PID controller.
Accuracy of grasping patterns of EMG controlling virtual prosthetic hand is 21/28.Misjudged patterns are mainly side pinching (4), and cylindrical grasping (2).As shown in Figure 9 the result of EMG controlling unidirection prosthetic hand, generally, shows high misjudgment rate of gripping with large force, high accuracy rate of cylindrical gripping and spherical gripping, and side pinching with lowest accuracy rate.As shown in Fig. 10, the trainer grips a sponge by using EMG controlling unidirection prosthetic hand in the experiment.It can be qualitatively demonstrated that force level of the trainer's hand almost shares the same level of the prosthetic hand.To be more specific, result of EMG force control is shown in Fig. 11.MAV characteristics of the threshold of EMG signal sets on enhanced non-stationary and nonlinearity of sEMG in the process of muscle relaxation, become higher as the stronger grasping force shifted to a weaker one.
In a word, the on-line experiment for myoelectric control of artificial limb certifies the grip mode and forces synchronization decoding algorithm.The stability and recognition rate of grasping mode decoding perform well, with about 80% of on-line recognition under the condition of small training set and the changing level of muscle contraction.However, the grip myoelectric decoding is not stable enough, waiting a further improvement of the EMG force decoding method.Besides, the bottom forces control effect of single degree of freedom prosthetic hand, whose controller needs to be improved more, is unsatisfying.

Conclusion
In this paper, it has been built the experiment platform for simultaneously control of virtual prosthetic hand and real unidirection prosthetic hand.EMG gripping patterns and force synchronous decoding algorithm are used to control the gripping forces of the virtual and the real prosthetic hand.The control method has certain feasibility that has been tested by online experiments, and the decoding method of grasping pattern shows excellent stability and high recognition rate.

Figure 2 Figure 3 Figure. 4
Figure 2 Soft and Hard Configuration of Emg Control System Of Virtual Prosthetic Hand Based on Movement Patterns and Synchronizing Decoding of Gripping Force

Figure 5
Figure 5 Prosthetic Hand Motion Modes and Grasping Force Separation and Synchronous Emg Control Methods

Figure. 6
Figure.6 Experimental Scene of Emg Force Control of Prosthetic Hand Based on Movement Pattern and Real-Time Decoding of Grasping Force

Figure 7 Figure 8 .
Figure 7 Result of Emg Control of Prosthetic Hand with Cylindrical Gripping

Figure. 9 Figure 10
Figure.9 Result of Emg Control of Prosthetic Hand in Different Patterns   correspond to sEMG decoding force 0N, 1.5N, 4.5N and 7.5N respectively.Fig.10 also shows that MAV feature fluctuates intensely under the identical grasping forces level, which means myoelectric signals have stronger non-stationary, especially in the initial stage, MAV feature fluctuates strongest, for the instability of muscle contraction at the moment, which easily causes the misjudgment during EMG force decoding.Besides, Fig.10 also reveals that the error rates of EMG force decoding, caused by the