Development of a Human Computer Interface

Abstract HCI(human computer interaction) has become one of the important aspect in human life. Signals generated from human body are biosignals and has huge potential to be used as an interface for human computer devices. Multiple devices are present that recognizes these boiosignals which is generated during muscle contraction and converting those signals into some command to be used as an input to the HCI devices. However, the task can be acquired through biosignals which forms a neural linkage with the computer techniques like Electro-Encephalogram(EEG), Electrooculogram(EOG), and Electromyogram(EMG). In past, there have been lots of studies wherein many researchers have used biosignals to control other device. EMG is hence, one of the least explored mechanism form of biosignal to be deployed in HCI and its studies are useful for neuromuscular system as certain diseases may slow down muscle contraction and muscle firing leading to paralysis of muscle.

Keywords: EMG, HCI, biosignals, skeletal muscles, neural linkage.

1 Introduction

HCI is the one of the research area that emerged in early 1980s, which has expanded rapidly it was previously known as a man- machine interaction. HCI focuses on the interface between user and the computer and deals with the design, execution and assessment of computer system and other related receptive that are for human use. Designing interactive computer systems to be effective, efficient, easy and enjoyable to use is important, so that people and society may realize the benefits of computation based devices [1]. The researchers observes the way human interacts with the computer system and design new technologies and interface that lets human and computers to interaction novel ways [2]. Some of the example of popular HCI techniques are image processing, speech recognition, bio signal processing etc. HCI’s goal is to minimize the differences between the human’s goal of what they want to achieve and the understanding level of computer to perform the task. It relates knowledge from both the human and machine side. Due to its multidisciplinary nature, people with different study areas contribute to its success. Figure 1 shows the areas where HCI can be implemented with distinctive importance.

Fig.1. Disciplines contribute to HCI [3]

EMG is an electro medical procedure for estimating and recording the electrical signals produced by skeletal muscle. EMG is performed using electromyography, to produce an electrical record or signal called electromyogram [4]. An electromyography detects the electric potential generated by skeletal muscle cells when these cells are activated electrically or neurologically. The EMG technology helps capture gestures as inputs for virtual joysticks, keyboards leading to new application in mobile computing etc [5]. This signal can also be analyzed to detect medical abnormalities, activation level, or biomechanics of human movement. The motor neurons of a human body transmit electrical signals that causes muscle to contract and an EMG translate this signals to graphs, sound or numerical values that can be interpreted by analyst. EMG’s signal can be easily acquired using electrodes and it is of two types, dry electrode that is direct contact with the skin that records muscles activity from the surface above the muscle on the skin and require more than one electrode, because EMG recording displays the electric potential difference between two separate electrodes, second is gel or inserted EMG which can be performed using a electrolytic gel as a chemical interface between the skin and electrolyte [6]. A needle electrode and fine wire electrode is the example of inserted electrode. Needle electrode is used in clinical areas and the tip of the electrode is bare and used for the surface detection. Fine wire electrode they are easily implanted in and withdrawn from the skeletal muscles, and is less painful then needle electrode. Thus EMG has a variety of clinical and biomedical applications where it is used to diagnose neuromuscular disease and many other disorders of motor control.

2EMG Used for HCI

Studies are being carried out for the use of EMG signals inorder to identify disabilities as a significant number of individuals are suffering from severe motor disabilities, due to variety of causes, such as Spinal Cord Injury (SCI), Amythorphic Lateral Sclerosis (ALS) and so on [7]. Therefore, EMG signal are not only used for identifying neuromuscular disorder but can also be as a control signals for prosthetic devices [8]. It is the least explored compared to others biosignals like EEG, EOG etc. EMGs are natural means of HCI because the electrical signals induced by human muscle movement during its contraction represents nueromuscular movement that can be interpreted and transformed into computer’s control command. EMG signals can be used for a number of applications including clinical applications, HCI and interactive computer gaming. Basically EMG can be used to sense isometric muscular activity which does not transalate into movement thus making it possible to classify subtle motionless gestures and to control interfaces without being noticed and without disrupting the surrounding environment [9]. The EMG signal have different signatures i.e, two peoples’ gesture might be identical but their characteristics EMG signals are different interms of their age, muscle development skin fat layer and gesture style. One of the problem of EMG is its signal contains a different type of noise that are caused by equipment noise, electromagnetic radiation etc and hence preprossing is needed to filter out the unwanted noise in EMG signal.

3Related works

Researchers have worked on regarding how EMG signal is used to command some other devices like prosthetic arm, robots or enabling people with certain disabilities. These are shown in following paper.

In 1996 Yasuharu Koike et.al, [10] developed a human interface employing a model of an arm, robot control of an artificial hand, and the learning of motion capability. The aim of this paper was to construct a complete forward dynamics model of the human arm by using Artificial Neural Network (ANN). The model has the ability to learn physiological recordings of EMG signals for simultaneous measurement of movement.

In 2000 Alsayegh et.al, [11] proposed an EMG based signal where EMG signal is limited to three arm muscles (medial Deltoid) MD, (anterior deltoid) AB, (biceps brachii) BB that was able to recognize 12 arm gestures. The processing of EMG signal is based on arm gestures having unique temporal coordination. The classification technique used is context dependent classification within the framework of Bayes theorem. Not only the unique arm gesture by using EMG signal was developed there were various researchers working in the field of EMG for the people suffering with motor disabilities like hand paralysis, leg paralysis etc.

In 2004 Jong sung kim et.al, [12] proposed a natural means of human computer interaction induced by human arm’s muscle movement and the generated EMG signal to be used as computer commands control. The paper developed an online EMG MOUSE system that controls movement of the cursor, which are interpretation of 6 pre-defined motions, up, down, left, right, click and rest. A Fuzzy Min Max Neural Network (FMMNN) is used as a classifier.

In 2005 Inhyuk Moon et.al, [13] proposed a novel wearable EMG based HCI for the wheelchair user with severe motor disabilities caused by C4 and C5 spinal cord injury. The EMG signal is acquired by left, right and both shoulder elevation motion. EMG wearable device directly generates MAV (Mean Average Value) signal from raw EMG. The MAV signal is converted to digital data using AD converter embedded in a high speed microcontroller. The recognized result is sent to the wheelchair controller via Bluetooth communication module. The following year one more paper regarding people suffering from motor disabilities was presented.

In 2006 Ki-Hong Kim et.al, [14] developed an interface that relies on EMG signal acquired from human face during contraction of muscle. Electrodes are placed around forehead, cheeks and eyes. The subject was made to perform some actions like blinking of eyes, clenching of teeth, wrinkling of forehead and frowning. The signal is acquired and analyzed using LPC (Linear Prediction Coefficient) and LPC entropy were calculated to find the characteristics information contained in the measured signal. For pattern recognition Hidden Markov Model (HMM) is used. Same year some were working on hand gesture recognition.

In 2006 Ganesh R Naik ei.al, [15] proposed an approach to identify hand gestures using muscle activity separated from electromyogram using ICA (Independent Component Analysis). The aim of the experiment in this paper was to test the use of ICA for separation of the EMG signals for the purpose of identifying hand gestures and actions.

After the recognition of hand gestures and enabling motor disabilities, in 2008 JonghwaKim et.al, [16] proposed modification of a RC car that is controlled by user’s hand signs, instead of using remote control unit. The interfacing system first calculates relevant features in the EMG signal of four hand signs, classifies the hand signs into the four classes, and assigns the result to certain steering commands for the RC car. For feature extraction RMS was used calculated by observing last 16 incoming values. For classification KNN and Bayes theorem was combined using decision tree and purpose a control the car via PC.

Similarly in 2009 Jun-Ru Ren et.al, [17] studied an Electromyogram Based on HCI. This paper showed a control system using forearm electromyography that is proposed for computer peripheral control and artificial prosthesis control. The system intends to realize the commands of six pre defined hand poses i.e. up, down, left, right, yes and no. Power spectral density (PSD) is used to measure signal power intensity and for classifier the Bayesian classifier is used for extracting feature. In the same year Ahsan et.al, [9] classified EMG signal techniques to help improve interface for disabled people. This paper discusses various methodologies and techniques for interpreting EMG signal.

Researchers extended their study to multistep EMG classification in 2010 Armando Barreto et.al, [19] proposed a system that can effectively help disabled people from the neck down to interact with computer or communicate with people through computers using point and click graphic interfaces. The EMG signal is generated using facial muscle with a corresponding cursor movement command.

In 2011 surface EMG has attracted an attention of researchers for interface signal. Ishii et.al, [20] studied about myoelectric prosthetic in which arm/hand gesture is distinguished by identification of the surface Electromyogram. For identification of motion neural network is used.

In 2012 Takeshi Tsujimura et.al, [21] studied Hand Sign Classification Employing Myoelectric Signals of Forearm. The purpose of this paper was to design an uncomplicated system to identify finger motion and to develop innovative HMI. This paper also distinguishes the hand signs by analyzing the forearm EMG signals. It relies on the proposition that the specific muscles of forearm work even if fingers are moved.

Researchers studied through multichannel surface EMG signals and in 2014 Han Li et.al, [4] showed HCI system Based on the multichannel SEMG of the hand gesture recognition based on the feature extraction, identification, classification and control of the SEMG which controls quad copter flight. In this paper, the four different gestures can be distinguished accurately to complete the real-time interactive process. The experimental results show that the HCI system based on SEMG has high accuracy. Auto regression method is used for analysis of SEMG signal and the classification is done using back propogation technique.

In 2015 Ahmed Mehaoua et.al, [18] designed a novel EMG based system that aims to control multimedia player in simple, efficient and flexible manner. The objective of this paper was to provide efficient control system seeking to simplify the life of hand amputee persons by allowing them to control media player through EMG signals generated by muscle activation from forearm contraction. The electrical potential generated allows start, stop a video or switching between a set of media. For detecting muscle contraction four steps is used rectification, filtering, linear envelop and onset contraction and turns the signal into usable form. After detection of muscle contraction, system was enhanced by adding commands like start, stop, previous, next and pause.

4Summary of Survey

The survey paper focuses on evaluation and detection of an EMG signal and use of this system for real time. There are many classification methodologies and artificial intelligence techniques based on neural network to classify EMG signal. Some of the techniques are ANN, K-Nearest Neighbor (KNN), and Support Vector Machine (SVM) etc.

4.1. Back Propagation Neural Network

Back propagation algorithm is applied on the multichannel SEMG [18] of the hand gesture recognition based on the featureextraction and control of the SEMG which controls quad copter flight. BP neural network contains three parts: the BP neural network building, the BP neural network traning and the BP neural network classification.BP neural network building is determined according to the input and output charasteristics of the system structure of the BP neural network. The number of the AR (auto regression) model coefficientof input vector is 16 and the number of the output is 4,so theinput layer has 16 nodes and the output layer has 4 nodes.

4.2. Fuzzy Min Max Neural Network

Jong-Sung Kim [12] applied fuzzy mean max neural network (FMMNN) as a classifier for online EMG mouse that controls computer cursor. Also, stochastic values such as integral absolute value were used as features for an appropriate classification of the intended wrist motions. 6 predefined wrist motions to left, right, up, down, click and rest operation were determined.

4.3. Hidden Markov Model

Ki-Hong Kim [14] developed an interface using EMG signal from human face.For pattern recognition HMM comprised three states and two Gaussian mixtures per state is employed which is used as a classifier. The standalone interface system was implemented and the subject (people as volunteers) were able to make the wheelchair turn left, right, forward and backward by simple action provided by them. Classification is done by comparing the likelihood values of an arbitrary feature sequence evaluated from four HMMs, HMML, HMMR, HMMF, and HMMB for left, right, forward, and backward, respectively, and selecting the model with the maximum value.

4.4. Bayes Network

Alsayegh et.al, [11] presented an EMG-based human-machine interface system that interprets arm gestures in the 3-dimensional (3D) space. Gestures are interpreted by sensing the activities of three muscles, namely, anterior deltoid (AD), medial deltoid (MD), and biceps brachii (BB) muscles. The problem of gesture classification is carried out in a framework of the statistical pattern recognition. The processing of the EMG signals utilizes the temporal coordination activity of the monitored muscles to identify a particular gesture. The classification procedure is carried out by constructing successive feature vectors for each gesture. These feature vectors describe the gesture’s temporal signature. This type of classification is referred to as the context-dependent classification, which is carried out in this study within the framework of Bayes theorem. The development of an EMG based interface for hand gesture recognition is presented by Jonghwa Kim et.al, [16]. For realizing real-time classification assuring acceptable recognition accuracy, they introduced the combination of two simple linear classifiers (K-nearest neighbour (KNN) KNN and Bayes) in decision level fusion.

Table 1 provides the summary of the survey in accordance with the methodologies used in various papers. It provides the description of the success rate resulted by the use of classification techniques.

Table 1.Summary of major methods used for EMG classification

Classifier used	Title & Researchers	Description
Back Propagation Neural Network	Human computer interaction system design based on surface EMG signals. Han Li , Xi Chen, et.al, (2014)	•93% success rate in the multichannel SEMG of the hand gesture recognition. •Auto-regressive model method is used.
Hidden Markov Model	A practical biosignal-based human interface applicable to the assistive systems for people with motor impairment. Ki-Hong Kim et.al (2006)	•97% success rate in developing an interface using human face. •Subject was able to turn left, right, forward and backward.
Fuzzy Min Max Neural Network	A new means of HCI: EMG-mouse. Jong-sung Kim et.al, (2004)	•Stochastic values such as integral absolute values were used as feature extraction. •Six distinctive wrist motions can be classified well. •Pattern recognition rate of each wrist motions is above 90%.
Bayes Network	A practical EMG-based human-computer interface for users with motor disabilities. Alsayegh et.al,(2000)	•classification is done in a framework of statistical pattern recognition. •classification rate reported was 96%.
	EMG-based hand gesture recognition for realtime biosignal interfacing. Jonghwa Kim et,al, (2008)	•K-Nearest Neighbour (k-NN) classifier added with Bayes to obtain good result •Average classification rate reported was over 94%.

5Conclusion

Developing better human computer interface will help improve quality of life of people suffering from physical disabilities. EMG signal is one of the natural technique that captures electrical signals from human body for the use of HCI and provides an interface for human and computer to interact appropriotely. This survey paper focuses on the work of various researchers, the methodologies used for the classification of EMG signal. Therefore, it can be concluded from the survey of various paper that neural network has been used as a prominent classification technique of EMG signal for HCI.

For future works new and more enchanced classification techniques can be developed besides neural network, a work can be done in creating light weight EMG signal, multiclass hand process and on-line processing.

References

1. Dix, A.: Human-computer interaction. (pp. 1327-1331). Springer US (2009)

2. Andurkar, A. G., Andurkar, R. G.: Human-computer interaction. International Research Journal of Engineering and Technology (IRJET), vol.2, issue.6, (2015)

3. Human computer Interaction: An Overview, http://www.ee.cityu.edu.hk/~hcso/ee4213_ch1.pdf

4. Li, H., Chen, X., & Li, P.: Human-computer interaction system design based on surface EMG signals. In: Modelling Identification & Control (ICMIC) Proceedings of the 6th International Conference on (pp. 94-98). IEEE (2014, December)

5. Chowdhury, R. H., Reaz, M. B., Bakar, A. A., & Hasan, M. S.: Muscle Technology. 6(12), 2192-2196, (2013)

6. Day, S.: Important factors in surface EMG measurement. Bortec Biomedical Ltd publishers, 1-17, (2002)

7. Barreto, A. B., Scargle, S. D., & Adjouadi, M.: A practical EMG-based human-computer interface for users with motor disabilities. Journal of rehabilitation research and development, 37(1), 53, (2000)

8. Ali, A. A., Albarahany, A., & Quan, L.: EMG signals detection technique in voluntary muscle movement. In: Information Science and Service Science and Data Mining (ISSDM), 6th International Conference on New Trends in (pp. 738-742). IEEE, (2012, October)

9. Ahsan, M. R., Ibrahimy, M. I., & Khalifa, O. O.: EMG signal classification for human computer interaction: a review. European Journal of Scientific Research, 33(3), 480-501, (2009)

10. Koike, Y., & Kawato, M.: Human interface using surface electromyography signals. Electronics and Communications in Japan (Part III: Fundamental Electronic Science), 79(9), 15-22, (1996)

11. Alsayegh, O. A.: EMG-based signal processing system for interpreting arm gestures. In: Signal Processing Conference, 2000 10th European (pp. 1-4). IEEE, (2000, September)

12. Kim, J. S., Jeong, H., & Son, W.: A new means of HCI: EMG-mouse. In: Systems, Man and Cybernetics, 2004 IEEE International Conference on (Vol. 1, pp. 100-104). IEEE, (2004, October)

13. Moon, I., Lee, M., Chu, J., & Mun, M.: Wearable EMG-based HCI for electric-powered wheelchair users with motor disabilities. In: Robotics and Automation, 2005. ICRA 2005. Proceedings of the 2005 IEEE International Conference on (pp. 2649-2654). IEEE, (2005, April)

14. Ki-Hong, K. I. M., Jae-Kwon, Y. O. O., Kim, H. K., Wookho, S. O. N., & Soo-Young, L. E. E.: A practical biosignal-based human interface applicable to the assistive systems for people with motor impairment. IEICE transactions on information and systems, 89(10), 2644-2652, (2006)

15. Naik, G. R., Kumar, D. K., Singh, V. P., & Palaniswami, M.: Hand gestures for HCI using ICA of EMG. In: Proceedings of the HCSNet workshop on Use of vision in human-computer interaction-Volume 56 (pp. 67-72). Australian Computer Society, Inc., (2006, November)

16. Kim, J., Mastnik, S., & André, E.: EMG-based hand gesture recognition for realtime biosignal interfacing. In: Proceedings of the 13th international conference on intelligent user interfaces (pp. 30-39). ACM, (2008, January)

17. Ren, J. R., Liu, T. J., Huang, Y., & Yao, D. Z.: A study of Electromyogram based on human-computer interface. Journal of electronic science and technology of China, 7(1), 69-73, (2009)

18. Hammi, M. T., Salem, O., & Mehaoua, A.: An EMG-based Human-Machine Interface to control multimedia player. In: E-health Networking, Application & Services (HealthCom), 2015 17th International Conference on (pp. 274-279). IEE, (2015, October).

19. Ren, P., Barreto, A., & Adjouadi, M.: Multi-step EMG classification algorithm for human-computer interaction. In: Innovations in Computing Sciences and Software Engineering (pp. 183-188). Springer Netherlands, (2010)

20. Ishii, C.: Recognition of Finger Motions for Myoelectric Prosthetic Hand via Surface EMG. INTECH Open Access Publisher, (2011)

21. Tsujimura, T., Yamamoto, S., & Izumi, K.: Hand Sign Classification Employing Myoelectric Signals of Forearm. CURRENT APPLICATIONS AND FUTURE CHALLENGES, 309, (2012)