MEMS Accelerometer Based Hand Gesture Recognition

ISSN: 2278 – 1323

International Journal of Advanced Research in Computer Engineering & Technology (IJARCET) Volume 2, No 5, May 2013

MEMS ACCELEROMETER BASED HAND GESTURE RECOGNITION

Meenaakumari.M¹, M.Muthulakshmi2

¹Dept.of ECE, Sri Lakshmi Aammal Engineering College, Chennai,

²Asst.Prof, Dept.of ECE, Sri Lakshmi Aammal Engineering College, Chennai,

Abstract –This paper presents an MEMS accelerometer mostly based on gesture recognition algorithm and its applications. The hardware module consists of a triaxial mems accelerometer, microcontroller, and zigbee wireless transmission module for sensing and collecting accelerations of handwriting and hand gesture trajectories. Users will use this hardware module to write down digits, alphabets in digital kind by making four hand gestures. The accelerations of hand motions measured by the accelerometer are transmitted wirelessly to a personal computer for trajectory recognition. The trajectory algorithm composed of information assortment collection, signal preprocessing for reconstructing the trajectories to attenuate the cumulative errors caused by drift of sensors. So, by changing the position of MEMS (micro electro mechanical systems) we can able to show the alphabetical characters and numerical within the PC.

Keywords – MEMS accelerometer, gesture, handwritten recognition, trajectory algorithm.

9. INTRODUCTION

NOW A DAYS, the expansion of human machine interaction technologies in electronic circuits has been greatly reduced the dimension and weight of consumer electronics products such as smart phones and handheld computers, and therefore will increases our day to day convenience. Recently, an attractive alternative, a conveyable embedded device with inertial sensors, has been projected to sense the activities of human and to capture their motion trajectory information from accelerations for handwriting and recognizing gestures. The foremost necessary advantage of inertial sensors for general motion sensing is that they can be operated without any external reference and limitation in operating conditions. However, motion trajectory recognition is comparatively tough for different users since they have different speeds and styles to generate various motion trajectories. Thus, several researchers have tried to avoid the problem domain for increasing the

 

accuracy of handwriting recognition systems. During this work a miniature MEMS accelerometer based recognition systems which acknowledge four hand gestures in 3-D is constructed by using this four gestures, numerical and alphabets will be recognized in the digital format.

MEMS are termed as micro electro mechanical system where mechanical parts like cantilevers or membranes have been manufactured at microelectronics circuits. It uses the technology known as micro-fabrication technology. It has holes, cavity, channels, cantilevers, membranes and additionally imitates mechanical parts. The emphasis on MEMS is based on silicon. The explanation that prompt that prompt the utilization of MEMS technology are for example miniaturization of existing devices, development of new devices based on principal that do not work at large scale and to interact with micro world. Miniaturization reduces cost by decreasing material consumption. It also increases applicability by reducing mass and size allowing placing the, MEMS in places where a traditional system. Instead of having a series of external components connected by wire or soldered to printed circuit board the MEMS on silicon can be integrated directly with the electronics. These are called smart integrated MEMS already include data acquisition, filtering, data storage, communication interfacing and networking. MEMS technology not only makes the things smaller but often makes them better. A typical example is brought by the accelerometer development.

An accelerometer is a device that measures the physical acceleration. The physical parameters are temperature, pressure, force, light etc. it measures the weight per unit mass. By contrast, accelerometers in free fall or at rest in outer space will measure zero. Another term for the type of acceleration that accelerometers can measure is g-force. It works on the principle of displacement of a small proof mass etched into the silicon surface of the integrated circuit and suspended by small beams.

35. RELATED WORK

1886

www.ijarcet.org

ISSN: 2278 – 1323

International Journal of Advanced Research in Computer Engineering & Technology (IJARCET) Volume 2, No 5, May 2013

There are mainly two existing types of gesture recognition methods, i.e., vision-based and accelerometer and/or gyroscope based. Due to some limitations like ambient optical noise, slower dynamic response, and relatively large data collections/processing of vision-based method [1], our recognition system is implemented based on an inertial measurement unit based on MEMS acceleration sensors. If gyroscopes are used for inertial measurement [2] it causes heavy computational burden, thus our system is based on MEMS accelerometers only and gyroscopes are not implemented. Many researchers have focused on developing effective algorithms for error compensation of inertial sensors to improve the recognition accuracy. For few examples, Yang et al. [3] proposed a pen-type input device to track trajectories in 3-D space by using accelerometers and gyroscopes. An efficient acceleration error compensation algorithm based on zero velocity compensation was developed to decrease the acceleration errors for acquiring accurate reconstructed trajectory. An extended Kalman filter with magnetometers (micro inertial measurement unit (μIMU) with magnetometers), proposed by Luo et al. [10], was employed to compensate the orientation of the proposed digital writing instrument. If the orientation of the instrument was estimated precisely, the motion trajectories of the instrument were reconstructed accurately. Dong et al. [4] proposed an optical tracking calibration method based on optical tracking system (OTS) to calibrate 3-D accelerations, angular velocities, and space attitude of handwriting motions. The OTS was developed for the following two goals: 1) to obtain accelerations of the proposed ubiquitous digital writing instrument (UDWI) by calibrating 2-D trajectories and 2) to obtain the accurate attitude angles by using the multiple camera calibration. However, in order to recognize or reconstruct motion trajectories accurately, the aforementioned approaches introduce other sensors such as gyroscopes or magnetometers to obtain precise orientation. This increases additional cost for motion trajectory recognition systems as well as computational burden of their algorithms.

.

In this paper, a portable device has been developed with a trajectory recognition algorithm. The portable device consists of a triaxial accelerometer, a microprocessor, and an zigbee wireless transmission module. The acceleration signals measured from the triaxial accelerometer are transmitted to a computer via the zigbee wireless module. Users can utilize this portal device to write digits and make hand gestures at normal speed. The measured acceleration signals of these motions can be recognized by the trajectory

 

recognition algorithm. The recognition procedure is composed of acceleration acquisition, signal preprocessing, feature generation, feature selection, and feature extraction. The acceleration signals of hand motions are measured by the portable device. The signal preprocessing procedure consists of calibration, a moving average filter, a high-pass filter, and normalization. First, the accelerations are calibrated to remove drift errors and offsets from the raw signals. These two filters are applied to remove high frequency noise and gravitational acceleration from the raw data, respectively. The features of the preprocessed acceleration signals of each axis include mean, correlation among axes, interquartile range (IQR), mean absolute deviation (MAD), root mean square (rms), VAR, standard deviation (STD), and energy. Before classifying the hand motion trajectories, we perform the procedures of feature selection and extraction methods. In general, feature selection aims at selecting a subset of size m from an original set of d features (d > m). Therefore, the criterion of kernel-based class separability (KBCS) with best individual N (BIN) is to select significant features from the original features (i.e., to pick up some important features from d) and that of linear discriminate analysis (LDA) is to reduce the dimension of the feature space with a better recognition performance (i.e., to reduce the size of m). The objective of the feature selection and featureextraction methods is not only to eradicate the burden of computational load but also to increase the accuracy of classification. The reduced features are used as the inputs of classifiers. The contributions of this paper include the following: 1) the development of a portable device with a trajectory recognition algorithm, i.e., with the hardware module , can give desired commands by hand motions to control electronics devices anywhere without space limitations, and 2) an effective trajectory recognition algorithm, i.e., the proposed algorithm can efficiently select significant features from the time and frequency domains of acceleration signals and project the feature space into a smaller feature dimension for motion recognition with high recognition accuracy.

III.HARDWARE DESIGN OF

PORTABLE DEVICE

The portable device consists of a triaxial accelerometer (MMA2240), a microcontroller (C8051F206 with a 12-b A/D converter), and a wireless transceiver (nRF2401, Nordic). The triaxial accelerometer measures the acceleration signals generated by a user’s hand motions. The microcontroller collects the analog acceleration signals and converts the signals to digital ones via the

1887

www.ijarcet.org

ISSN: 2278 – 1323

International Journal of Advanced Research in Computer Engineering & Technology (IJARCET) Volume 2, No 5, May 2013

A/D converter. The wireless transceiver transmits the acceleration signals wirelessly to a personal computer (PC).The MMA2240 is a low-cost capacitive micro machined accelerometer with a temperature compensation function and a g-select function for a full-scale selection of +_}2 g to +_}6 gand is able to measure accelerations over the bandwidth of 0.5 kHz for all axes. The accelerometer’s sensitivity is set from −2 g to +2 g. The C8051F206 integrates a high-performance 12-b A/D converter and an optimized signal cycle 25-MHz 8-b microcontroller unit (MCU) (8051 instruction set compatible) on a signal chip. The output signals of the accelerometer are sampled at 100 Hz by the 12-b A/D converter. Then, all the data sensed by the accelerometer are transmitted wirelessly to a PC by an zigbee transceiver at 2.4-GHz transmission band with 1-Mb/s transmission rate. The overall power consumption of the digital pen circuit is 30 mA at 3.7 V. The block diagram of the portable device is shown in Fig. 1.

MEMS	PIC
ACCELEROMTER	MICROCON	ZIGBEE  TX
			TROLLER

PC

RS 232

ZIGBEE RX

Fig.1. Block diagram of the portable device.

 

IV. TRAJECTORY RECOGNITION

ALGORITHM

The proposed trajectory recognition algorithm consisting of acceleration acquisition, signal preprocessing, feature generation, feature selection, and feature extraction. In this paper, the motions for recognition include Arabic numerals alphabets. The acceleration signals of the hand motions are measured by a triaxial accelerometer and then preprocessed by filtering and normalization.

Consequently, the features are extracted from the preprocessed data to represent the characteristics of different motion signals, and the feature selection process based on KBCS picks p features out of the original extracted features. To reduce the computational load and increase the recognition accuracy of the classifier, LDA is utilized to decrease the dimension of the selected features. The reduced feature vectors are then fed into a PNN classifier to recognize the motion to which the feature vector it belongs.

A.Signal Preprocessing

The microcontroller collects the acceleration signals of hand motions which are generated by the accelerometer. Due to slight tremble movement of hand certain amount of noise is generated. The signal preprocessing consists of calibration, a moving average filter, a high-pass filter, and normalization. First, the accelerations are calibrated to remove drift errors and offsets from the raw signals. The second step of the signal preprocessing is to use a moving average filter to reduce the high-frequency noise of the calibrated accelerations, and the filter is expressed as

where x[t] is the input signal, y[t] is the output signal, and N is the number of points in the average filter. In this paper, we set N = 8. The decision of using an eight-point moving average filter is based on our empirical tests. Then, a high-pass filter is used to remove the gravitational acceleration from the filtered acceleration to obtain accelerations caused by hand movement. In general, the size of samples of each movement between fast and slow writers is different. Therefore, after filtering the data, we first segment each movement signal properly to extract the exact motion interval. Then, we normalize each segmented motion interval into equal sizes via interpolation.

1888

www.ijarcet.org

ISSN: 2278 – 1323

International Journal of Advanced Research in Computer Engineering & Technology (IJARCET)

Volume 2, No 5, May 2013

5) Correlation among axes: The correlation among axes is computed as the ratio of the covariance to the product of the STD for each pair of axes. For example, the correlation (corrxy) between two variables x on x-axis and y on y-axis is defined as

where E represents the expected value, σx and σx are STDs, and mx and my are the expected values of x and y, respectively.

6)MAD

7)rms

Y-, and z-axis signals, and we extract eight features									where xi is the acceleration instance and m is
from the triaxial acceleration signals, including mean,									the mean value of xi in (6) to (7).
STD, VAR, IQR [6], correlation between axes [7],
MAD, rms, and energy [8] . They are explicated as
follows.	8)   Energy: Energy is calculated as the sum of
1)   Mean: The mean value of the acceleration									the  magnitudes  of  squared  discrete  fast
signals of each hand motion is the dc						Fourier transform (FFT) components of the
component of the signal				signal in a window. The equation is defined
as
where W is the length of each hand motion.
2)   STD: STD is the square root of VAR							where Fi is the ith FFT component of the
							window and |Fi| is the magnitude of Fi.
C. Feature Selection
Feature selection comprises a selection criterion. The
KBCS can be computed as follows: Let (x, y) (Rd Ã-
3)	VAR	Y)	represents  a  sample,  where  Rd   denotes  a  d–
		dimensional feature space, Y symbolizes the set of
class labels, and the size of Y is the number of class c.
This method projects the samples onto a kernel space,
where xi is the acceleration instance and m is								and m i is defined as the mean vector for the I th
								class in the kernel space, ni denotes the number of
the mean value of xi in (3) and (4).
						samples in the ith class, m denotes the mean vector
4)	IQR:  When  different			classes  have				similar		for all classes in the kernel space, S B denotes the
							between-class scatter matrix in the kernel space, and
mean   values,		the	interquartile			range
								S/ Wdenotes the within-class scatter matrix in the
represents	the	dispersion		of	the  data		and
								kernel  space.  Let  (・)  be  a  possible  nonlinear
eliminates	the	influence		of	outliers	in	the
								mapping from the feature space Rd to a kernel space
data.
	κ and tr(A) represents the trace of a square matrixA.

1889

www.ijarcet.org

ISSN: 2278 – 1323

International Journal of Advanced Research in Computer Engineering & Technology (IJARCET) Volume 2, No 5, May 2013

The following two equations are used in the class separability measure:

1

2

3

4

The class separability in the kernel space can be measured as

To maintain the numerical stability in the maximization of J ^∅ , the denominator tr(S^∅ _W ) has to be prevented from approaching zero.

IV. EXPERIMENTAL RESULTS

In this section, the effectiveness of trajectory recognition algorithm is validated.

A.Handwritten Digit Recognition

The acceleration signals after the signal preprocessing procedure of the proposed trajectory recognition algorithm for the digit 0. The calibrated acceleration signals acquired from the accelerometer module are shown. With the preprocessed accelerations, alphabets and numerical features are generated by the feature generation procedure. Subsequently, the KBCS is adopted to choose characteristic features from the generated features. We choose digits 0 and 6 to illustrate the effectiveness of the KBCS, since their accelerations and handwritten trajectories are pretty similar and difficult to classify. The IQR features of these two digits are closely overlapped. Thus, the features are not effective for

1

2

3

4

5

6

7

8

9

10

Fig. 4. Trajectories of four hand gestures.

corrxy, meanz, energyx, energyy, and energyz selected by the KBCS. Finally, the dimension of the selected features was further reduced by the LDA not only to ease the burden of computational load but also to increase the accuracy of classification.

Fig. 5.a Trajectories of alphabets

Fig. 5.b. Trajectories of alphabets.

Fig. 6. IQR features of (red star) digit 0 and (blue diamond) digit 6.

Fig. 3. Generation of numerical

1890

www.ijarcet.org

ISSN: 2278 – 1323

International Journal of Advanced Research in Computer Engineering & Technology (IJARCET) Volume 2, No 5, May 2013

Fig. 6.a. Mean feature of (red star) digit 0 and digit (blue diamond) 6.

Therefore, the total testing samples were 100 (10 Ã- 10 Ã- 1) for the testing procedure, and the total training samples were 900 (10 Ã- 10 Ã- 9) for the raining procedure. Because there are ten digits needed to be classified, the maximum of the dimension of the feature extraction by the LDA was nine. To see the performance variation caused by feature dimensions, we varied the dimensions of the LDA from one to nine. In Fig. 10, the best average recognition rate of

Fig. 7. Average recognition rates versus the feature dimensions of the PNN classifier by using the LDA.

Fig. 8. Average recognition rates versus the feature dimensions of the PNN classifier by using the KBCS.

V. CONCLUSION

The development of a portable device, is used to generate desired commands by hand motions to

 

control electronic devices without space limitations. The time and frequency domains of acceleration signals of motion recognition, which has high recognition accuracy. The acceleration made by the hand gesture is measured by accelerometer are wirelessly transmitted to computer. In the experiments, we used 2-D handwriting digits, alphabets by using four hand gestures to validate the effectiveness of the proposed device and algorithm. The overall handwritten digit recognition rate was 98%, and the gesture recognition rate was also 98.75%. This result encourages us to further investigate the possibility of using our digital pen as an effective tool for HCI applications. In this project, an additional button can be used to allow users to indicate the starting point and ending point of motion. That is, the limitation of the proposed trajectory recognition algorithm is that it can only recognize a letter or a number finished with a single stroke.

VI. FUTURE ENHANCEMENT

The algorithms can be developed for letters or words with multistrokes which involve more challenging problems.

REFERENCES

1. S. Zhou, Q. Shan, F. Fei, W. J. Li, C. P. Kwong, and C. K. Wu et al.,”Gesture recognition for interactive controllers using MEMS motion sensors,” in Proc. IEEE Int. Conf. Nano/Micro Engineered and MolecularSystems, Jan. 2009,pp. 935-940.

2. S. Zhang, C. Yuan, and V. Zhang, “Handwritten character recognition using orientation quantization based on 3-D accelerometer,” presented at the 5th Annu. Int. Conf. Ubiquitous Systems, Jul. 25th, 2008.

3. J. Yang, W. Chang, W. C. Bang, E. S. Choi, K. H.Kang, S. J. Cho, and D. Y. Kim, “Analysis and compensation of errors in the input device based on inertial sensors,” in Proc. IEEE Int. L. Wang, “Feature selection with kernel class separability,” IEEE Trans.Pattern Anal. Mach. Intell., vol. 30, no. 9, pp. 1534-1546, Sep. 2008.

4. Z. Dong, U. C. Wejinya, and W. J. Li, “An optical-tracking calibration method for MEMS-based digital writing instrument,” IEEE Sens. J.,vol. 10, no. 10, pp. 1543-1551, Oct. 2010.

5. J. S.Wang, Y. L. Hsu, and J. N. Liu, “An inertial-measurement-unit-based pen with a trajectory reconstruction algorithm and its applications,” IEEE Trans. Ind. Electron., vol. 57, no. 10, pp. 3508-3521, Oct. 2010.

6. S. Zhou, Z. Dong, W. J. Li, and C. P. Kwong, “Hand-written character recognition using

1891

www.ijarcet.org

ISSN: 2278 – 1323

International Journal of Advanced Research in Computer Engineering & Technology (IJARCET) Volume 2, No 5, May 2013

MEMSmotionsensingtechnology,”in

Proc.IEEE/ASME Int. Conf. Advanced Intelligent

Mechatronics, 2008, pp.1418-1423.

7. J.  K.  Oh,  S.  J.  Cho,  and  W.  C.  Bang  et  al.,

“Inertial sensor based recognition of 3-D character gestures with an ensemble of classifiers,” presented at the 9th Int. Workshop on Frontiers in Handwriting Recognition, 2004.

8. Conf. Inf.Technol.-Coding and Computing,2004, pp. 790-796.

9. Y. Luo, C. C. Tsang, G. Zhang, Z. Dong, G. Shi,

19. Y. Kwok, W. J. Li, P. H. W. Leong, and M. Y.

Wong, “An attitude compensation technique for a MEMS motion sensor based digital writing instrument,” in Proc.IEEE Int. Conf. Nano/Micro Eng. Mol. Syst., 2006, pp. 909-914.

10. Z. Dong, G. Zhang, Y. Luo, C. C. Tsang, G. Shi,

19. Y. Kwok, W. J. Li,P. H. W. Leong, and M. Y.

Wong, “A calibration method for MEMS inertial sensors based on optical tracking,” in Proc. IEEE Int. Conf.Nano/Micro Eng. Mol. Syst., 2007, pp.542-547.

11. S. J. Preece, J. Y. Goulermas, L. P. J. Kenney, and D. Howard, “A comparison of feature extraction methods for the classification of dynamic activities from accelerometer data,”

IEEE Trans. Biomed. Eng., vol. 56,no. 3, pp.871-879, Mar. 2009.

12. L. Bao and S. S. Intille, “Activity recognition from user-annotated acceleration data,”

Pervasive, Lecture Notes in Computer Science,no. 3001,pp. 1-17, 2004.

13. Y. P. Chen, J. Y. Yang, S. N. Liou, G. Y. Lee, and J. S. Wang, “Online classifier construction algorithm for human activity detection using a triaxial accelerometer,” Appl. Math. Comput., pp. 849-860, Nov. 2008.

14. L. Wang, “Feature selection with kernel class separability,” IEEE Trans.Pattern Anal. Mach. Intell., vol. 30, no. 9, pp. 1534-1546, Sep. 2008.

15. Z. Dong, U. C. Wejinya, and W. J. Li, “An optical-tracking calibration method for MEMS-based digital writing instrument,” IEEE Sens.

10. ,vol. 10, no. 10, pp. 1543-1551, Oct. 2010.

16. J. S.Wang, Y. L. Hsu, and J. N. Liu, “An inertial-measurement-unit-based pen with a trajectory reconstruction algorithm and its applications,” IEEE Trans. Ind. Electron., vol. 57, no. 10, pp. 3508-3521, Oct. 2010.

17. S. Zhou, Z. Dong, W. J. Li, and C. P. Kwong, “Hand-written character recognition using

MEMS   motion   sensing   technology,”   in

Proc.IEEE/ASME Int. Conf. Advanced Intelligent Mechatronics, 2008, pp.1418-1423.

18. J.  K.  Oh,  S.  J.  Cho,  and  W.  C.  Bang  et  al.,

“Inertial   sensor   based   recognition   of   3-D

 

character gestures with an ensemble of classifiers,” presented at the 9th Int. Workshop on Frontiers in Handwriting Recognition, 2004.

19. A.  H.  F.  Lam,  W.  J.  Li,  Y.  Liu,  and  N.  Xi,

“MIDS: Micro input devices system using MEMS sensors,” presented at the IEEE/RSJ Int. Conf.Intelligent Robots and Systems (IROS), Oct. 2002.

20. G. Zhang, G. Shi, Y. Luo, H. Wong, W. J. Li, P.

H. W. Leong, and M. Y. Wong, “Towards an ubiquitous wireless digital writing instrument using MEMS motion sensing technology,” presented at the IEEE/ASME Int. Conf. Advanced Intelligent Mechatronics (AIM), Monterey, CA, Jul. 2005.

1892

www.ijarcet.org