SHS Web Conf.
Volume 139, 2022The 4th ETLTC International Conference on ICT Integration in Technical Education (ETLTC2022)
|Number of page(s)||8|
|Section||Topics in Computer Science|
|Published online||13 May 2022|
Study of Deep Learning-based Hand Gesture Recognition Toward the Design of a Low-cost Prosthetic Hand
1 1The University of Aizu, School of Computer Science and Engineering, Adaptive Systems Laboratory, Japan
* Corresponding author: SAIKAWA Yamatoe-mail: email@example.com
Background: The surface EMG (sEMG) signal is inherently noisy and, therefore, not a robust input source for prosthetic systems, especially for fatigue, electrode displacement, and sweat conditions. We propose to address these issues by designing a multi-modal approach that combines vision and EMG empowered with appropriate dataset collection.
Methods: In Frame-based, the machine learning model used for recognition was a 2D-CNN. The data is image data that is input to the model by preparing videos showing 10 patterns of hand gestures along with multiple backgrounds, and dividing these videos into frames. These image data are then pre-processed and input to the machine learning model. The model is then evaluated in terms of the accuracy of hand gesture identification using the test data and the loss value, which represents the error between the expected data and the correct data output. In EMG, the Myo armband is placed on the forearm and the sEMG of 200 (Hz) is measured. There are six patterns of hand gestures in this process. Similar to the images, these sEMG data are preprocessed and input to a machine learning model for classification. The model is evaluated the model by the accuracy of hand gesture identification using the test data and the loss value, precision, recall , F1-score.
Results: The value of the loss function in case of frame-based was 0.0770 and the accuracy was 0.9739 at 1000 epochs of the training data. And the value of the loss function values in the test data were 0.1011 for the loss value and 0.9657 for the accuracy. In the case of EMG, the loss value was 0.931 when the time to maintain the gesture was the longest, and the loss value was 0.171. However, Precision, Recall, and F1-score were not the highest at the longest time for some gestures.
Conclusion: In this paper, we created a hand gesture identification software using Frame-based and sEMG, and measured its accuracy and loss value. For sEMG, we used Precision, Recall, and F1-score to check the metrics of each gesture identification. The frame-based results showed good results in both precision and loss values. sEMG showed an improvement in precision and loss values as the time length increased, but there was a tendency to decrease in some indices. In the future, it is necessary to explore the local relationship between finger and forearm to optimize out learning model.
Key words: Multi-modal / prosthetic / control system
© The Authors, published by EDP Sciences, 2022
This is an Open Access article distributed under the terms of the Creative Commons Attribution License 4.0, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Current usage metrics show cumulative count of Article Views (full-text article views including HTML views, PDF and ePub downloads, according to the available data) and Abstracts Views on Vision4Press platform.
Data correspond to usage on the plateform after 2015. The current usage metrics is available 48-96 hours after online publication and is updated daily on week days.
Initial download of the metrics may take a while.