arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Computer Science · Human-Computer Interaction

Gesture Recognition in Indonesian Sign Language Using Hybrid Deep Learning Models

Muhammad Yusuf Daffa Izzalhaqqi, Wahyono Wahyono

See on the map

Abstract

Sign language comprises unique hand gestures used to communicate between the hearing and hearing-impaired communities, with variations based on regions or countries. Due to the increasing global prevalence of hearing loss, there is a growing need for efficient sign language recognition systems. This paper presents the development of a hybrid CNN-LSTM model specifically designed to recognize static alphabetical gestures in Indonesian Sign Language (BISINDO). Primary data was collected to address the issue of publicly available dataset. These self-collected images underwent augmentation and preprocessing stages prior to model training. The architecture and hyperparameter configuration of the hybrid model were fine-tuned using the Randomized Search CV method. Performance analysis was conducted on both the hybrid model, its constituent models, and state-of-the-art model. The experimental results revealed remarkable performance by the hybrid model with excellent accuracies of 99.60%, 84.87%, and 98.00% on the training, validation, and testing sets respectively, along with a macro average score of 0.98, indicating high precision, recall, and F1-score across all the classes. However, it was observed that the VGG-16 and CNN exhibited slightly superior performance compared to the hybrid model, while the LSTM demonstrated the least favorable performance among the models.

More like this