Finger Spelling Gesture Recognition for Nepali Sign Language Using Hybrid Classical Quantum Deep Learning Model

Abstract

Sign language stands as the principal means of communication for individuals who are deaf or mute. The communication dynamics between verbally challenged individuals and those without such impairments have consistently presented barriers. The breakthrough of sign language recognition indicates a significant advancement in this field, with the precise and affordable commercialization of such recognition systems becoming a pressing global research challenge. Neural network-based sign language recognition systems, particularly those employing image processing, have been favored over traditional gadget methods, due to their enhanced accuracy and simplicity. This project endeavors to craft a highly precise and user-friendly sign language recognition system that leverages neural networks to convert input gestures into text and speech. Focused on the detection of Nepali Sign Language (NSL) gestures as endorsed by the NDFN, the project's fundamental goal is to foster unmediated communication between differently abled individuals and those unfamiliar with sign language, thus eliminating the need for a translator. In our research, we have introduced a hybrid classical-quantum deep learning model specifically designed to detect and categorize six distinct NSL gestures. Utilizing VGG16, we have extracted image features, then reduced these features using transformation. These reduced features have been subsequently embedded on a Variational Quantum Circuit (VQC), composed of various quantum gates that possess superposition and entanglement characteristics. The results of this project underlined a marked enhancement in the VGG-16 architecture's performance, owed to the integration of a hybrid classical-quantum neural network. This enhancement represents a promising step towards more effective and accessible communication solutions for the deaf and mute community.