Enhancing Tamil Sign Language Recognition In Video Data Using Deep Learning

Abstract

Tamil Sign Language (TSL) gratitude using video datasets is a critical step in connecting letter fences for the unhearing community by identifying sign language gestures using automated computer vision systems. Recognizing and classifying gestures in a real-time video context is a challenging task due to the variability of signs, variability of signers, background variability, and altering illumination. The inherent spatial and temporal dependencies and combinations are difficult for traditional methods to capitalize on. To address these challenges, the Deep Convolve Spatial Layer–Gated Recurrent Neuronet (DCSL-GRN) model is proposed, effectively apprehending both spatial and temporal dependencies in video sequences. The DCSL-GRN model integrates convolutional layers for spatial feature extraction and Gated Recurrent Units (GRUs) for temporal modeling, significantly enhancing the accuracy and robustness of TSL gesture recognition. The Tamil Sign Language video dataset from Kaggle is used in the research. Sobel edge detection enhances image contrast by emphasizing key edges, while GrabCut segmentation isolates gesture regions accurately. Fourier Transform extracts frequency-based features, and Gaussian augmentation increases data diversity for better generalization. With a classified accuracy of 98%, precision of 97%, recall of 98%, and F1-score of 97%, experimental findings validate the model's resilience to failure that show its exceptional performance. A Graphical User Interface (GUI) was created to display Tamil sign letter outputs as combined text. Moreover, the use of Retrieval-Augmented Generation (RAG) allows for contextual awareness and increased accuracy and interpretability of the outputs. An implemented method using a Python tool, the suggested approach offers an accurate, accessible answer for real-time credit and translation of Tamil Sign Language.