Isolated Sign Language Recognition with Gloss-to-Text Smoothing for Assistive Translation
Luan Fernandes de Franca, JosÃƒÂ© Everardo Bessa Maia
Journal of Digital Information Management, Vol. 24, No. 2, 2026.
DOI: https://doi.org/10.6025/jdim/2026/24/2/67-84
Link: https://www.dline.info/fpaper/jdim/v24i2/jdimv24i2_1.pdf
Abstract:
This work presents a modular two layer architecture for recognizing isolated sign language video and
converting gloss sequences into fluent natural language text. The proposed pipeline comprises: (1) glosslevel
sign recognition using spatiotemporal feature extraction (I3D, EfficientNet, MobileNet) with lightweight
classifiers, and (2) natural language smoothing via large language models (LLMs) using prompt engineering.
Unlike prior claims of continuous translation, this paper explicitly focuses on isolated signs as a practical
building block for assistive systems where segmentation is either manually provided or handled by external
modules. Comprehensive evaluations on Brazilian Sign Language (Libras) datasets demonstrate high accuracy
in isolated sign classification (F1 > 0.97 for I3D-RGB + LR). A critical analysis of near perfect AUC scores
reveals dataset limitations that are openly discussed. For gloss to text conversion, we evaluate LLM smoothing
on both clean and noise injected gloss sequences and report BLEU scores under realistic conditions. The
proposed architecture is scalable, modular, and adaptable to other sign languages, advancing accessibility
for deaf and hard of hearing communities.