Dynamic Letter Recognition Utilizing Motion Analysis and Voice Based Error Feedback

¹ Student, Department of Electronics Engineering, Madras Institute of Technology Campus, Anna University Chennai, Tamil Nadu, India.
² Student, Department of Electronics Engineering, Madras Institute of Technology Campus, Anna University Chennai, Tamil Nadu, India.
³ Student, Department of Electronics Engineering, Madras Institute of Technology Campus, Anna University Chennai, Tamil Nadu, India.
⁴ Assistant Professor(Sl. Gr.), Department of Electronics Engineering, Madras Institute of Technology Campus, Anna University Chennai, Tamil Nadu, India.

^* Corresponding author: This email address is being protected from spambots. You need JavaScript enabled to view it.

Abstract

This paper presents a dynamic letter recognition system that utilizes motion based analysis and real time auditory feedback to enhance handwriting recognition accuracy, accessibility, and learner engagement. The suggested framework uses a combination of a hybrid Bidirectional Long Short Term Memory (BiLSTM) and Multi Layer Perceptron (MLP) model to find alphanumeric characters (A-Z, 0-9) from motion based stroke patterns that the Intel RealSense D405 depth camera picks up. The handwritten motion dataset is in JSON format and has both temporal stroke sequences and geometric data. It was gathered with a custom Streamlit interface and marked up with the Computer Vision Annotation Tool (CVAT). Since the method examines both the spatial and temporal aspects of handwriting dynamics instead of fixed image, it has been successfully applied across various writing forms.The TTS based modules providing real time audio feedback to students during interactive and corrective processes increase the students ability to correct their identification mistakes rapidly through real time reminders. Experimental results demonstrated a 96 percent classification rate, demonstrating that this system is reliable and capable of adapting to a wide variety of applications, which are suitable as an affordable and flexible solution for assistive education and communication for individuals with disabilities.