A Depth Video Dataset and Neural Framework for Real-Time Handwriting Character Generation and Guidance

¹ Student, Department of Electronics Engineering, Madras Institute of Technology Campus, Anna University Chennai, Tamil Nadu, India.
² Assistant Professor(Sl. Gr.), Department of Electronics Engineering, Madras Institute of Technology Campus, Anna University Chennai, Tamil Nadu, India.
³ Research Scholar, Department of Electronics Engineering, Madras Institute of Technology Campus, Anna University Chennai, Tamil Nadu, India.

^* Corresponding author: This email address is being protected from spambots. You need JavaScript enabled to view it.

Abstract

The acquisition of handwriting requires movement in space and time to be monitored. This research introduces a depth-sensing system which registers and recognizes the handwritten signs through the analysis of the 3D movement. By using the Intel RealSense D405 camera, we computed a new dataset to record fine fixed hand and pen trajectories for each of the 26 English alphabetic characters. Each recording was converted to sequences of standardized point-cloud data and annotated with information on action label: pen_down, pen_lift, hand_position. An LSTM network has been used to train this temporal 3-D sequences with an accuracy of 82%, precision of 0.78, a recall of 0.82 and an F1 score of 0.79. By spatiotemporally transcending 2D trajectories, this methodology captures spatial as well as kinematic cues, which features important subtle dynamics of handwriting, which can be useful for real time feedback, educational applications, and assistive technologies. Potential research will extend the data target and explore transformer-based models with the aim of making it better than the current performance words.