Improving Online Handwriting Trajectory Reconstruction Based on Temporal Convolutional Networks

Online handwriting trajectory acquisition enables a broad spectrum of educational applications by providing relevant and individualized feedback during the learning process. Acquiring handwriting with Inertial Measurement Units (IMUs) is a natural choice due to the compact form factor of this kind of sensor, allowing to integrate them into small size microcontroller-based devices like ball point pens, without compromising their ergonomics. Existing approaches have successfully utilized Temporal Convolutional Networks (TCNs) for trajectory reconstruction, achieving notable results. However, it is still a challenging task to achieve precise handwriting reconstruction due to unevenly sampled ground truth data in publicly available datasets and alignment errors of both the training and ground truth data. In order to address these issues, we introduce a new preprocessing pipeline that incorporates longer valid training sequences and employs spline interpolation for more accurate ground truth data representation. We demonstrate that downsampling the training data to 50 Hz leads to better reconstruction accuracy compared to the baseline while extending the effective receptive field of the TCN. Consequently, we evaluate different sensor configurations, showing that a minimal setup with one accelerometer, gyroscope, and writing force sensor can achieve results comparable to more complex configurations. Additionally, we provide insights into the interpretation of Fréchet distance metrics for assessing handwriting reconstruction quality.