Handwritten Mathematical Expression Recognition (HMER) is essential to online education and scientific research. However, discerning the length and characters of handwritten mathematical expressions remains a formidable challenge. In this study, we propose an encoder–decoder architecture incorporating a novel Adaptive momentum-based Regularized Gated Recurrent Unit (AdamR-GRU) to solve this problem. It incorporates a novel gate within the GRU framework to maintain input histories, enabling extended memory retention and faster convergence. To assess the performance of AdamR-GRU, we conducted a series of comprehensive experiments on the CROHME 2014, 2016, and 2019 datasets. The results indicate that the AdamR-GRU model exhibits competitive performance with existing state-of-the-art methods regarding Expression Recognition Rate (exprate). Moreover, our model achieves state-of-the-art accuracy concerning 1, 2, and 3 symbol-level errors (top 1, 2, 3 error accuracy) on the CROHME 2014 and 2016 datasets. For the CROHME 2019 dataset, the model attains state-of-the-art performance in top 2 and 3 error accuracy. AdamR-GRU demonstrates reduced computational complexity and memory usage compared to alternative contemporary models.