Recent advancements in molecular simulations highlight the substantial computational demands of generating high-precision quantum mechanical labels for training neural network potentials. These challenges emphasize the need for improvements in delta-machine learning techniques. The Equivariant Graph Neural Network (EGNN) framework, grounded in a message-passing mechanism that preserves structural equivariance, enables refined atomic representations through interaction-driven updates. We introduce the Δ-EGNN model, which achieves high prediction accuracy for both molecular and condensed-phase systems. For example, in periodic water box systems, a mean absolute error of 1.722 meV/atom for energy (global property) and 0.0027 e for partial charge (local property) were achieved with training on just 800 labels. Δ-EGNN is computationally efficient, achieving speedups of 1-2 orders of magnitude compared to conventional methods at the MP2 level. In contrast to models directly trained on total energies, such as NequIP, MACE, and Allegro, the Δ-EGNN model employs delta-machine learning to predict the difference between energies derived from low- and high-level electronic structure methods, providing a significant advantage in reducing computational costs while preserving high accuracy. In summary, Δ-EGNN opens a new avenue for exploring energy landscapes and constructing machine learning potentials with afforable computational overhead, facilitating routine quantum mechanical calculations for complex molecular systems.