Acoustic waves at microwave frequencies are widely used in wireless communication and are potential information carriers in quantum applications. However, most acoustic devices are passive components, and the development of phononic integrated circuits is limited by the inability to control acoustic waves in a low-loss, scalable manner. Here we report the electrical control of gigahertz travelling acoustic waves at room temperature and millikelvin temperatures. We achieve phase modulation by tuning the elasticity of a lithium niobate acoustic waveguide via the electro-acoustic effect. This phase modulator is then used to build an acoustic frequency shifter based on serrodyne phase modulation, and phase modulators in a Mach–Zehnder interferometer configuration are used to create an electro-acoustic amplitude modulator. By tailoring the phase matching between acoustic and quasi-travelling electric fields, we achieve reconfigurable non-reciprocal modulation with a non-reciprocity of over 40 dB. To illustrate the potential of the approach in quantum applications, we show that our electro-acoustic modulator can provide coherent modulation of single-phonon-level acoustic waves at 50 mK. The electro-acoustic effect can be used to electrically control the phase velocity of travelling acoustic waves in a lithium niobate waveguide, and to construct devices that can modulate the phase, frequency and amplitude of acoustic waves, even at the limit of single phonons.