We describe a terahertz imaging system that uses a single pixel detector in combination with a series of random masks to enable high-speed image acquisition. The image formation is based on the theory of compressed sensing, which permits the reconstruction of a N-by-N pixel image using much fewer than N2 measurements. This approach eliminates the need for raster scanning of the object or the terahertz beam, while maintaining the high sensitivity of a single-element detector. We demonstrate the concept using a pulsed terahertz time-domain system and show the reconstruction of both amplitude and phase-contrast images. The idea of compressed sensing is quite general and could also be implemented with a continuous-wave terahertz source.