The increasing use of high-throughput density-functional theory (DFT) calculations in the computational design and optimization of materials requires the availability of a comprehensive set of soft and transferable pseudopotentials. Here we present design criteria and testing results for a new open-source “GBRV” ultrasoft pseudopotential library that has been optimized for use in high-throughput DFT calculations. We benchmark the GBRV potentials, as well as two other pseudopotential sets available in the literature, to all-electron calculations in order to validate their accuracy. The results allow us to draw conclusions about the accuracy of modern pseudopotentials in a variety of chemical environments.