In the data-driven finite element analysis proposed by Kirchdoerfer and Ortiz (2016) the material data are a direct part of the optimization problem. The mechanical problem is rewritten as a minimization problem of a distance function subject to the conservation laws. For every material point of the finite element geometry nearest neighbor searches need to be performed to find the closest data points describing equilibrium. With increasing dimensions and data density the numerical cost increases significantly. The fact that detailed data are required for good results and that three dimensional examples cannot always be simplified leads to a high computation time and memory demand. Here a multi-level method is proposed to reduce the computational cost of such analyses. Starting point is a coarser initial subset of the data, a first solution is then approximated and successively improved by adaptively refined data sets. Repeated simulations lead to an adequate solution using just a fraction of the total data set. While introducing the method on a simple truss structure example it is shown that this multi-level method significantly reduces computational costs and enables complex data-driven finite element computations by means of an engineering example.