Can neural networks learn finite elements?

The aim of this note is to construct a neural network for which the linear finite element approximation of a simple one dimensional boundary value problem is a minimum of the cost function to find out if the neural network is able to reproduce the finite element approximation. The deepest goal is to shed some light on the problems one encounters when trying to use neural networks to approximate partial differential equations.