We consider a class of parametric Schr\"odinger equations driven by the fractional $p$-Laplacian operator and involving continuous positive potentials and nonlinearities with subcritical or critical growth. By using variational methods and Ljusternik-Schnirelmann theory, we study the existence, multiplicity and concentration of positive solutions for small values of the parameter.