When the light field of a scene is generated with a finite number of subviews, the defocused regions would appear to be split-up if a camera is used to capture the light field. Yet, the split-up effect is unnoticeable when the light field is viewed directly through a human eye. In this paper, we attribute the unobservability of the split-up effect to the decrease in visual acuity as a function of retinal eccentricity and to the low-pass filtering property of visual attention. Theoretical and experimental results are provided to support our claim. Furthermore, we set an observability criterion for the split-up effect and discuss design strategies for performance improvement of light field displays.