Investigating the effects of a virtual reality vs. screen-based testing setup on incidental memory after visual search through scenes

Many experiments investigating visual search are performed on a computer screen under highly controlled settings. However, the generalizability of the investigated search mechanisms to the real world is unclear. Here, we tested whether the formation of incidental object memories during visual search follows similar patterns in virtual reality (VR) and on a computer screen. We present two studies that were identical in the administered tasks, but differed in the setup they were tested in. In both experiments, participants searched for ten out of 20 objects in indoor scenes, either in full illumination or with constrained visual input via a controller-contingent 8-degree window (flashlight condition). After the search task, participants’ incidental memory was tested with two surprise tasks: an object recognition and a location memory task (scene rebuilding). Critically, the first experiment took place in an immersive virtual environment, while the second experiment displayed the same interactive 3D environment on a computer screen. In VR, we found that participants relied more on memory when searching through scenes with a flashlight: Search times decreased more strongly over trials and non-targets were recognized better than ones from illuminated scenes. Both results indicate stronger memorization of objects when only limited visual input was available. Additionally, participants were able to reallocate target and non-target objects from the flashlight condition just as accurately as from illuminated scenes, despite never having seen them in full view, indicating the formation of a similarly holistic scene representation in both conditions. In contrast, when administered on a computer screen, preliminary analyses suggest no benefit for object recognition in the flashlight condition, and a detrimental effect on location memory. Our results indicate that visual search on a computer screen elicits weaker scene memory than in VR. Sensorimotor signals engaged in the search process might play a role for strengthening scene memory.