Recognizing musical melodies: Effects of audio and visual representations of emotion

Memory for music was tested with a set of emotionally valenced melodies encoded with and without an accompanying visual scene. The music consisted of original melodies that were normed for their emotional valence. During the training phase, participants were presented a set of happy, neutral, and sad musical melodies. Half of the melodies were presented with an affective, visual stimulus, depicting a scene consistent with emotional expression of the melody. After a distraction period, participants were tested in an audio-only recognition task. The recognition task included a mix of melodies from the training phase (with equal representation of audiovisual and audio conditions and each of the affective conditions) and emotionally-matched novel solo guitar melodies. Using signal detection analyses participants were found to be more sensitive to stimuli for which they had been trained in the Auditory-only conditions than in the Audiovisual conditions. Sensitivity also tended to be highest with the neutral affective melodies and lowest with the sad melodies. These findings show the potential for distraction with visual imagery to musical memory formation—despite the consistency of that emotional content to the melody. Moreover, less emotionally valenced music tended to be more effectively encoded.