Perceived congruence between auditory and visual elements in multimedia

Abstract Multimedia productions often combine motion pictures and sound. A vital element in making visual media presentations more impressive is ‘perceived congruence’ between sound and moving pictures. This chapter introduces a series of empirical studies on perceived congruence between sound and moving pictures and discusses how to make congruent audiovisual combinations. Perceived congruence between sound and moving pictures is created by the matching of auditory and visual temporal structures, and by the similarity between auditory and visual affective impressions. Further, the similarity of imagery between changing (or dynamic) patterns of auditory and visual signals creates perceived congruence. These findings enhance our understanding of how people process audiovisual materials in television programs, films, and electronic multimedia such as video games. The practical implications of these findings are relevant for producers, sound designers, and film-music composers in assisting them to create more effective and impressive audiovisual works.