Working Memory Capacity for Gesture-Command Associations in Gestural Interaction

The study addressed the working memory capacity (WMC) of gesture-command associations in gestural interaction and investigated the impact of compatibility between users’ mental models and the predefined gesture-command associations on WMC. Gestural interaction is a popular representative of natural interactions. Although gestural interaction intends to be natural, it has been criticized for not being so. One of the critical problems lies in learning and memorizing. WMC is a pivotal bottleneck that underlies learning and memorizing gesture-command associations, yet it remains unknown. Two standardized paradigms were used to estimate the WMC of gesture-command associations: change-detection task in Experiment 1, and span task in Experiment 2. Besides, we further examined the impact of compatibility on WMC. We found that two gesture-command associations can be retained in working memory under low-compatibility conditions, while three to five associations can be retained under high-compatibility conditions. The result implies that WMC of gesture-command associations is highly limited, while this cognitive limitation could be reduced by promoting the compatibility between users’ mental model and the predefined gesture-command associations. Designers for gestural interactions may require users to memorize two to five gesture-command associations, considering different application scenarios.