适配器(计算)
残余物
透视图(图形)
计算机科学
心理学
人工智能
计算机硬件
算法
作者
Chunlei Wu,Qinfu Xu,Yiwei Wei,Shaozu Yuan,Jie Wu,Leiquan Wang
标识
DOI:10.1016/j.knosys.2024.111790
摘要
Visual emotion analysis (VEA), a crucial task that identifies the emotional content conveyed in an image, has garnered increasing attention due to the rapid growth of image posts in modern social media. Prompted by the remarkable generalization ability of prompt learning, recent studies have endeavored to develop adaptive contextual prompts to adjust the representation spaces in visual emotion analysis dynamically. However, the majority of these studies have focused on non-oriented prompting strategies, overlooking the sentiment information associated with emotion labels. In this paper, we propose a novel approach, Multi-Perspective Prompt Learning (MPP-CLIP), within the context of CLIP, for visual emotion analysis. Our approach not only involves non-oriented prompting using learnable contexts but also leverages emotion-oriented prompting based on emotion labels. Thus, our design facilitates a strong coupling between the vision-emotion features, ensuring mutual synergy and discouraging the adoption of solely non-oriented solutions. In addition, we introduce Residual-Enhanced Adapter, which employs residual-style feature mixing to achieve efficient transfer learning from pre-trained CLIP to visual emotion analysis. Extensive experimental evaluations demonstrate our approach outperforms previous state-of-the-art (SOTA) methods while maintaining significantly lower computational and memory costs.
科研通智能强力驱动
Strongly Powered by AbleSci AI