计算机科学
关系抽取
自然语言处理
语义学(计算机科学)
人工智能
情态动词
关系(数据库)
知识抽取
信息抽取
情报检索
数据挖掘
化学
高分子化学
程序设计语言
作者
Junhao Feng,Guohua Wang,Changmeng Zheng,Yi Cai,Ze Fu,Yaowei Wang,Xiaoyong Wei,Qing Li
出处
期刊:IEEE Transactions on Circuits and Systems for Video Technology
[Institute of Electrical and Electronics Engineers]
日期:2023-06-09
卷期号:34 (1): 561-575
被引量:2
标识
DOI:10.1109/tcsvt.2023.3284474
摘要
In natural language processing, relation extraction (RE) is to detect and classify the semantic relationship of two given entities within a sentence. Previous RE methods consider only the textual contents and suffer performance decline in social media when texts lack contexts. Incorporating text-related visual information can supplement the missing semantics for relation extraction in social media posts. However, textual relations are usually abstract and of high-level semantics, which causes the semantic gap between visual contents and textual expressions. In this paper, we propose RECK - a neural network for relation extraction with cross-modal knowledge representations. Different from previous multimodal methods training a common subspace for all modalities, we bridge the semantic gaps by explicitly selecting knowledge paths from external knowledge through the cross-modal object-entity pairs. We further extend the paths into a knowledge graph, and adopt a graph attention network to capture the multi-grained relevant concepts which can provide higher level and key semantics information from external knowledge. Besides, we employ a cross-modal attention mechanism to align and fuse the multimodal information. Experimental results on a multimodal RE dataset show that our model achieves new state-of-the-art performance with knowledge evidence 1 .
科研通智能强力驱动
Strongly Powered by AbleSci AI