A Pipeline-Based Multimodal Military Event Argument Extraction Framework

This paper presents a winning solution for the CCKS2022 Competition of Event Argument Extraction from Open Source Multimodal Military Equipment Data. The task is to match the textual information extracted from the text corpus with the visual information from images for event argument extraction. For this aim, we introduce a pipeline-based multimodal information extraction framework consisted of three models. The first one is a global pointer model which extracts entity information from the given text corpus, the second one is a yolo model which detects the bounding boxes of entities in the images and the last one is a multimodal matcher which matches the textual information with the visual information of entities for event argument extraction. With our pipeline-based multimodal information extraction framework, we have achieved the first place in the CCKS-2022 competition with a F1-score of 54.15%.