A Two-Level Noise-Tolerant Model for Relation Extraction with Reinforcement Learning

Distant supervision has been widely used to automatically label data for relation extraction, but inevitably suffers from wrong labeling problems. Existing methods solve the noisy problem by merely focusing on one aspect, either at the sentence-level or the bag-level. However, none consider the two levels as a whole. In this paper, we propose a deep reinforcement learning model to solve the noisy problem at both the bag level and the sentence level. For a bag, i.e., a set of sentences containing the same pair of entities, the sentence-level extractor serves as an agent which predicts the label for each sentence, and then determines the label for the bag. The bag-level extractor provides a delayed reward to the agent, and iteratively promotes its performance. The experimental results show that our two-level denoising model effectively improves the performance of distant supervision relation extraction compared to previous methods.