Using Semantic Clustering And Autoencoders For Detecting Novelty In Corpora Of Short Texts

Semantic analysis of text corpora is of broad utility, including for data from conversations, on-line chats, brainstorming sessions, comments on blogs, etc. - all of which are potentially interesting sources of information and ideas. In the present paper, we look at data from a large group brainstorming experiment that generated thousands of mostly brief statements. The ultimate goal is to detect which statements are semantically atypical within the overall corpus. In contexts such as spam detection or detection of on-line intrusions, autoencoders have been used successfully to separate typical from atypical data, and we consider this approach in the present paper. Texts are embedded in a semantic space obtained through topic analysis, and an autoencoder network is used to reconstruct each embedded text. The results show that, while difficulty of reconstruction is related to quantitative measures of atypicality in the embedding vector space, it is not well correlated with novelty assignments made by a human rater. However, this is not the case when the data is first clustered in the embedding space: The reconstruction error for each data cluster indicates that some clusters represent more novel data than others, and that the inverse size of the cluster and the mean reconstruction error of the texts in the cluster capture this well. In particular, autoencoders that enforce dimensionality reduction improve discrimination. The results also show that, in the reconstruction process, the autoencoder implicitly discovers the same clusters in the data that are discovered explicitly by an optimized k-means approach.