Relationship existence recognition-based social group detection in urban public spaces

In urban public spaces, a social group consists of two or more individuals who share some social relationships and interact based on mutual expectations. However, most existing studies found people's F-formations on a top view, which is hard to observe their social contexts and the top-view videos are not easily accessible in real urban life. Recently, some researchers turned to urban scenes and analysed front-view human behaviours for social group detection. But these methods still cannot grasp the nature of social groups, i.e., the relationships among individuals. It is the key to finding social groups to judge whether any two individuals belong to the same cluster. Therefore, this paper proposes a new paradigm: relationship existence recognition-based social group detection. Additionally, on top of the paradigm, we designed a new social group detection algorithm incorporated with the visual cue-based and non-visual cue-based components. Specifically, the former exploits the spatial interactions and the temporal information to recognise the existence of social relationships through supervised deep learning. The latter estimates the similarities of trajectory pairs using the unsupervised spatial–temporal position information. Social group detection achieves superior accuracy with the two components' complementary results. On Social-CAD (Social Collective Activity Dataset) and PLPS (Public Life in Public Space) datasets, extensive experiments demonstrate that our algorithm outperforms the state-of-the-art (SOTA) methods.