Class-agnostic counting and localization with feature augmentation and scale-adaptive aggregation

Compared to traditional class-specific counting, class-agnostic counting (CAC) enables the counting of arbitrary novel categories with only a few exemplars. The core issue of CAC lies in feature matching between the query image and exemplars. However, the scarce user-given exemplars make it difficult to adequately cover the diverse appearance of target objects, which poses challenges for feature matching. In this paper, we propose a Class-Agnostic Counting and Localization model (CACL) to enrich the exemplars by mining more inherent instances from the query feature. Specifically, CACL consists mainly of Self-Augment Feature Matching (SAFM) and Scale-Adaptive Spatial Aggregation (SASA). SAFM collects features of target instances frequently occurring in the query feature to augment the limited user-given exemplar, thus producing more comprehensive semantic features of the target category. To improve the robustness against scale variation, we design the SASA to enable the model to aggregate multiscale features adaptively. Compared to previous density map-based approaches, our method can locate the target objects which is more practical in real-world applications. Extensive experiments on FSC-147 suggest that our method achieves a competitive counting performance compared to state-of-the-art methods. Moreover, cross-dataset experiments demonstrate superior generalization in class-specific counting tasks.