Privacy-Preserving Image Retrieval with Multi-Modal Query

Abstract The ever-growing multi-modal images pose great challenges to local image storage and retrieval systems. Cloud computing provides a solution to large-scale image data storage but suffers from privacy issues and lacks the support for multi-modal image retrieval. To address these, a searchable encryption-empowered privacy-preserving multi-modal image retrieval method is proposed. First, we design a hybrid image retrieval framework that fuses visual features and textual features at a decision level and further supports similar image retrieval and multi-keyword image retrieval. Second, we construct a new hybrid inverted index structure to distinguish high-frequency terms from low-frequency terms and index them through hierarchical index trees and data blocks, respectively, which greatly improves query efficiency. Third, we design a prime encoding-based multi-keyword query method that converts mapping operations in bloom filters into inner product calculations, and further implements secure multi-keyword image query. Experiments against the Baseline schemes are conducted to verify the performance of the scheme in terms of high efficiency.