Multi-camera Collaboration for 3D Visualization via Correlated Information Maximization

A critical component for various interactive visual Internet-of-Things (IoT) applications is to reconstruct three-dimensional (3D) scenes from RGB images, i.e., 3D visualization. When multiple cameras are involved, the visualization outcome mainly depends on the quality of input images, which carry correlated and complementary visual information from different camera perspectives. One main challenge to improve visualization performance is how to efficiently coordinate multiple cameras under complex environmental conditions. To overcome this challenge, we propose a situation-aware multi-camera collaboration scheme based on the maximization of correlated information among different inputs. First, the information gain of a single camera is modelled by quantifying the effect of view direction, resolution and signal-to-noise-ratio (SNR) on image quality. A spherical Gaussian is then designed to model the mutual information among neighbouring viewpoints and further calculate the total correlated information of the camera group by considering their information redundancy and complementarity. An adaptive coarse-to-fine algorithm is proposed to maximize the correlated information, which achieves effective decision-making of optimal multi-camera collaboration strategy, including cameras’ location, direction and focal length configurations. Simulation and realistic experiments demonstrate the accuracy of the correlated information model and the efficacy of the scheme to improve reconstruction quality.