Pre-sensor computing with compact multilayer optical neural network

Moving computation units closer to sensors is becoming a promising approach to addressing bottlenecks in computing speed, power consumption, and data storage. Pre-sensor computing with optical neural networks (ONNs) allows extensive processing. However, the lack of nonlinear activation and dependence on laser input limits the computational capacity, practicality, and scalability. A compact and passive multilayer ONN (MONN) is proposed, which has two convolution layers and an inserted nonlinear layer, performing pre-sensor computations with designed passive masks and a quantum dot film for incoherent light. MONN has an optical length as short as 5 millimeters, two orders of magnitude smaller than state-of-the-art lens-based ONNs. MONN outperforms linear single-layer ONN across various vision tasks, off-loading up to 95% of computationally expensive operations into optics from electronics. Motivated by MONN, a paradigm is emerging for mobile vision, fulfilling the demands for practicality, miniaturization, and low power consumption.