Neuromorphic computing with hybrid CNN–Stochastic Reservoir for time series WiFi based human activity recognition

Wi-Fi Channel State Information (CSI) based human activity recognition (HAR) which using channel disturbances caused by signal reflection is a novel way of environment sensing and motion recognition. The collected channels characteristics are heavily influenced by the environment, human activity patterns and subject’s weight and height. These signal variations reflected from body components are mainly affected by static multipath effects comprises random noise and behave differently in individuals, and thus an active field of research. To reach further for achieving automated real-time classification, lower computational cost and easy adaptability to hardware are necessary. In this work, a CSI-based HAR with hybrid framework, Convolutional Neural Network (CNN)-Stochastic Reservoir (SR) (CNN-SR) has been proposed, enabling a subject adaptable and more efficient hardware implementation with minimal computational complexity. A subcarrier correlation matrix is first computed and portrayed in image without preprocessing based on the reflection of the raw CSI signal induced by human activities at regular intervals, allowing visual observation of whole pattern changes. The time-based features are subsequently extracted through CNN and these feature arrays are then feed into SR which based on stochastic spiking neural network (SSNN) in simple cycle reservoir architecture for template matching. SR offers attractive power savings over typical von Neumann systems, by doing stochastic computations. The proposed method has also been demonstrated that is capable for HAR based on partially captured signals. The signal pattern of each segment can be observed in a single sight and then employed for person-to-person template recognition. This enables HAR with minimal computational complexity and solving the inter-person variability concerns. The results demonstrate that the proposed CNN-SR achieves impressive performance in recognizing human activities and surpasses existing models with an average accuracy of 93.49%.