Integration of deep adaptation transfer learning and online sequential extreme learning machine for cross-person and cross-position activity recognition

• Deep adaptation transfer learning combined with OS-ELM to handle cross domain HAR. • GAP and SE structure in CNN facilitate feature extraction and adapt to input data. • DANN and DDC have different effects on cross-person and cross-position transfer. • OS-ELM classifier improves HAR accuracy with a few annotated data in target domain. Deep learning (DL) has been evolving to a prevalent method in human activity recognition (HAR). However, the performance of wearable sensor based HAR models decline significantly when training data come from different persons or sensor positions, and a time-consuming data annotation is indispensible to cater for the big-data driven DL models. In this paper we proposed a fast and robust hybrid model to handle the transfer issues of wearable sensor based HAR between different persons (cross-person) and different positions (cross-position) with just a few annotated data in target domain. The model consists of three parts: (1) A convolutional neural network (CNN) with global average pooling layer to facilitate the extraction of advanced common features in source domain and target domain; (2) A domain adaptive neural network with a gradient reversal layer (DANN) and deep domain confusion network with an adaptive layer (DDC) to reduce domain shift caused by the change of persons and sensor positions; (3) An adaptive classifier based on online sequential extreme learning machine (OS-ELM) to achieve fast and accurate classification with a few annotated data in target domain. Experimental results on four public datasets verified the superiority of the proposed hybrid model over standard CNN and deep transfer learning models in adapting the classifier to new sensor locations and subjects quickly, where the HAR accuracy can be improved by at least 12% for cross-person transfer and 20% for cross-position transfer, respectively.