Multivariate pattern recognition by machine learning methods

In pattern recognition, the goal is to identify patterns and regularities from data with minimal intervention. Finding the patterns and regularities is done using machine learning algorithms. In other words, pattern recognition is the engineering application, while machine learning origin back in computer science. Machine learning algorithms map the input data to the output which these algorithms have two main steps: the training step and the inference step. In the training step, the training data is utilized to learn a model, and in the inference step, the learned model is used to map unseen data to the output space. Machine learning algorithms are divided into four categories from the view of the available training data: (1) supervised learning, (2) semi-supervised learning, (3) weakly supervised learning, and (4) unsupervised learning. In supervised learning, all the available training data are fully annotated (labeled) compared to unsupervised learning, where none of the training data is labeled. In semi-supervised learning, some portion of training data is labeled, and a large portion is unlabeled. If the whole training data is not fully labeled (i.e., partially labeled), then it is weakly supervised learning. In this chapter, we focus on supervised learning.