过度拟合
特征选择
铰链损耗
人工智能
加权
计算机科学
超平面
子空间拓扑
模式识别(心理学)
特征(语言学)
梯度下降
规范(哲学)
肯定性
算法
数学优化
数学
支持向量机
正定矩阵
人工神经网络
医学
物理
放射科
哲学
量子力学
语言学
特征向量
政治学
法学
几何学
作者
Jingyu Wang,Hongmei Wang,Feiping Nie,Xuelong Li
出处
期刊:Neurocomputing
[Elsevier]
日期:2023-04-29
卷期号:543: 126268-126268
被引量:12
标识
DOI:10.1016/j.neucom.2023.126268
摘要
Feature selection can help to reduce data redundancy and improve algorithm performance in actual tasks. Most of the embedded feature selection models are constructed based on square loss and hinge loss. However, these models based on the square loss cannot directly evaluate the discriminability of the samples in the feature subspace, and these methods based on the hinge loss are difficult to solve due to their complex objective functions. To deal with these problems, a Feature Selection method with Multi-class Logistic Regression (FSMLR) is proposed in this paper. Firstly, we construct a linear function to measure the difference between the distance from samples to their regression hyperplane and the distance from these samples to regression hyperplanes of other classes, which could be used to strengthen the discriminant property of the embedded model. Then, we design a re-weighting matrix with a ℓ2,0-norm sparse condition as well as a discrete condition, which is used to select features in the subspace. Considering that it is difficult to solve the re-weighting matrix with the discrete and sparse conditions in an optimization problem, we relax these two conditions and present a feature selection model via a re-weighted multi-class logistic regression with the two relaxed constraints. Finally, we add the F-norm regularization in our model to avoid overfitting, and its unconstrained equivalent transformation with ℓ2,p-norm regularization is derived to explore the function of the re-weighting matrix. The gradient descent algorithm could be used to solve the FSMLR. Especially, when the regularization term in the equivalence problem is set to ℓ2,1-norm, the global optimal solution can be obtained. Extensive experiments on multiple public data sets prove that FSMLR outperforms other competitors.
科研通智能强力驱动
Strongly Powered by AbleSci AI