Abstract This paper addresses the problem of identifying optimal portfolio parameters in nonsparse and sparse models. Generally, using the sample estimates to construct a mean–variance portfolio often leads to undesirable portfolio performance. We propose a novel bi‐level programming framework to identify the optimal values of expected return and cardinality, which can be estimated separately or simultaneously. In the general formulation of our approach, outer‐level is designed to maximize the utility of the portfolio, which is measured by Sharpe ratio, while the inner‐level is to minimize the risk of a portfolio under a given expected return. Considering the nonconvex and nonsmooth characteristics of the outer‐level, we develop a hybrid derivative‐free optimization algorithm embedded with alternating direction method of multipliers to solve the problem. Numerical experiments are carried out based on both simulated and real‐life data. During the process, we give a prior range of cardinality using the data‐driven method to promote the efficiency. Estimating the parameters by our approach achieves better performance both in the stock and fund‐of‐funds markets. Moreover, we also demonstrate that our results are robust when the risk is measured by conditional value‐at‐risk.