The complex compositional space of high entropy alloys (HEAs) has shown a great potential to reduce the cost and further increase the catalytic activity for hydrogen evolution reaction (HER) by compositional optimization. Without uncovering the specifics of the HER mechanism on a given HEA surface, it is unfeasible to apply compositional modifications to enhance the performance and save costs. In this work, a combination of density functional theory and Bayesian machine learning is used to demonstrate the unique catalytic mechanism of IrPdPtRhRu HEA catalysts for HER. At high coverage of underpotential-deposited hydrogen, a d-band investigation of the active sites of the HEA surface is conducted to elucidate the superior catalytic performance through electronic interactions between elements. At low coverage, a novel Bayesian learning with oversampling approach is then outlined to optimize the HEA composition for performance improvement and cost reduction. This approach proves more efficacious and efficient and yields higher-quality structures with less training set bias compared with neural-network optimization. The proposed HEA optimization theoretically outperforms benchmark Pt catalysts' overpotential by ≈40% at a 15% reduced synthesis cost comparing to the equiatomic ratio HEA.