Abstract Fishways are essential for restoring river connectivity and facilitating fish migration, necessitating ongoing research for efficiency optimization. This study developed a fish behavior model to optimize fishway efficiency cost‐effectively. The model simulates swimming trajectories of Schizothorax wangchiachii within a vertical slot fishway incorporating upstream behavior, random behavior, obstacle avoidance, and swimming speed variations. The simulated trajectory exhibits a high consistency with 20 real trajectories, achieving a trajectory overlap rate of 86.7%. This result highlights the model's ability to accurately simulate fish movements, demonstrating its significant potential to revolutionize fishway design.