The flexible manipulator is widely used in robots, aerospace, medical, manufacturing and other fields due to its light weight, fast response, high adaptability, low energy consumption and high load to weight ratio. However, the flexible manipulator is prone to vibration, and it is difficult to achieve high-speed and high-precision motion control. In this paper, the dynamic model of the rotary flexible manipulator with piezoelectric actuators is established based on Lagrange equation. The optimal configuration criterion and the optimal solution via genetic algorithm are designed for the piezoelectric actuators on the rotary flexible manipulator. The flexible manipulator is decomposed into fast and slow subsystems by singular perturbation principle. A two-time scale composite control strategy with an adaptive robust controller in the slow subsystem and a LQR controller in the fast subsystem is proposed. Experiments show that the flexible manipulator can achieve both good trajectory tracking and vibration suppression under the joint drive of motor and piezoelectric actuators.