Owing to their capabilities of solid-state conversion between heat and electricity, zero-emission, and high flexibility, flexible thermoelectric devices (F-TEDs) have exhibited great application possibilities for both portable power generation and localized refrigeration. However, with the rapid development of thermoelectric science and technology, there is still a lack of comprehensive review on the rational design of F-TEDs from the fundamentals to structures, which critically determines the performance and conformality of F-TEDs. To address this issue, here, we timely overview the latest progress on the up-to-the-date F-TEDs with their unique designs. We carefully summarize the structure-related principles and factors that determine the performance of F-TEDs and the advanced strategies for improving their utilities. Besides, we focus on the timeliest designs for the inorganic-based devices, organic-based devices, and hybrid-based devices targeting both power generation and refrigeration. In the end, we point out the current challenges, controversies, and prospects of F-TEDs.