Bidirectional transport in cilia is carried out by polymers of the IFTA and IFTB protein complexes, called anterograde and retrograde intraflagellar transport (IFT) trains. Anterograde trains deliver cargoes from the cell to the cilium tip, then convert into retrograde trains for cargo export. We set out to understand how the IFT complexes can perform these two directly opposing roles before and after conversion. We use cryoelectron tomography and in situ cross-linking mass spectrometry to determine the structure of retrograde IFT trains and compare it with the known structure of anterograde trains. The retrograde train is a 2-fold symmetric polymer organized around a central thread of IFTA complexes. We conclude that anterograde-to-retrograde remodeling involves global rearrangements of the IFTA/B complexes and requires complete disassembly of the anterograde train. Finally, we describe how conformational changes to cargo-binding sites facilitate unidirectional cargo transport in a bidirectional system.