High field superconducting magnets are subjected to a large Lorenz force during operation. The main superconductors for high field applications like Nb3Sn are strain sensitive, therefore it is essential to measure the strain in the superconducting (SC) coils during the testing process. Compared with the traditional way by using the Resistance Strain Sensor (RSS), the fiber optic sensor based on the Fiber Bragg grating (FBG) method has significant advantages in high field applications. The FBG is an anti-electromagnetic interference device, which makes it possible to measure the local strain inside the superconducting coils. Meanwhile, it is sensitive only to the strain at a constant low temperature (4.2 K). In this study, several FBGs and RSSs were impregnated in SC coils of a 12 Tesla dipole magnet, and FBGs were attached to the outer surfaces of the aluminum shell and rods. The test was performed in the whole process from room temperature assembly of the magnet to the excitation at 4.2 K. The experimental results show that FBGs can continually monitor the strain of the magnet from a macro perspective, as well as reflect the internal strain state of the SC coils during the whole process. Compared with RSSs, FBGs have a better performance in strain measurement during the excitation due to the anti-electromagnetic interference. Strain measurement inside the magnet can offer more useful information about the strain distribution of the magnet and the origin of the quench, which may help developers improve the design and fabrication of SC magnets.