The frequency doubling and tripling in a Q-switched all-fiber laser are studied. It is demonstrated that the main limitations on the efficiency of the harmonic generation are related to the random polarization that is nonuniform with respect to axes, the asymmetric pulse shape with a flat trailing edge, and the significant spectral broadening of the multimode radiation of the fiber master oscillator in the fiber amplifier. The methods to increase the efficiency are proposed. For an IR pulse energy of about 0.3 mJ, duration of about 40 ns, and repetition rate of 1 kHz, the second- and third-harmonic pulse energies are greater than 60 and about 10 μJ, respectively.