High speed small target detection is a challenging problem for ground-based radar due to its maneuverability and low radar cross section (RCS). The range migration (RM) and Doppler frequency migration (DFM) will occur during the coherent integration period, which makes it difficult to improve the coherent integration ability and radar detection performance. In this study, a novel algorithm based on Keystone transform (KT) and linear canonical transform (LCT) for high speed small target detection with narrowband radar is proposed. Firstly, it employs KT to eliminate RM. Thereafter, the LCT is applied to compensate DFM and realize coherent integration for the target in the LCT domain. Two typical forms of LCT are given for easy realization and good detection performance. Finally, the constant false alarm ratio (CFAR) detector is performed to confirm a target and motion parameters are then estimated. Moreover, in order to realize fast compensation for velocity ambiguity effect, an improved method is proposed based on coarse and fine search. Compared with the generalized Radon Fourier transform (GRFT), the proposed method can acquire a close detection performance but with relatively low computational cost. Simulation results are provided to demonstrate the validity of proposed method.