Background Diffusion weighted imaging (DWI) at multiple b‐values has been used to predict the pathological complete response (pCR) to neoadjuvant chemoradiotherapy for locally advanced rectal cancer. Non‐Gaussian models fit the signal decay of diffusion by several physical values from different approaches of approximation. Purpose To develop a deep learning method to analyze DWI data scanned at multiple b‐values independent on Gaussian or non‐Gaussian models and to apply to a rectal cancer neoadjuvant chemoradiotherapy model. Study Type Retrospective. Population A total of 472 participants (age: 56.6 ± 10.5 years; 298 males and 174 females) with locally advanced adenocarcinoma were enrolled and chronologically divided into a training group ( n = 200; 42 pCR/158 non‐pCR), a validation group ( n = 72; 11 pCR/61 non‐pCR) and a test group ( n = 200; 44 pCR/156 non‐pCR). Field Strength/Sequence A 3.0 T MRI scanner. DWI with a single‐shot spin echo‐planar imaging pulse sequence at 12 b‐values (0, 20, 50, 100, 200, 400, 600, 800, 1000, 1200, 1400, and 1600 sec/mm 2 ). Assessment DWI signals from manually delineated tumor region were converted into a signature‐like picture by concatenating all histograms from different b‐values. Pathological results (pCR/non‐pCR) were used as the ground truth for deep learning. Gaussian and non‐Gaussian methods were used for comparison. Statistical Tests Analysis of variance for age; Chi‐square for gender and pCR/non‐pCR; area under the receiver operating characteristic (ROC) curve (AUC); DeLong test for AUC. P < 0.05 for significant difference. Results The AUC in the test group is 0.924 (95% CI: 0.866–0.983) for the signature‐like pictures converted from 35 bins, and it is 0.931 (95% CI: 0.884–0.979) for the signature‐like pictures converted from 70 bins, which is significantly ( Z = 3.258, P < 0.05) larger than D app , the best predictor in non‐Gaussian methods with AUC = 0.773 (95% CI: 0.682–0.865). Data Conclusion The proposed signature‐like pictures provide more accurate pretreatment prediction of the response to neoadjuvant chemoradiotherapy than the fitted methods for locally advanced rectal cancer. Evidence Level 3 Technical Efficacy Stage 2