Breast radiation therapy fluence painting with multi‐agent deep reinforcement learning

The electronic compensation (ECOMP) technique for breast radiation therapy provides excellent dose conformity and homogeneity. However, the manual fluence painting process presents a challenge for efficient clinical operation. To facilitate the clinical treatment planning automation of breast radiation therapy, we utilized reinforcement learning (RL) to develop an auto-planning tool that iteratively edits the fluence maps under the guidance of clinically relevant objectives. With institutional review board (IRB) approval, 70 patients treated with 6MV tangential photon beams with ECOMP technique were retrospectively collected and included in this study (20/50 for training/testing). Each pixel in the fluence map was assigned a reinforcement learning agent to perform independent action. Beam-eye-view projected dose profiles were generated to form state information as the input of the RL network. By predicting the Q value, pixel-wise actions were selected to modify specific pixel value in the fluence maps to improve overall plan quality. After dose calculation, reward signal calculated from the variation of target coverage and dose homogeneity was fed back to the RL framework and used to update network parameters. The RL generated plans were evaluated with dose distribution and dosimetric endpoints (i.e., Breast PTV V90%, Breast PTV V95%, Breast PTV V105%, Lung V20 Gy, Heart V5 Gy, Dmax) and compared with clinical plans. The RL agent took around 90 s to generate a ECOMP treatment plan. The RL plans exhibited plan quality comparable to clinical plans in terms of isodose distribution and dosimetric endpoints. The mean Breast PTV V95%, Breast PTV V105% of RL plans are 77.759%(±8.904%)$77.759{\mathrm{\ \% }}( { \pm 8.904{\mathrm{\ \% }}} )$ and 8.522cc(±11.469cc)$8.522{\mathrm{\ cc\ }}( { \pm 11.469{\mathrm{\ cc}}} )$ , compared to 78.568%(±9.094%)$78.568{\mathrm{\ \% }}( { \pm 9.094{\mathrm{\ \% }}} )$ and 34.298cc(±36.297cc)$34.298\ {\mathrm{cc}}\ ( { \pm 36.297{\mathrm{\ cc}}} )$ cc of clinical plans. The developed RL framework efficiently generates breast ECOMP plans with clinical acceptable plan quality.