Pancreatic ductal adenocarcinoma (PDAC) remains an aggressive malignancy. The occurrence of perineural invasion is associated with neuropathic pain and poor prognosis of PDAC, underscoring the active participation of nerves and their potential as therapeutic targets. Lidocaine is a local anesthetic with antitumor properties in some tumors in the clinic. Nevertheless, its clinical application in PDAC is constrained by the insufficient tumor accumulation and potential neurovirulence associated with a high-dose regimen. Here, a tumor microenvironment-targeted and -responsive liposome was constructed to deliver lidocaine for restraining PDAC growth through single nerve regulation. By conjugation of a collagen binding peptide, the pH-responsive liposomes accumulate in the extracellular matrix. The released lidocaine selectively reduces neurite length and density, thereby indirectly halting the progression and metastasis of PDAC in an orthotopic mouse model without noticeable adverse effects. This study highlights the potential of anesthetic-based nanomodulation of crosstalk between nerve and tumor cells for PDAC treatment.