Optical coherence tomography (OCT) is being investigated as an intraoperative margin assessment tool for breast cancer. In this work, we developed a customized deep convolutional neural network (CNN) for classification of breast cancer in OCT images. Images were acquired with a custom ultrahigh-resolution OCT system and a standard resolution system. We classify healthy tissues such as stroma and adipose tissue, as well as diseased tissue including ductal carcinoma in situ (DCIS) and invasive ductal carcinoma (IDC). Future work involves increasing representation from different kinds of tumors such as mucinous carcinoma, papillary carcinoma, and phyllodes tumors.