Background The rising prevalence of type 2 diabetes mellitus (T2DM) in the elderly is associated with mental health disorders like cognitive impairment and depression due to hyperglycemia and inflammation. The present study aims to estimate the prevalence of cognitive impairment and depression and its association with clinical, biochemical, and psychosocial factors to identify high-risk subjects. Methods A cross-sectional study was conducted at a tertiary care teaching hospital and enrolled 99 patients of T2DM, aged equal to or more than 60 years from North India. Cognitive function and depression were assessed by the Hindi Mental Scale Examination (HMSE) and the Hindi version of the Geriatric Depression Scale (GDS-H), respectively. Fasting plasma glucose (FPG), glycated hemoglobin (HbA1C), and the lipid profile were measured. Univariate and multivariate binary logistic regression analyses were applied to identify association and independent predictors, respectively, and receiver operating characteristic (ROC) curve analysis to determine the optimal cut-off values. Results The mean age of the patients was 66.68 years, and 38.38% were females. The prevalence of impaired cognition and depression in elderly T2DM subjects was 37.37% and 43.43%, respectively. HMSE was inversely correlated while GDS-H was positively correlated with FPG and HbA1C. Cognitive impairment was independently predicted by age, HbA1C levels, and rural residence. Depression was independently associated with HbA1C levels and being unmarried or widowed. The optimal cut-off for cognitive impairment was age >63 years and HbA1C >7.7%. For depression, the cut-off for HbA1C was >6.9%. Conclusion This study revealed a higher prevalence of cognitive impairment and depression among elderly T2DM patients in North India. The age of more than 63 years, poor glycemic control, rural residents, and marital status are high-risk groups for cognitive impairment and depression. This study suggests the integration of routine mental health screening for high-risk elderly T2DM patients and the development of comprehensive diabetes management programs that address both physical and mental health aspects.