Brown sugar (non-centrifugal cane sugar) is popular for its pleasant caramel-like aroma and sweetness. Vacuum simultaneous steam distillation and extraction (V-SDE) and gas chromatography–mass spectrometry (GC–MS) was used to study the volatile fraction of brown sugar. To further determine the aroma-active compounds in brown sugar, gas chromatography–mass spectrometry–olfactometry (GC–MS–O) were used in conjunction with aroma extraction dilution analysis (AEDA), odor activity values (OAVs), and sensory evaluation to analyze the effects of the key aroma-active compounds on sweetness. A total of 37 aroma-active compounds were obtained, mainly including ketones, pyrazines, alkanes, phenols and alcohols, which contributed caramel, sweet and fruity notes to brown sugar. Among them, furfural, benzeneacetaldehyde, 2,3-butanedione, β-damascenone, 2-methoxyphenol, dihydro-2-methyl-3(2H)-furanone, 2-furanmethanol and butyrolactone could significantly enhance the sweetness of sugar solution because of the congruency of the aroma attributes and sweetness.