Walnut, which is one of specific hollow shell granules, is inevitably subjected to damage deriving from repeated impacts in postharvest and processing. In this regard, the existence of damage was validated through force-deformation profile. The damage model was established and validated, respectively. Effects of walnut size and impact energy on damage behaviors, including damage variable characterizing degree of damage and damage accumulative coefficient describing sensibility to damage, were discussed, respectively. Results indicate that damage model by repeated impacts has been demonstrated to be valid for walnut shell fracture, in a way that is relatively independent of specific impact energy, only varying with walnut size. There exists a threshold size determining fracture mechanisms. Below threshold size, it is due to damage driving initial micro-crack gradually occurs, whereas beyond threshold size, it attributes to losing structural stability. The piecewise function between damage accumulation coefficient and walnut size was established.