Pore structure, a critical factor influencing the physical properties of oil and gas reservoirs, shows great variation with the reservoir rock type. Efficient exploration and development of oil and gas resources require comprehensive understanding of pore structure differences between various reservoirs. In this study, to clarify these differences, pore size distributions and pore structure parameters were obtained through low-pressure nitrogen (N2) adsorption–desorption experiments using shale, low-permeability sandstone, and tight sandstone as rock types. Pore space fractal dimension obtained from a proposed calculating method and pore surface fractal dimension obtained from the Frenkel–Halsey–Hill model were combined with N2 adsorption results to analyze quantitatively the pore structures of shale and sandstone. The results show that shale consisted mainly of inkbottle-shaped pores whereas sandstone was composed of slit-shaped pores. Compared with sandstone, shale had a larger specific surface area but smaller average pore diameter and pore volume. In addition, the pore structure heterogeneity and pore surface irregularity of shale were more significant, considering its larger fractal dimensions. Relationships among fractal dimensions and porosity, permeability, and pore structure parameters explained that pore size and pore distribution were the main influencing factors of porosity and permeability. These results highlight the practicability of fractal theory in characterizing pore structure and petrophysical properties of unconventional reservoirs.