Schottky-barrier height (SBH) for intimate Al/Si contacts is investigated in relation to the interfacial crystallographic alignment, which is observed by transmission-electron microscopy. Epitaxial and rotational Al films are obtained by low-temperature molecular-beam epitaxy on both Si(111) and (100) surfaces, by controlling the surface structures. The SBH measurements for n- and p-type samples reveal that the absence of epitaxial alignment at the interfaces significantly lowers the Fermi-level pinning position for the contacts on both (111) and (100) surfaces.