Understanding the nature of low thermal conductivity and phonon transport properties is critical for the design of potential thermoelectric materials. Based on density-functional theory combined with the phonon Peierls-Boltzmann transport equation, we reveal that Ag-based intercalated layered materials $\mathrm{AgBi}{X}_{2}$ (X = S, Se, and Te) have inherently low lattice thermal conductivity, which is mainly attributed to the anticrossing behavior of low-frequency optical phonons and longitudinal acoustic phonons induced by the rattling mode of the cations. It is found that the optical phonons in $\mathrm{AgBi}{\mathrm{S}}_{2}$ contribute dominantly (up to 65%) to the total thermal conductivity, originating from the weak bonding nature of intercalated Ag atoms that leads to strong anharmonicity and softening of transverse acoustic phonons. Electronic relaxation times under acoustic deformation potential scattering, polar optical phonon scattering, and ionized impurity scattering are considered to obtain reasonable electron transport properties. The ZT value of $p$-type $\mathrm{AgBi}{\mathrm{S}}_{2}$ reaches 1.77 for optimal doping at room temperature, which can further be enhanced to \ensuremath{\sim}2.6 through strain engineering. The present work demonstrates that chemically controlled weak bonding in Ag-based intercalated layered structure produces intrinsically low thermal conductivity and provides important theoretical insight for thermal insulator and thermoelectric applications.