Strong Self‐Enhancement of Optical Nonlinearity in a Topological Insulator with Generation of Tamm State

Abstract As new states of matter currently attracting broad attention in the optics field, topological insulators (TIs) present topologically‐protected conducting surface states surrounding insulating bulk and prominent third‐order optical nonlinearities in an ultrawide frequency range. Optical frequency conversion in TI nanofilms plays a crucial role in TI nonlinear optical functionalities, whose enhancement is in demand for high‐performance devices. Herein, an optical Tamm state is observed between the Sb 2 Te 3 TI nanofilm and 1D photonic crystal (PC) with a self‐enhancement behavior of optical nonlinearity in the TI nanofilm. The experiments demonstrate that there exists a distinct reflection dip in the forbidden band of the PC, which is attributed to the appearance of Tamm state in the TI/PC structure. The wavelength of the TI‐based Tamm state possesses a nearly linear redshift with increasing TI film thickness. The measurements are in excellent agreement with simulation and theoretical calculations. Moreover, a strong third‐harmonic generation reinforcement of 30‐fold is experimentally observed in a 90 nm Sb 2 Te 3 film, which stems from the field enhancement induced by the TI self‐excited Tamm state at the fundamental frequency. This work opens a new door for generation of optical Tamm states and applications of TIs in high‐efficiency nonlinear optical functionalities, especially frequency conversion.