The realm of aspect-based sentiment analysis (ABSA), which delves into the nuanced sentiment expressions individuals hold towards specific services or products, has demonstrated immense potential in real-world applications. Recently, ABSA has evolved into the development of aspect-based sentiment quadruple extraction (ASQP). ASQP's objective is to predict four crucial sentiment elements: aspect, sentiment, opinion and category, where such comprehensive approach paints a holistic description of sentiment, facilitating downstream applications. However, prevailing ASQP models suffer from various limitations, such as inefficiency in decoding, inadequate handling of implicit aspects and opinions, and underutilization of structural information. In this paper, we explore an innovative approach to enhance ASQP. Firstly, we adopt a pointer-based non-autoregressive generative framework, enabling the parallel generation of all sentiment quadruples. This approach preserves the advantages of generative methods while significantly boosting decoding efficiency. Additionally, we introduce latent variable learning to model the aspect and opinion elements, effectively enhancing our ability to reason about implicit ASQP components. Furthermore, we propose an aspect-and-opinion-guided latent structure to bolster sentiment-aware context learning. This dynamically induced graph structure adapts to the specific requirements of the task, offering optimal support for ASQP. Our method outperforms current state-of-the-art models on four benchmark ASQP datasets, demonstrating its significant superiority. A detailed analysis highlights the benefits of non-autoregressive decoding in achieving high inference efficiency, the effectiveness of the variational module in capturing implicit sentiment elements, and the value of the dynamically induced latent structure in accurate sentiment feature learning. Moreover, our system excels in producing interpretable predictions.