Automated Design of Complex Analog Circuits with Multiagent based Reinforcement Learning

Despite the effort of analog circuit design automation, currently complex analog circuit design still requires extensive manual iterations, making it labor intensive and time-consuming. Recently, reinforcement learning (RL) algorithms have been demonstrated successfully for the analog circuit design optimization. However, a robust and highly efficient RL method to design analog circuits with complex design space has not been fully explored yet. In this work, inspired by multiagent planning theory as well as human expert design practice, we propose a multiagent based RL (MA-RL) framework to tackle this issue. Particularly, we (i) partition the complex analog circuits into several sub-blocks based on topology information and effectively reduce the complexity of design search space; (ii) leverage MA-RL for the circuit optimization, where each agent corresponds to a single sub-block, and the interactions between agents delicately mimic the best design tradeoffs between circuit sub-blocks by human experts; (iii) introduce the multiagent twin-delayed techniques to further boost training stability and accomplish higher performances. Experiments on two different analog circuit topologies and knowledge transfers between two technology nodes are demonstrated. It's shown that MA-RL framework can achieve the best FoM for complex analog circuits design. This work shines the light for future large scale analog circuit system design automation.