Detecting and Learning Against Unknown Opponents for Automated Negotiations

Learning in automated negotiations, while successful for many tasks in recent years, is still hard when coping with different types of opponents with unknown strategies. It is critically essential to learn about the opponents from observations and then find the best response in order to achieve efficient agreements. In this paper, we propose a novel framework named Deep BPR+ (DBPR+) negotiating agent framework, which includes two key components: a learning module to learn a new coping policy when encountering an opponent using a previously unseen strategy, and a policy reuse mechanism to efficiently detect the strategy of an opponent and select the optimal response policy from the policy library. The performance of the proposed DBPR+ agent is evaluated against winning agents of ANAC competitions under varied negotiation scenarios. The experimental results show that DBPR+ agent outperforms existing state-of-the-art agents, and is able to make efficient detection and optimal response against unknown opponents.