Toward a Novel Methodology in Economic Experiments: Simulation of the Ultimatum Game with Large Language Models

This study explores the optimal settings for a prevalent simulation in which agents powered by large language models (LLMs) make decisions without predetermined actions as a substitute for economic experiments. Economic experiments are essential methods in economics where the behaviors of participants are observed under controlled conditions to test hypotheses and theories, involving significant time, effort, and cost. If a simulation can supplant economic experiments, researchers can overcome these limitations by utilizing it. We focused on three essential factors for applying the simulation using LLMs to economic experiments and conducted simulations for both the proposer and responder sides of the one-shot ultimatum game in various settings of the three factors. The sensitivity analysis revealed that, on the proposer side, there was a setting that produced results similar to actual human-centered experiments. However, on the responder side, none of the simulation setups yielded results consistent with real-world experimental data. These findings suggest both the potential and limitations of the novel simulation with LLMs as a substitute for economic experiments.