Position: LLM Social Simulations Are a Promising Research Method
Authors: Jacy Reese Anthis, Ryan Liu, Sean M Richardson, Austin C. Kozlowski, Bernard Koch, Erik Brynjolfsson, James Evans, Michael S. Bernstein
ICML 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | In this position paper, we argue that the promise of LLM social simulations can be achieved by addressing five tractable challenges. We ground our argument in a review of empirical comparisons between LLMs and human research subjects, commentaries on the topic, and related work. Our argument is grounded in a literature review of empirical studies that have compared human research subjects to LLMs, commentaries on the topic, and related work in social science and other LLM applications. |
| Researcher Affiliation | Academia | 1University of Chicago 2Stanford University 3Sentience Institute 4Princeton University 5Santa Fe Institute. Correspondence to: Jacy Reese Anthis <EMAIL>. |
| Pseudocode | No | The paper does not contain any clearly labeled pseudocode or algorithm blocks. The methodologies discussed are descriptive and refer to methods used in other studies. |
| Open Source Code | No | This paper is a position paper and a literature review, it does not present a novel methodology with associated source code. Therefore, it does not provide concrete access to source code for its own methodology. |
| Open Datasets | No | The paper is a literature review that discusses various datasets used in other research (e.g., Moral Foundations Questionnaire-2, American National Election Studies, General Social Survey in Table A1). However, it does not provide concrete access information (links, DOIs, repositories, or direct citations with authors/year for dataset access) for a dataset used in its own work or created by the authors of this paper. |
| Dataset Splits | No | This paper is a position paper and literature review; it does not present new experimental work or utilize specific dataset splits for its own analysis. Therefore, it does not provide specific dataset split information. |
| Hardware Specification | No | The paper does not describe any experiments conducted by the authors. Thus, it does not provide specific hardware details (GPU/CPU models, processor types, or memory amounts) used for running experiments. |
| Software Dependencies | No | The paper does not describe any specific software implementation of a methodology, and therefore does not list software dependencies with version numbers. |
| Experiment Setup | No | The paper is a position paper and literature review, not a report of new experimental work. Therefore, it does not provide details about an experimental setup, hyperparameters, or system-level training settings. |