Reliable simulation of human behaviour is essential for explaining, predicting, and intervening in our society. Recent advances in large language models (LLMs) have shown promise in emulating human behaviours, interactions, and decision-making, offering a powerful new lens for social science studies. However, the extent to which LLMs diverge from authentic human behaviour in social contexts remains underexplored, posing risks of misinterpretation in scientific studies and unintended consequences in real-world applications.
This study introduces a systematic framework for analysing LLMs’ behaviour in social simulation. Our approach simulates multi-agent interactions through chatroom-style conversations and analyzes them across five linguistic dimensions, providing a simple yet effective method to examine emergent social cognitive biases. We conduct extensive experiments involving eight representative LLMs across three families.
The results show that LLMs do not faithfully reproduce genuine human behaviour but instead reflect overly idealised versions of it, shaped by the social desirability bias. In particular, LLMs show social role bias, primacy effect, and positivity bias, resulting in “Utopian” societies that lack the complexity and variability of real human interactions. These findings suggest that LLM-based simulations reproduce how people wish others to behave rather than how people themselves behave, a divergence that amplifies through multi-agent interaction and, if uncritically adopted in social analysis, may produce an artificially narrowed representation of social life.
This paper calls for more socially grounded LLMs that better capture the diversity of human social behaviour. Supported by the China Postdoctoral Science Foundation under Grant Number 2025M782534. School of Information and Control Engineering, China University of Mining and Technology, Xuzhou, China Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences, Beijing, China School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, China Correspondence to Ning Bian or Jun Wang.
The authors declare no competing interests. This article does not contain any studies with human participants performed by any of the authors. Informed consent is not applicable to this study, as no human participants were directly involved and no new human data were collected by the authors.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it.
Extract — continue reading at the source.
