Faculty of Data and Decision Sciences, Technion（以色列理工学院数据与决策科学学院）；The Hebrew University of Jerusalem（耶路撒冷希伯来大学）；Google Research（谷歌研究院）；Hebrew University Business School（希伯来大学商学院）；Federmann Center for the Study of Rationality, Hebrew University（希伯来大学费德曼理性研究中心）；Department of Cognitive and Brain Sciences, Hebrew University（希伯来大学认知与脑科学系）
Amir Taubenfeld, Yaniv Dover, Roi Reichart, Ariel Goldstein
The emergence of Large Language Models (LLMs), has opened exciting possibilities for constructing computational simulations designed to replicate human behavior accurately. Current research suggests that LLM-based agents become increasingly human-like in their performance, sparking interest in using these AI agents as substitutes for human participants in behavioral studies. However, LLMs are complex statistical learners without straightforward deductive rules, making them prone to unexpected behaviors. Hence, it is crucial to study and pinpoint the key behavioral distinctions between humans and LLM-based agents. In this study, we highlight the limitations of LLMs in simulating human interactions, particularly focusing on LLMs' ability to simulate political debates on topics that are important aspects of people's day-to-day lives and decision-making processes. Our findings indicate a tendency for LLM agents to conform to the model's inherent social biases despite being directed to debate from certain political perspectives. This tendency results in behavioral patterns that seem to deviate from well-established social dynamics among humans. We reinforce these observations using an automatic self-fine-tuning method, which enables us to manipulate the biases within the LLM and demonstrate that agents subsequently align with the altered biases. These results underscore the need for further research to develop methods that help agents overcome these biases, a critical step toward creating more realistic simulations.
Counterfactual Debating with Preset Stances for Hallucination Elimination of LLMs
University of Science and Technology of China（中国科学技术大学）；National University of Singapore（新加坡国立大学）；Electronic Science Research Institute of China Electronics（中国电子科技集团电子科学研究院）
Debating Truth: Debate-driven Claim Verification with Multiple Large Language Model Agents
Department of Interactive Media, Hong Kong Baptist University（香港浸会大学互动媒体系）；School of Computing and Data Science, The University of Hong Kong（香港大学计算与数据科学学院）；College of Computer Science and Artificial Intelligence, Fudan University（复旦大学计算机科学与人工智能学院）；Beijing Normal-Hong Kong Baptist University（北京师范大学-香港浸会大学联合国际学院）；Shenzhen Institute of Advanced Technology（深圳先进技术研究院）
AI Debaters are More Persuasive when Arguing in Alignment with Their Own Beliefs
FAIR, IALAB, Universidad de Buenos Aires（FAIR、IALAB、布宜诺斯艾利斯大学）；Universidad de Buenos Aires（布宜诺斯艾利斯大学）；Universidad Nacional de Córdoba（科尔多瓦国立大学）；BAISH, Universidad de Buenos Aires（BAISH、布宜诺斯艾利斯大学）；Instituto de Investigación en Informática LIDI, Universidad Nacional de La Plata（信息研究所LIDI、拉普拉塔国立大学；CONICET）；CONICET（计算机科学与工程系，南大学及ICIC UNS-CONICET）；Dept. of Comp. Sci. and Eng., Universidad Nacional del Sur & ICIC UNS-CONICET（人工智能研究所（IIIA-CSIC），西班牙）；Artificial Intelligence Research Institute (IIIA-CSIC), ES
With a Grain of SALT: Are LLMs Fair Across Social Dimensions?
Lahore University of Management Sciences（拉合尔管理科学大学）；National University of Computer and Emerging Sciences（国家计算机与新兴科学大学）；EMBL European Bioinformatics Institute（欧洲分子生物学实验室欧洲生物信息研究所）
本文提出 SALT 基准，从性别、宗教和种族维度评估 Llama、Gemma 等开源 LLM 的社会偏见，并结合 DeepSeek-R1 自动评估与人工验证揭示不同群体遭受系统性差别对待。
Debatable Intelligence: Benchmarking LLM Judges via Debate Speech Evaluation
School of Computer Science and Engineering, The Hebrew University of Jerusalem（计算机科学与工程学院，特拉维夫大学）；IBM Research（IBM研究院）；The Allen Institute for AI (AI2)（人工智能研究所）
Behavioral Economics of AI: LLM Biases and Corrections
Questrom School of Business, Boston University（波士顿大学奎斯特罗姆商学院）；SC Johnson College of Business, Cornell University（康奈尔大学SC约翰逊商学院）；National Bureau of Economic Research（美国国家经济研究局）
LLMs Are Biased Towards Output Formats! Systematically Evaluating and Mitigating Output Format Bias of LLMs
National University of Singapore（新加坡国立大学）；VinAI Research（VinAI研究院）；Institute of High Performance Computing (IHPC), A*STAR（新加坡科技研究局高性能计算研究所）；Salesforce Research（Salesforce研究院）；Nanyang Technological University（南洋理工大学）
University of California, Los Angeles（加州大学洛杉矶分校）；Seoul National University（首尔国立大学）；Qatar Computing Research Institute（卡塔尔计算研究所）；Google（谷歌）；George Washington University（乔治华盛顿大学）；Stanford University（斯坦福大学）；Scale AI；University of Washington（华盛顿大学）
