When making decisions under uncertainty, individuals often deviate from rational behavior, which can be evaluated across three dimensions: risk preference, probability weighting, and loss aversion. Given the widespread use of large language models (LLMs) in decision-making processes, it is crucial to assess whether their behavior aligns with human norms and ethical expectations or exhibits potential biases. Several empirical studies have investigated the rationality and social behavior performance of LLMs, yet their internal decision-making tendencies and capabilities remain inadequately understood. This paper proposes a framework, grounded in behavioral economics, to evaluate the decision-making behaviors of LLMs. Through a multiple-choice-list experiment, we estimate the degree of risk preference, probability weighting, and loss aversion in a context-free setting for three commercial LLMs: ChatGPT-4.0-Turbo, Claude-3-Opus, and Gemini-1.0-pro. Our results reveal that LLMs generally exhibit patterns similar to humans, such as risk aversion and loss aversion, with a tendency to overweight small probabilities. However, there are significant variations in the degree to which these behaviors are expressed across different LLMs. We also explore their behavior when embedded with socio-demographic features, uncovering significant disparities. For instance, when modeled with attributes of sexual minority groups or physical disabilities, Claude-3-Opus displays increased risk aversion, leading to more conservative choices. These findings underscore the need for careful consideration of the ethical implications and potential biases in deploying LLMs in decision-making scenarios. Therefore, this study advocates for developing standards and guidelines to ensure that LLMs operate within ethical boundaries while enhancing their utility in complex decision-making environments.
Who Does What? Archetypes of Roles Assigned to LLMs During Human-AI Decision-Making
University of Cambridge（剑桥大学）；National Cancer Institute, National Institutes of Health（国家癌症研究所，国立卫生研究院）；Research Centre Trust, UA Ruhr, University Duisburg-Essen（研究信托中心，UA Ruhr，杜伊斯堡-埃森大学）
Logic Distillation: Learning from Code Function by Function for Decision-making Tasks
The School of Computer and Artificial Intelligence of Zhengzhou University（郑州大学计算机与人工智能学院）；Engineering Research Center of Intelligent Swarm Systems, Ministry of Education（教育部智能群体系统工程研究中心）；National Supercomputing Center In Zhengzhou（郑州国家超级计算中心）；Zhejiang University（浙江大学）
SOLID: a Framework of Synergizing Optimization and LLMs for Intelligent Decision-Making
Department of Industrial & Systems Engineering, University of Washington（工业与系统工程系，华盛顿大学）；College of Engineering, University of Washington（工程学院，华盛顿大学）；Michael G. Foster School of Business, University of Washington（Michael G. Foster商学院，华盛顿大学）
Rare Diseases, Common Dilemmas: LLMs Prioritize Equal Resource Distribution over Patient Benefit in Decision-Making
Harvard T.H. Chan School of Public Health（哈佛陈曾熙公共卫生学院）；Harvard Medical School（哈佛医学院）；Clalit Health Services（克拉利特医疗服务机构）；Harvard College（哈佛学院）；Harvard-MIT Program in Health Sciences and Technology（哈佛-麻省理工健康科学与技术项目）
Rethinking Prospect Theory for LLMs: Revealing the Instability of Decision-Making under Epistemic Uncertainty
Hong Kong University of Science and Technology（香港科技大学）；Huazhong University of Science and Technology（华中科技大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Beyond MedQA: Towards Real-world Clinical Decision Making in the Era of LLMs
Department of Computer Science, Emory University（埃默里大学计算机科学系）；Children’s Healthcare of Atlanta（亚特兰大儿童医院）；Nell Hodgson Woodruff School of Nursing, Emory University（埃默里大学奈尔·霍德森·伍德鲁夫护理学院）
Adversarial Testing in LLMs: Insights into Decision-Making Vulnerabilities
School of Computing, Dublin City University（都柏林城市大学计算机学院）；Insight SFI Research Centre for Data Analytics（数据分析SFI研究中心）；North China Electric Power University（华北电力大学）；Harbin Institute of Technology (Weihai)（威海工业大学）
