Large Language Models (LLMs) are increasingly utilized in autonomous decision-making, where they sample options from vast action spaces. However, the heuristics that guide this sampling process remain under explored. We study this sampling behavior and show that this underlying heuristics resembles that of human decision-making: comprising a descriptive component (reflecting statistical norm) and a prescriptive component (implicit ideal encoded in the LLM) of a concept. We show that this deviation of a sample from the statistical norm towards a prescriptive component consistently appears in concepts across diverse real-world domains like public health, and economic trends. To further illustrate the theory, we demonstrate that concept prototypes in LLMs are affected by prescriptive norms, similar to the concept of normality in humans. Through case studies and comparison with human studies, we illustrate that in real-world applications, the shift of samples toward an ideal value in LLMs' outputs can result in significantly biased decision-making, raising ethical concerns.
Can Graph Descriptive Order Affect Solving Graph Problems with LLMs?
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of California, Merced(加州大学默塞德分校);SAIDS, University of Science and Technology of China(中国科学技术大学SAIDS);Tsinghua University(清华大学)
Research Trends for the Interplay between Large Language Models and Knowledge Graphs
University of Lübeck(吕贝克大学);University of Khenchela(肯切拉大学);Université de Toulouse(图卢兹大学);Université Paris Cité(巴黎西岱大学);Alliance University(联盟大学);IBM Research(IBM研究院)
Large Language Models Might Not Care What You Are Saying: Prompt Format Beats Descriptions
National Key Laboratory for Multimedia Information Processing, Peking University(北京大学多媒体信息处理国家重点实验室);MOE Key Laboratory of Computational Linguistics, Peking University School of Computer Science, Peking University(教育部计算语言学重点实验室,北京大学计算机科学学院,北京大学)
Empathy by Design: Aligning Large Language Models for Healthcare Dialogue
Department of Public Health Sciences, The University of Texas at El Paso, USA(公共卫生科学系,德克萨斯理工大学埃尔帕索分校);Department of Nursing, The University of Texas at El Paso, USA(护理系,德克萨斯理工大学埃尔帕索分校);Department of Rehabilitation Sciences, The University of Texas at El Paso, USA(康复科学系,德克萨斯理工大学埃尔帕索分校)
Controllable Narrative Rendering for Enhanced Assisted Writing
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院);HiThink Research(海天瑞声)
Comparative analysis of privacy-preserving open-source LLMs regarding extraction of diagnostic information from clinical CMR imaging reports
The D-Lab, Department of Precision Medicine, GROW - Research Institute for Oncology and Reproduction, Maastricht University(D实验室,精准医学系,GROW癌症与生殖研究研究所,马斯特里赫特大学);Division of Cardiovascular Imaging, Department of Cardiology I, University Hospital Münster(心血管影像科,心内科第一部门,穆恩斯特大学医院)
How LLMs Build Fictional Worlds: Setting and Narrative Space in AI-Generated Creative Storytelling
FAU Erlangen-Nürnberg(埃尔朗根-纽伦堡大学);Munich Center for Machine Learning(慕尼黑机器学习中心);LMU München(慕尼黑大学);Institute of AI for Health, Helmholtz Zentrum München(慕尼黑亥姆霍兹中心人工智能健康研究所);University of Birmingham(伯明翰大学)
The AI Language Proficiency Monitor -- Tracking the Progress of LLMs on Multilingual Benchmarks
Bundesministerium für wirtschaftliche Zusammenarbeit und Entwicklung (BMZ)(经济合作与发展部);Gesellschaft für Internationale Zusammenarbeit (GIZ)(国际发展合作署);Deutsches Forschungszentrum für Künstliche Intelligenz (DFKI)(德国人工智能研究中心)
本文提出AI Language Proficiency Monitor,一个评估多达200种语言(含低资源语言)的多语言基准,整合翻译、问答等任务,提供开源自动更新的排行榜和仪表板,促进多语言AI的透明与包容。
Simulating Clinical AI Assistance using Multimodal LLMs: A Case Study in Diabetic Retinopathy
Dana-Farber Cancer Institute & Tufts University School of Medicine(达纳-法伯癌症研究所及塔夫茨大学医学院);Dana-Farber Cancer Institute Brigham and Women’s Hospital & Harvard Medical School(达纳-法伯癌症研究所布里特妇女医院及哈佛医学院)