The University of Hong Kong(香港大学);Tsinghua University(清华大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Virginia(弗吉尼亚大学);Peking University(北京大学);WeChat AI(微信人工智能);The Chinese University of Hong Kong(香港中文大学)
Siheng Li, Cheng Yang, Taiqiang Wu, Chufan Shi, Yuji Zhang, Xinyu Zhu, Zesen Cheng, Deng Cai, Mo Yu, Lemao Liu, Jie Zhou, Yujiu Yang, Ngai Wong, Xixin Wu, Wai Lam
Honesty is a fundamental principle for aligning large language models (LLMs) with human values, requiring these models to recognize what they know and don't know and be able to faithfully express their knowledge. Despite promising, current LLMs still exhibit significant dishonest behaviors, such as confidently presenting wrong answers or failing to express what they know. In addition, research on the honesty of LLMs also faces challenges, including varying definitions of honesty, difficulties in distinguishing between known and unknown knowledge, and a lack of comprehensive understanding of related research. To address these issues, we provide a survey on the honesty of LLMs, covering its clarification, evaluation approaches, and strategies for improvement. Moreover, we offer insights for future research, aiming to inspire further exploration in this important area.
BeHonest: Benchmarking Honesty in Large Language Models
Shanghai Jiao Tong University(上海交通大学);Carnegie Mellon University(卡内基梅隆大学);Fudan University(复旦大学);Shanghai AI Laboratory(上海人工智能实验室);Generative AI Research Lab (GAIR)(生成式人工智能研究实验室(GAIR))
HonestLLM: Toward an Honest and Helpful Large Language Model
MBZUAI(Mohamed bin Zayed 人工智能大学);Huazhong University of Science and Technology(华中科技大学);University of Notre Dame(圣母大学);University of Washington(华盛顿大学);Peking University(北京大学);Lehigh University(理海大学)
State Key Laboratory of AI Safety(人工智能安全国家重点实验室);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学)
Do Large Language Models Know What They Don't Know II? A Fully Behavioral, Non-Cognitive Measure of Epistemic Honesty
Tarsus University(塔尔苏斯大学);Arizona State University (ASU)(亚利桑那州立大学);Institute for Social Science Research(社会科学研究所);School of Human Evolution and Social Change(人类进化与社会变革学院);School of Computing and Augmented Intelligence (SCAI)(计算与增强智能学院)
Truthful AI Advisors: A Pre-Specified Benchmark for Large Language Model Honesty Under Preference Misalignment
Amazon Lab126, HW Tech Org.(亚马逊实验室126,硬件技术组织);Computational Modeling and Simulation University of Pittsburgh(计算建模与仿真大学匹兹堡分校);Mathematics & Statistics Department University of Minnesota Duluth(数学与统计学系明尼苏达大学 Duluth 分校)
Enhancing Multiple Dimensions of Trustworthiness in LLMs via Sparse Activation Control
State Key Lab of CAD&CG, Zhejiang University(浙江大学CAD&CG国家重点实验室);Alibaba Cloud(阿里云);Hong Kong Baptist University(香港浸会大学);School of Software Technology, Zhejiang University(浙江大学软件学院);Zhiyuan Research Institute(智源研究院);Fabu Inc.(fabu公司)
Representation Engineering for Large-Language Models: Survey and Research Challenges
Wisent AI;University of Warwick(华威大学);Amazon Web Services(亚马逊网络服务);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);Perplexity;University of Cambridge(剑桥大学);Mila;University of Technology Sydney(悉尼科技大学)
A Comprehensive Survey of Accelerated Generation Techniques in Large Language Models
Amirkabir University of Technology(阿米尔卡比尔理工大学);Stanford University(斯坦福大学);Massachusetts Institute of Technology(麻省理工学院);Columbia University(哥伦比亚大学);Amazon GenAI(亚马逊生成式AI部门)
Histoires Morales: A French Dataset for Assessing Moral Alignment
Laboratoire Hubert Curien(于贝尔·屈里安实验室);CNRS(法国国家科学研究中心);Université Lumière Lyon 2(里昂第二大学);Université Claude Bernard Lyon 1(里昂第一大学);ERIC(信息、表示与智能认知研究室);Télécom Paris(巴黎高等电信学院);Institut Polytechnique de Paris(巴黎理工学院)
A Survey on Model Compression for Large Language Models
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院);Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院)
Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey
Raytheon BBN Technologies(雷神BBN技术公司);Harvard University(哈佛大学);University of Pennsylvania(宾夕法尼亚大学);University of Oregon(俄勒冈大学);University of the Basque Country (UPV/EHU)(巴斯克大学 (UPV/EHU))