语言大模型 / LLM

Hyperbolic Deep Learning for Foundation Models: A Survey

Neil He, Hiren Madhu, Ngoc Bui, Menglin Yang, Rex Ying

机构 * Yale University（耶鲁大学） ； Hong Kong University of Science（香港科学大学） 纠错

专题命中 评测与基准 ：foundation model(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 11 Pages, SIGKDD 2025

Multi-Level Explanations for Generative Language Models

Lucas Monteiro Paes, Dennis Wei, Hyo Jin Do, Hendrik Strobelt, Ronny Luss, Amit Dhurandhar, Manish Nagireddy, Karthikeyan Natesan Ramamurthy, Prasanna Sattigeri, Werner Geyer, Soumya Ghosh

机构 * Harvard University（哈佛大学） ； IBM Research（IBM研究院） ； Merck Research Labs（默克研究实验室） 纠错

专题命中 评测与基准 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted as an oral presentation at ACL 2025. Code available at https://github.com/IBM/ICX360

Can External Validation Tools Improve Annotation Quality for LLM-as-a-Judge?

Arduin Findeis, Floris Weers, Guoli Yin, Ke Ye, Ruoming Pang, Tom Gunter

机构 * University of Cambridge（剑桥大学） ； Apple（苹果公司） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted at ACL 2025

A Practical Guide for Evaluating LLMs and LLM-Reliant Systems

Ethan M. Rudd, Christopher Andrews, Philip Tully

机构 * Google（谷歌） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

A Survey of the Evolution of Language Model-Based Dialogue Systems: Data, Task and Models

Hongru Wang, Lingzhi Wang, Yiming Du, Liang Chen, Jingyan Zhou, Yufei Wang, Kam-Fai Wong

机构 * The Chinese University of Hong Kong（香港中文大学） ； Huawei Noah Ark Lab（华为诺亚实验室） ； MoE Key Laboratory of High Confidence Software Technologies, The Chinese University of Hong Kong（高可信软件技术重点实验室，香港中文大学） 纠错

专题命中 评测与基准 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.AI

VerilogDB: The Largest, Highest-Quality Dataset with a Preprocessing Framework for LLM-based RTL Generation

Paul E. Calzada, Zahin Ibnat, Tanvir Rahman, Kamal Kandula, Danyu Lu, Sujan Kumar Saha, Farimah Farahmandi, Mark Tehranipoor

机构 * University of Florida（佛罗里达大学） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Towards Geo-Culturally Grounded LLM Generations

Piyawat Lertvittayakumjorn, David Kinney, Vinodkumar Prabhakaran, Donald Martin, Sunipa Dev

机构 * Google（谷歌） ； Washington University in St. Louis（华盛顿大学圣路易斯分校） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments ACL 2025 (main conference)

Perspective Dial: Measuring Perspective of Text and Guiding LLM Outputs

Taejin Kim, Siun-Chuon Mau, Konrad Vesey

机构 * CACI International Inc.（CACI国际公司） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 7 pages, 5 main pages of text, 5 figures, 2 tables. Research work performed at CACI INTL INC

SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agents

Jonathan Kutasov, Yuqi Sun, Paul Colognese, Teun van der Weij, Linda Petrini, Chen Bo Calvin Zhang, John Hughes, Xiang Deng, Henry Sleight, Tyler Tracy, Buck Shlegeris, Joe Benton

机构 * Anthropic ； Reduct Video ； Scale AI ； Constellation ； Redwood Research 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

hdl2v: A Code Translation Dataset for Enhanced LLM Verilog Generation

Charles Hong, Brendan Roberts, Huijae An, Alex Um, Advay Ratan, Yakun Sophia Shao

机构 * UC Berkeley（伯克利大学） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Published at ACM/IEEE International Symposium on Machine Learning for CAD (MLCAD) 2025

Agents Are All You Need for LLM Unlearning

Debdeep Sanyal, Murari Mandal

机构 * RespAI Lab, School of Computer Engineering, KIIT Bhubaneswar（RespAI实验室，计算机工程学院，KIIT大学） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted to COLM 2025

Evaluation of LLM-based Strategies for the Extraction of Food Product Information from Online Shops

Christoph Brosch, Sian Brumm, Rolf Krieger, Jonas Scheffler

机构 * Institute for Software Systems, University of Applied Sciences Trier（软件系统研究所，应用科学大学特里尔） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments Preprint for paper presented at DATA 2025 in Bilbao, Spain. Corrected -2.27 to -1.61 in abstract and +2.27 to +1.61 in discussion. Reference to journal and publication will follow

Journal ref In Proceedings of the 14th International Conference on Data Science, Technology and Applications, 2025, ISBN 978-989-758-758-0, ISSN 2184-285X, pages 709-715

Gradient-Based Model Fingerprinting for LLM Similarity Detection and Family Classification

Zehao Wu, Yanjie Zhao, Haoyu Wang

机构 * Huazhong University of Science（华中科技大学） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Kalahi: A handcrafted, grassroots cultural LLM evaluation suite for Filipino

Jann Railey Montalan, Jian Gang Ngui, Wei Qi Leong, Yosephine Susanto, Hamsawardhini Rengarajan, Alham Fikri Aji, William Chandra Tjhi

机构 * AI Singapore（AI新加坡） ； National University of Singapore（国立新加坡大学） ； MBZUAI ； Monash Indonesia（墨尔本印尼分校） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted for presentation at Paclic 38, 2024

Journal ref https://aclanthology.org/2024.paclic-1.49/

NatureLM-audio: an Audio-Language Foundation Model for Bioacoustics

David Robinson, Marius Miron, Masato Hagiwara, Benno Weck, Sara Keen, Milad Alizadeh, Gagan Narula, Matthieu Geist, Olivier Pietquin

机构 * Earth Species Project（地球物种项目） 纠错

专题命中 评测与基准 ：foundation model(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Demo page: https://earthspecies.github.io/naturelm-audio-demo/

LLM-Based Human-Agent Collaboration and Interaction Systems: A Survey

Henry Peng Zou, Wei-Chieh Huang, Yaozu Wu, Yankai Chen, Chunyu Miao, Hoang Nguyen, Yue Zhou, Weizhi Zhang, Liancheng Fang, Langzhou He, Yangning Li, Dongyuan Li, Renhe Jiang, Xue Liu, Philip S. Yu

机构 * University of Illinois Chicago（伊利诺伊大学芝加哥分校） ； University of Tokyo（东京大学） ； Tsinghua University（清华大学） ； MBZUAI ； McGill University（麦吉尔大学） ； University of California Irvine（加州大学 Irvine 分校） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments Paper lists and resources are available at https://github.com/HenryPengZou/Awesome-Human-Agent-Collaboration-Interaction-Systems

Automatic Demonstration Selection for LLM-based Tabular Data Classification

Shuchu Han, Wolfgang Bruckner

机构 * Capital One 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

The Evolution of Natural Language Processing: How Prompt Optimization and Language Models are Shaping the Future

Summra Saleem, Muhammad Nabeel Asim, Shaista Zulfiqar, Andreas Dengel

机构 * RPTU Rheinland-Pfälzische Technische Universität Kaiserslautern-Landau（莱茵-普法尔茨理工大学） ； German Research Centre for Artificial Intelligence（德国人工智能研究中心） 纠错

专题命中 评测与基准 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.AI

NepaliGPT: A Generative Language Model for the Nepali Language

Shushanta Pudasaini, Aman Shakya, Siddhartha Shrestha, Sahil Bhatta, Sunil Thapa, Sushmita Palikhe

机构 * Technological University Dublin（都柏林技术大学） ； Institute of Engineering（工程学院） ； Kathmandu Engineering College（加德满都工程学院） ； Fanshawe College（Fanshawe学院） ； Lambton College（Lambton学院） 纠错

专题命中 评测与基准 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments 11 pages, 9 figures

BLUR: A Benchmark for LLM Unlearning Robust to Forget-Retain Overlap

Shengyuan Hu, Neil Kale, Pratiksha Thaker, Yiwei Fu, Steven Wu, Virginia Smith

机构 * Carnegie Mellon University（卡内基梅隆大学） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

RePCS: Diagnosing Data Memorization in LLM-Powered Retrieval-Augmented Generation

Le Vu Anh, Nguyen Viet Anh, Mehmet Dik, Luong Van Nghia

机构 * 1 Institute of Information Technology, Vietnam Academy of Science ； 2 Institute of Information Technology, Vietnam Academy of Science ； 3 Department of Mathematics, Computer Science \& Physics, Rockford University, Illinois, United States Email ； 4 Department of Information Technology, Dong A University, Da Nang, Vietnam Email 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 11 pages, 7 figures, 5 tables

Hypothesis Testing for Quantifying LLM-Human Misalignment in Multiple Choice Settings

Harbin Hong, Sebastian Caldas, Liu Leqi

机构 * Princeton University（普林斯顿大学） ； University of Texas at Austin（德克萨斯大学奥斯汀分校） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

PsychBench: A comprehensive and professional benchmark for evaluating the performance of LLM-assisted psychiatric clinical practice

Shuyu Liu, Ruoxi Wang, Ling Zhang, Xuequan Zhu, Rui Yang, Xinzhu Zhou, Fei Wu, Zhi Yang, Cheng Jin, Gang Wang

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

ConsistencyChecker: Tree-based Evaluation of LLM Generalization Capabilities

Zhaochen Hong, Haofei Yu, Jiaxuan You

机构 * University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted at ACL 2025 Main Conference

Can Time-Series Foundation Models Perform Building Energy Management Tasks?

Ozan Baris Mulayim, Pengrui Quan, Liying Han, Xiaomin Ouyang, Dezhi Hong, Mario Bergés, Mani Srivastava

机构 * Carnegie Mellon University（卡内基梅隆大学） ； University of California, Los Angeles（加州大学洛杉矶分校） ； Hong Kong University of Science and Technology（香港科学与技术大学） ； Amazon（亚马逊公司） 纠错

专题命中 评测与基准 ：foundation model(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 30 pages, 5 tables, 8 figures. Under review for Data-Centric Engineering journal

Evolutionary Perspectives on the Evaluation of LLM-Based AI Agents: A Comprehensive Survey

Jiachen Zhu, Menghui Zhu, Renting Rui, Rong Shan, Congmin Zheng, Bo Chen, Yunjia Xi, Jianghao Lin, Weiwen Liu, Ruiming Tang, Yong Yu, Weinan Zhang

机构 * Shanghai Jiao Tong University（上海交通大学） ； Huawei Noah’s Ark Lab（华为诺亚实验室） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

STRCMP: Integrating Graph Structural Priors with Language Models for Combinatorial Optimization

Xijun Li, Jiexiang Yang, Jinghao Wang, Bo Peng, Jianguo Yao, Haibing Guan

机构 * Shanghai Key Laboratory of Scalable Computing and Systems（上海可扩展计算与系统重点实验室） ； School of Computer Science, Shanghai Jiao Tong University（上海交通大学计算机科学学院） 纠错

专题命中 评测与基准 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.AI、cs.LG

OPT-BENCH: Evaluating LLM Agent on Large-Scale Search Spaces Optimization Problems

Xiaozhe Li, Jixuan Chen, Xinyu Fang, Shengyuan Ding, Haodong Duan, Qingwen Liu, Kai Chen

机构 * Tongji University（同济大学） ； Shanghai AI Lab（上海人工智能实验室） ； Nanjing University（南京大学） ； Zhejiang University（浙江大学） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Time To Impeach LLM-as-a-Judge: Programs are the Future of Evaluation

Tzu-Heng Huang, Harit Vishwakarma, Frederic Sala

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Unify Graph Learning with Text: Unleashing LLM Potentials for Session Search

Songhao Wu, Quan Tu, Hong Liu, Jia Xu, Zhongyi Liu, Guannan Zhang, Ran Wang, Xiuying Chen, Rui Yan

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG