语言大模型 / LLM - arXivDaily 专题

2302.04662 2023-05-01 cs.PL cs.AI cs.CL 88%

Generating High-Precision Feedback for Programming Syntax Errors using Large Language Models

Tung Phung, José Cambronero, Sumit Gulwani, Tobias Kohn, Rupak Majumdar, Adish Singla, Gustavo Soares

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments Published in International Conference on Educational Data Mining (EDM) 2023

2210.10723 2023-03-20 cs.CL cs.AI 88%

TabLLM: Few-shot Classification of Tabular Data with Large Language Models

Stefan Hegselmann, Alejandro Buendia, Hunter Lang, Monica Agrawal, Xiaoyi Jiang, David Sontag

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

2303.04864 2023-03-10 cs.LO cs.AI cs.LG 88%

nl2spec: Interactively Translating Unstructured Natural Language to Temporal Logics with Large Language Models

Matthias Cosler, Christopher Hahn, Daniel Mendoza, Frederik Schmitt, Caroline Trippel

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI、cs.LG

2203.13474 2023-03-01 cs.LG cs.CL cs.PL 88%

CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis

Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, Caiming Xiong

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

2302.12927 2023-02-28 cs.RO cs.AI cs.CL 88%

Robot Behavior-Tree-Based Task Generation with Large Language Models

Yue Cao, C. S. George Lee

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments The extended abstract of this paper is accepted in AAAI 2023 Spring Symposium on Challenges Requiring the Combination of Machine Learning and Knowledge Engineering (AAAI-MAKE 2023)

2302.11957 2023-02-24 cs.CL cs.AI 88%

Sentence Simplification via Large Language Models

Yutao Feng, Jipeng Qiang, Yun Li, Yunhao Yuan, Yi Zhu

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

2302.03491 2023-02-08 cs.CL cs.LG 88%

Learning Translation Quality Evaluation on Low Resource Languages from Large Language Models

Amirkeivan Mohtashami, Mauro Verzetti, Paul K. Rubenstein

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

2203.15556 2023-01-08 cs.CL cs.LG 88%

Training Compute-Optimal Large Language Models

Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Jack W. Rae, Oriol Vinyals, Laurent Sifre

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

2208.10063 2022-09-14 cs.CL cs.AI 88%

Selection Collider Bias in Large Language Models

Emily McMilin

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments 12 pages, 16 figures, UAI 2022 Causal Representation Learning Workshop

2207.08982 2022-07-20 cs.CL cs.AI 88%

Selection Bias Induced Spurious Correlations in Large Language Models

Emily McMilin

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments 8 pages, 5 figures, Published at the ICML 2022 Workshop on Spurious Correlations, Invariance, and Stability

2603.25681 2026-08-20 cs.CL 版本更新 88%

Self-Improvement of Large Language Models: A Technical Overview and Future Outlook

大语言模型的自我提升：技术概述与未来展望

Haoyan Yang, Mario Xerri, Solha Park, Huajian Zhang, Yiyang Feng, Sai Akhil Kogilathota, Jiawei Zhou

机构 * Zesearch NLP Lab, Stony Brook University（石溪大学 Zesearch NLP 实验室）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL；LLM(comments)

AI总结本文探讨大语言模型的自我提升技术，提出闭环生命周期框架，分析各组件方法并讨论未来研究方向。

Comments Accepted by TMLR and awarded the Survey Certification; 128 pages, 12 figures, and 14 tables. Github Repo: https://github.com/Zesearch/self-improvement-llm

2603.01423 2026-03-03 cs.CL 88%

Quantifying Conversational Reliability of Large Language Models under Multi-Turn Interaction

量化大型语言模型在多轮交互中的对话可靠性

Jiyoon Myung

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL；foundation model(comments)

AI总结研究评估了大型语言模型在多轮对话中的可靠性，发现其在复杂交互中存在显著下降，揭示了指令漂移、意图混淆等失败模式，强调了对LLM进行压力测试和改进评估方法的重要性。

Comments Accepted at the Workshop on Assessing and Improving Reliability of Foundation Models in the Real World (AAAI 2026)

2505.10309 2026-01-23 cs.AI cs.HC cs.SI 88%

A large-scale evaluation of commonsense knowledge in humans and large language models

对人类和大语言模型常识知识的大规模评估

Tuan Dung Nguyen, Duncan J. Watts, Mark E. Whiting

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI；LLM(comments)

AI总结本文提出了一种评估人类和大语言模型常识知识的方法，发现较小的开放权重模型在常识能力上表现更优，强调常识知识的文化基础与人类群体差异。

Comments Code and data: https://github.com/Watts-Lab/commonsense-llm-eval

2407.18008 2025-12-02 cs.CY cs.CL 88%

GermanPartiesQA: Benchmarking Commercial Large Language Models and AI Companions for Political Alignment and Sycophancy

GermanPartiesQA: 对商业大语言模型和AI伴侣在政治倾向和趋炎附势方面的基准测试

Jan Batzner, Volker Stocker, Stefan Schmid, Gjergji Kasneci

机构 * TUM（慕尼黑大学）； TUB（慕尼黑大学）

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL；LLM(comments)

AI总结 GermanPartiesQA研究了六个商业LLMs在政治倾向和趋炎附势方面的表现，揭示了LLMs在生成事实性政党立场上的局限性以及模型特定的意识形态倾向模式。

Comments Published at AAAI/ACM AIES 2025. Presented at NeurIPS 2025 Workshop on LLM Evaluation and the International Monetary Fund's 12th Statistical Forum. GermanPartiesQA Benchmark under https://github.com/janbatzner/germanpartiesqa

Journal ref Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, 8(1), 2025, pp. 330-342

2501.00593 2025-01-07 cs.CL 88%

Setting Standards in Turkish NLP: TR-MMLU for Large Language Model Evaluation

M. Ali Bayram, Ali Arda Fincan, Ahmet Semih Gümüş, Banu Diri, Savaş Yıldırım, Öner Aytaş

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL；LLM(comments)

Comments 6 pages, 2 tables, submitted to arXiv for review. Includes a comprehensive evaluation framework for Turkish NLP tasks and state-of-the-art LLM evaluations

2408.14964 2024-08-28 cs.LG 88%

Cross-Modal Learning for Chemistry Property Prediction: Large Language Models Meet Graph Machine Learning

Sakhinana Sagar Srinivas, Venkataramana Runkana

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.LG；foundation model(comments)

Comments Paper Accepted at Workshop on Robustness of Few-shot and Zero-shot Learning in Foundation Models at NeurIPS 2023

2311.18215 2023-12-01 cs.CL 88%

Automatic Construction of a Korean Toxic Instruction Dataset for Ethical Tuning of Large Language Models

Sungjoo Byun, Dongjun Jang, Hyemi Jo, Hyopil Shin

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL；instruction tuning(comments)

Comments NeurIPS 2023 Workshop on Instruction Tuning and Instruction Following

2310.02457 2023-11-16 cs.CL cs.CY 88%

The Empty Signifier Problem: Towards Clearer Paradigms for Operationalising "Alignment" in Large Language Models

Hannah Rose Kirk, Bertie Vidgen, Paul Röttger, Scott A. Hale

专题命中评测与基准：language model(title,abstract)；large language model(title,abstract)；分类 cs.CL

Comments Socially Responsible Language Modelling Research (SoLaR) @ NeurIPs 2023

2310.20111 2023-11-01 cs.CL 88%

Making Large Language Models Better Data Creators

Dong-Ho Lee, Jay Pujara, Mohit Sewak, Ryen W. White, Sujay Kumar Jauhar

专题命中评测与基准：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL；LLM(comments)

Comments Accepted to EMNLP 2023 main conference. 12 pages, 5 figures, 6 tables. Code is available at https://github.com/microsoft/llm-data-creation

2608.21206 2026-08-24 cs.CL cs.AI cs.LG 新提交 88%

No PUN Intended: Plausible Unknown Names for Person-Centred LLM Evaluation

并非双关：面向以人为中心的大语言模型评估的合理未知姓名（PUN）

Dimitri Staufer, David Hartmann, Ibrahim Baroud

机构 * Technische Universität Berlin（柏林工业大学）； Weizenbaum Institute for the Networked Society（魏茨曼网络社会研究所）； German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）

专题命中评测与基准：LLM(title,summary_cn)；分类 cs.CL、cs.AI、cs.LG

AI总结针对LLM评估中人名使用导致的测量混淆问题，提出PUN协议构建合理未知姓名，经204人研究验证其有效性并发布300个对照姓名。

Comments Under review

2608.19760 2026-08-21 cs.LG cs.AI cs.CL 新提交 88%

Credit Without Ground Truth: Auditing Step-Level Credit Assignment in LLM Agents Against Executed Replay

无真实值的信用：针对执行重放的大语言模型智能体的步骤级信用分配审计

Haiyue Zhang

机构 * University of Southern California（南加州大学）

专题命中评测与基准：LLM(title,summary_cn)；分类 cs.CL、cs.AI、cs.LG

AI总结该研究针对LLM智能体，在ALFWorld环境中审计步骤级信用分配，发现现有信用信号无法识别关键步骤，提出需匹配有效样本量比较信用规则。

Comments 49 pages, 7 figures. Pre-registered; frozen analysis plans and prompts included in the appendices. Under review

2608.10213 2026-08-12 cs.CE 新提交 88%

VeriFin: A Neurosymbolic Framework for Verifying LLM-Generated Financial Claims

VeriFin：用于验证大语言模型生成的财务声明的神经符号框架

Bethel Hall, Sachi Shome, William Eiers

专题命中评测与基准：LLM(title,summary_cn)；large language model(abstract)；language model(abstract)

AI总结研究针对LLM生成财务声明易出错的问题，提出神经符号框架VeriFin，基于XBRL事实与Z3验证，在XBRLFiling和FinanceBench基准上实现零错误接受，还可通过求解器反馈提升修复效果。

Comments 9 pages

2608.04302 2026-08-06 cs.CV cs.IR cs.MM 新提交 88%

CLIP-CC-Bench: Evaluating Paragraph-Level Video Descriptions in Video-Language Models

CLIP-CC-Bench：评估视频语言模型中的段落级视频描述

Mukhtiar Ali, Harsh Dubey, Sugam Mishra, Chulwoo Pack

机构 * South Dakota State University（南达科他州立大学）

专题命中评测与基准：LLM(summary_cn,abstract)；language model(title,abstract)

AI总结 CLIP-CC-Bench是针对视频语言模型的长篇段落级视频描述评估套件，采用多LLM嵌入模型集成与粗细粒度语义匹配方法，评估17种模型填补了现有短片段基准的空白。

Comments Accepted and presented at EvalMG 2026, the Second Workshop on Evaluation for Multimodal Generation, co-located with ACM SIGIR 2026

2605.06307 2026-08-06 cs.HC 版本更新 88%

LLM-Based Educational Simulation: Evaluating Temporal Student Persona Stability Across ADHD Profiles

基于LLM的教育模拟：评估跨ADHD类型的时间学生人格稳定性

Jana Gonnermann-Müller, Jennifer Haase, Nicolas Leins, Thomas Kosch, Sebastian Pokutta

专题命中评测与基准：LLM(title,title_cn)；large language model(abstract)；language model(abstract)

AI总结本研究通过双重评估框架，测试大型语言模型在长时间交互中维持学生人格稳定性的能力，发现结构化交互设计可消除行为漂移，对教师培训和自适应辅导有重要意义。

2607.29252 2026-08-03 cs.CL cs.AI cs.LG 新提交 88%

CalibratedRubric: Task-Adaptive Rubric Banks for Open-Ended LLM Evaluation

CalibratedRubric：面向开放式大语言模型评估的任务自适应评分规则库

Mengting Chen, Yanshu Sun, Wanting Liang, Beidi Luan, Rui Sun, Dezhi Chen, Jing Li, Zuo Bai

专题命中评测与基准：LLM(title,summary_cn)；分类 cs.CL、cs.AI、cs.LG

AI总结 CalibratedRubric是结合特定类型评分、贝叶斯过滤与IRT的任务自适应框架，可提升开放式LLM评估的人工-黄金标准一致性与排序保真度，减少所需评分规则数量。

2410.06458 2026-07-31 cs.CL cs.AI cs.LG 版本更新 88%

LLM Self-Correction with DeCRIM: Decompose, Critique, and Refine for Enhanced Following of Instructions with Multiple Constraints

基于DeCRIM的大语言模型自校正：分解、批判与优化，提升多约束指令遵循能力

Thomas Palmeira Ferraz, Kartik Mehta, Yu-Hsiang Lin, Haw-Shiuan Chang, Shereen Oraby, Sijia Liu, Vivek Subramanian, Tagyoung Chung, Mohit Bansal, Nanyun Peng

专题命中评测与基准：LLM(title,summary_cn)；分类 cs.CL、cs.AI、cs.LG

AI总结针对LLM处理多约束指令的不足，研究推出RealInstruct基准，提出DeCRIM自校正流程，可提升开源模型性能，结合强反馈时其表现能超越GPT-4。

Comments EMNLP 2024, see https://aclanthology.org/2024.findings-emnlp.458/

2607.20485 2026-07-24 cs.AI cs.CL cs.LG 新提交 88%

Expectation Alignment of Language Models for Real-World User Expectations

语言模型与现实世界用户期望的期望对齐

Miaomiao Li, Yang Wang, Bin Liang, Shudong Liu, Zhiwei Zhang, Kam-Fai Wong

专题命中评测与基准：language model(title,abstract)；LLM(abstract,abstract_cn)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结研究LLMs是否满足用户期望，提出提取期望程序并引入ExpectBench基准，分析发现LLMs存在问题，进而提出LENS框架，可让模型内化期望以生成更契合的响应，凸显明确建模用户期望对实现现实人机对齐的重要性。

Comments Accepted by ICML 2026

2607.10080 2026-07-14 cs.IR 新提交 88%

The Effect of Multi-Lingual and Keyword Adversarial Injection on LLM Relevance Judgment

多语言和关键词对抗性注入对大语言模型相关性判断的影响

Nguyen Khoi Vo, Duy Duong Tuong, Oleg Zendel, Mark Sanderson

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

AI总结研究跨语言提示注入攻击对基于大语言模型的相关性判断的影响，利用TREC数据集和开放权重模型在多语言环境下研究不同注入策略，发现多语言查询注入有效且能绕过现有防御，凸显当前防御差距，强调需更强大评估框架。

2606.29033 2026-07-09 cs.IR 版本更新 88%

Human-in-the-Loop Nugget Annotation for Accountable LLM-as-a-Judge Evaluations

人在环路的金块标注：用于可问责的LLM作为评判者评估

Laura Dietz

专题命中评测与基准：LLM(title,title_cn)

AI总结提出一种原型标注工具，通过人类识别信息金块、LLM进行匹配的分工方式，实现高效且可问责的AI系统评估。

2607.04061 2026-07-07 cs.CL cs.AI cs.LG stat.ML 新提交 88%

Telescope: Improving Zero Shot Detection of LLM Generated Content By Measuring Token Repetition Probability

望远镜：通过测量令牌重复概率改进大语言模型生成内容的零样本检测

Christopher Nassif, Josh F. Cooper

专题命中评测与基准：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结研究区分大语言模型生成文本与人类写作的难题，提出用望远镜困惑度衡量模型令牌重复，揭示该特征在预训练早期出现，能有效进行零样本检测，性能优于其他方法。

Comments 50 pages, ICML, 20 figures, Equal contribution

1. 评测与基准 32096 篇

Generating High-Precision Feedback for Programming Syntax Errors using Large Language Models

TabLLM: Few-shot Classification of Tabular Data with Large Language Models

nl2spec: Interactively Translating Unstructured Natural Language to Temporal Logics with Large Language Models

CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis

Robot Behavior-Tree-Based Task Generation with Large Language Models

Sentence Simplification via Large Language Models

Learning Translation Quality Evaluation on Low Resource Languages from Large Language Models

Training Compute-Optimal Large Language Models

Selection Collider Bias in Large Language Models

Selection Bias Induced Spurious Correlations in Large Language Models

Self-Improvement of Large Language Models: A Technical Overview and Future Outlook

Quantifying Conversational Reliability of Large Language Models under Multi-Turn Interaction

A large-scale evaluation of commonsense knowledge in humans and large language models

GermanPartiesQA: Benchmarking Commercial Large Language Models and AI Companions for Political Alignment and Sycophancy

Setting Standards in Turkish NLP: TR-MMLU for Large Language Model Evaluation

Cross-Modal Learning for Chemistry Property Prediction: Large Language Models Meet Graph Machine Learning

Automatic Construction of a Korean Toxic Instruction Dataset for Ethical Tuning of Large Language Models

The Empty Signifier Problem: Towards Clearer Paradigms for Operationalising "Alignment" in Large Language Models

Making Large Language Models Better Data Creators

No PUN Intended: Plausible Unknown Names for Person-Centred LLM Evaluation

Credit Without Ground Truth: Auditing Step-Level Credit Assignment in LLM Agents Against Executed Replay

VeriFin: A Neurosymbolic Framework for Verifying LLM-Generated Financial Claims

CLIP-CC-Bench: Evaluating Paragraph-Level Video Descriptions in Video-Language Models

LLM-Based Educational Simulation: Evaluating Temporal Student Persona Stability Across ADHD Profiles

CalibratedRubric: Task-Adaptive Rubric Banks for Open-Ended LLM Evaluation

LLM Self-Correction with DeCRIM: Decompose, Critique, and Refine for Enhanced Following of Instructions with Multiple Constraints

Expectation Alignment of Language Models for Real-World User Expectations

The Effect of Multi-Lingual and Keyword Adversarial Injection on LLM Relevance Judgment

Human-in-the-Loop Nugget Annotation for Accountable LLM-as-a-Judge Evaluations

Telescope: Improving Zero Shot Detection of LLM Generated Content By Measuring Token Repetition Probability