语言大模型 / LLM

PhonologyBench: Evaluating Phonological Skills of Large Language Models

Ashima Suvarna, Harshita Khandelwal, Nanyun Peng

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 17 pages, 7 figures, 6 tables

Evaluating the Factuality of Large Language Models using Large-Scale Knowledge Graphs

Xiaoze Liu, Feijie Wu, Tianyang Xu, Zhuo Chen, Yichi Zhang, Xiaoqian Wang, Jing Gao

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigation

Niels Mündler, Jingxuan He, Slobodan Jenko, Martin Vechev

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；prompting(abstract)；分类 cs.CL、cs.AI、cs.LG

Mol-Instructions: A Large-Scale Biomolecular Instruction Dataset for Large Language Models

Yin Fang, Xiaozhuan Liang, Ningyu Zhang, Kangwei Liu, Rui Huang, Zhuo Chen, Xiaohui Fan, Huajun Chen

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；instruction tuning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ICLR 2024. Project homepage: https://github.com/zjunlp/Mol-Instructions

Label-free Node Classification on Graphs with Large Language Models (LLMS)

Zhikai Chen, Haitao Mao, Hongzhi Wen, Haoyu Han, Wei Jin, Haiyang Zhang, Hui Liu, Jiliang Tang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments The code is available via https://github.com/CurryTang/LLMGNN; ICLR 2024

Time Travel in LLMs: Tracing Data Contamination in Large Language Models

Shahriar Golchin, Mihai Surdeanu

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published at ICLR 2024 as a Spotlight paper (notable top 5%)

Natural Language Dataset Generation Framework for Visualizations Powered by Large Language Models

Hyung-Kwon Ko, Hyeon Jeon, Gwanmo Park, Dae Hyun Kim, Nam Wook Kim, Juho Kim, Jinwook Seo

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments 22 pages, 5 figures

Journal ref In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI '24), May 11-16, 2024, Honolulu, HI, USA

Improving Automatic VQA Evaluation Using Large Language Models

Oscar Mañas, Benno Krojer, Aishwarya Agrawal

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at AAAI 2024 (main track)

Self-Evaluation Improves Selective Generation in Large Language Models

Jie Ren, Yao Zhao, Tu Vu, Peter J. Liu, Balaji Lakshminarayanan

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Towards Auditing Large Language Models: Improving Text-based Stereotype Detection

Wu Zekun, Sahan Bulathwela, Adriano Soares Koshiyama

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 2023 NeurIPS SoLaR Workshop Accepted

Instruction-Following Evaluation for Large Language Models

Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, Le Hou

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Long-Horizon Dialogue Understanding for Role Identification in the Game of Avalon with Large Language Models

Simon Stepputtis, Joseph Campbell, Yaqi Xie, Zhengyang Qi, Wenxin Sharon Zhang, Ruiyi Wang, Sanketh Rangreji, Michael Lewis, Katia Sycara

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP, Findings of the Association for Computational Linguistics)

Evaluating Diverse Large Language Models for Automatic and General Bug Reproduction

Sungmin Kang, Juyeon Yoon, Nargiz Askarbekkyzy, Shin Yoo

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments This work is an extension of our prior work, available at arXiv:2209.11515

LLMMaps -- A Visual Metaphor for Stratified Evaluation of Large Language Models

Patrik Puchert, Poonam Poonam, Christian van Onzenoodt, Timo Ropinski

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

End-to-End Speech Recognition Contextualization with Large Language Models

Egor Lakomkin, Chunyang Wu, Yassir Fathullah, Ozlem Kalinli, Michael L. Seltzer, Christian Fuegen

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

SwitchGPT: Adapting Large Language Models for Non-Text Outputs

Xinyu Wang, Bohan Zhuang, Qi Wu

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；pretraining(abstract)

An Effective Data Creation Pipeline to Generate High-quality Financial Instruction Data for Large Language Model

Ziao Wang, Jianning Wang, Junda Wu, Xiaofeng Zhang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；instruction tuning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments (FinLLM 2023)@IJCAI 2023

ChatGPT in the Age of Generative AI and Large Language Models: A Concise Survey

Salman Mohamadi, Ghulam Mujtaba, Ngan Le, Gianfranco Doretto, Donald A. Adjeroh

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments The paper was uploaded in error, before it was ready for submission. The paper requires a deep revision, and significant changes and modifications before uploading to the archive

Large Language Models for Supply Chain Optimization

Beibin Li, Konstantina Mellou, Bo Zhang, Jeevan Pathuri, Ishai Menache

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

covLLM: Large Language Models for COVID-19 Biomedical Literature

Yousuf A. Khan, Clarisse Hokia, Jennifer Xu, Ben Ehlert

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

On the Tool Manipulation Capability of Open-source Large Language Models

Qiantong Xu, Fenglu Hong, Bo Li, Changran Hu, Zhengyu Chen, Jian Zhang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Implicit Identity Technologies for LLMs: Fingerprinting and Watermarking across Datasets, Models, and Generated Content

LLM的隐式身份技术：跨数据集、模型和生成内容的指纹识别与水印

Bing Liu, Shunping Wang, Yufan Zhu, Xinyi Yu, Jing Huang, Linkang Du, Hongbin Pei, Wei Luo

机构 * School of Cyber Science and Engineering, Xi’an Jiaotong University, Xi’an, China（西安交通大学计算机科学与工程学院） ； State Grid Henan Marketing Service Center, Henan, China（国网河南营销服务中心） ； Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China（中国科学院信息工程研究所） ； School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China（中国科学院大学网络安全学院） ； School of Information Technology, Deakin University, Geelong, Australia（迪金大学信息技术学院） 纠错

专题命中 评测与基准 ：LLM(title_cn,summary_cn)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

AI总结 本文综述了LLM指纹识别和水印技术，提出隐式身份统一抽象，并基于生命周期分类法组织数据集、模型和生成内容的技术，建立评估框架。

Comments Accepted by IJCAI-ECAI 2026. 11 pages, 1 figure. Survey and taxonomy of LLM fingerprinting and watermarking for identity, provenance, generated-content attribution, and asset protection

Jailbreak-Zero: A Path to Pareto Optimal Red Teaming for Large Language Models

Jailbreak-Zero：大型语言模型安全评估的帕累托最优渗透测试路径

Kai Hu, Abhinav Aggarwal, Mehran Khodabandeh, David Zhang, Eric Hsin, Li Chen, Ankit Jain, Matt Fredrikson, Akash Bharadwaj

机构 * Meta Superintelligence Labs（Meta超智能实验室） ； Carnegie Mellon University（卡内基梅隆大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

AI总结 Jailbreak-Zero通过生成多样化对抗性提示并微调攻击模型，实现了LLM安全评估的帕累托最优，提高了攻击成功率并减少了人工干预需求。

Comments Socially Responsible and Trustworthy Foundation Models at NeurIPS 2025

Bayesian Evaluation of Large Language Model Behavior

Rachel Longjohn, Shang Wu, Saatvik Kher, Catarina Belém, Padhraic Smyth

机构 * Department of Statistics, University of California, Irvine（统计系，加州大学伊文斯分校） ； Department of Computer Science, University of California, Irvine（计算机科学系，加州大学伊文斯分校） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract,comments)；分类 cs.CL、cs.LG

Comments Accepted to NeurIPS 2025 Workshop on Evaluating the Evolving LLM Lifecycle: Benchmarks, Emergent Abilities, and Scaling

When Text Embedding Meets Large Language Model: A Comprehensive Survey

Zhijie Nie, Zhangchi Feng, Mingxin Li, Cunwang Zhang, Yanzhao Zhang, Dingkun Long, Richong Zhang

机构 * School of Computer Science and Engineering, Beihang University（计算机科学与工程学院，北京航空航天大学） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract,comments)；分类 cs.CL、cs.AI

Comments Version 4: We added the latest works of LLM-based Embedders

A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?

Qiyuan Zhang, Fuyuan Lyu, Zexu Sun, Lei Wang, Weixu Zhang, Wenyue Hua, Haolun Wu, Zhihan Guo, Yufei Wang, Niklas Muennighoff, Irwin King, Xue Liu, Chen Ma

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Comments v3: Expand Agentic and SFT Chapters. Build Website for better visualization

Contextual Evaluation of Large Language Models for Classifying Tropical and Infectious Diseases

Mercy Asiedu, Nenad Tomasev, Chintan Ghate, Tiya Tiyasirichokchai, Awa Dieng, Oluwatosin Akande, Geoffrey Siwo, Steve Adudans, Sylvanus Aitkins, Odianosen Ehiakhamen, Eric Ndombi, Katherine Heller

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments Accepted at 2 NeurIPS 2024 workshops: Generative AI for Health Workshop and Workshop on Advancements In Medical Foundation Models: Explainability, Robustness, Security, and Beyond

Quantifying perturbation impacts for large language models

Paulius Rauba, Qiyao Wei, Mihaela van der Schaar

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

Comments Statistical Foundations of LLMs and Foundation Models Workshop at NeurIPS 2024

Pedestrian Attribute Recognition: A New Benchmark Dataset and A Large Language Model Augmented Framework

Jiandong Jin, Xiao Wang, Qian Zhu, Haiyang Wang, Chenglong Li

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract,comments)；分类 cs.CL、cs.AI

Comments MSP60K PAR Benchmark Dataset, LLM based PAR model, In Peer Review

The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspective

Zhen Qin, Daoyuan Chen, Wenhao Zhang, Liuyi Yao, Yilun Huang, Bolin Ding, Yaliang Li, Shuiguang Deng

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract,comments)；分类 cs.AI、cs.LG

Comments Ongoing work. 21 pages. Related materials are continually maintained and available at https://github.com/modelscope/data-juicer/blob/main/docs/awesome_llm_data.md