语言大模型 / LLM

GEP: A GCG-Based method for extracting personally identifiable information from chatbots built on small language models

Jieli Zhu, Vi Ngoc-Nha Tran

机构 * Department of Computer Science（计算机科学系） ； The Arctic University of Norway（挪威北极大学） 纠错

专题命中 评测与基准 ：language model(title,abstract)；small language model(title,abstract)；large language model(abstract)；SLM(abstract)

Comments 16 pages, 5 figures, 4 tables

Evaluating Open-Source Large Language Models for Technical Telecom Question Answering

Arina Caraus, Alessio Buscemi, Sumit Kumar, Ion Turcanu

机构 * Luxembourg Institute of Science and Technology (LIST)（卢森堡科学与技术研究院） ； RMT Labs（RMT实验室） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Accepted at the IEEE GLOBECOM Workshops 2025: "Large AI Model over Future Wireless Networks"

VocalAgent: Large Language Models for Vocal Health Diagnostics with Safety-Aware Evaluation

Yubin Kim, Taehan Kim, Wonjune Kang, Eugene Park, Joonsik Yoon, Dongjae Lee, Xin Liu, Daniel McDuff, Hyeonhoon Lee, Cynthia Breazeal, Hae Won Park

机构 * Massachusetts Institute of Technology, USA（麻省理工学院, 美国） ； Seoul National University Hospital, South Korea（首尔国立大学医院, 韩国） ； Doctor Diary, South Korea（Doctor Diary, 韩国） ； Google Research, USA（谷歌研究, 美国） ； Independent Researcher（独立研究者） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

Comments Accepted by Proceedings of Interspeech 2025; Website: https://han811.github.io/VocalAgent2025/

DivLogicEval: A Framework for Benchmarking Logical Reasoning Evaluation in Large Language Models

Tsz Ting Chung, Lemao Liu, Mo Yu, Dit-Yan Yeung

机构 * The Hong Kong University of Science and Technology（香港科技大学） ； Fudan University（复旦大学） ； WeChat AI, Tencent（腾讯微信AI） 纠错

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted by EMNLP 2025. Project Page: https://ttchungc.github.io/projects/divlogiceval/

Safety and Security Analysis of Large Language Models: Benchmarking Risk Profile and Harm Potential

Charankumar Akiri, Harrison Simpson, Kshitiz Aryal, Aarav Khanna, Maanak Gupta

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

SPELUNKER: Item Similarity Search Using Large Language Models and Custom K-Nearest Neighbors

Ana Rodrigues, João Mata, Rui Rego

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

Comments 6 pages, 4 figures

MUCAR: Benchmarking Multilingual Cross-Modal Ambiguity Resolution for Multimodal Large Language Models

Xiaolong Wang, Zhaolu Kang, Wangyuxuan Zhai, Xinyue Lou, Yunghwei Lai, Ziyue Wang, Yawen Wang, Kaiyu Huang, Yile Wang, Peng Li, Yang Liu

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

Development and Validation of a Large Language Model for Generating Fully-Structured Radiology Reports

Chuang Niu, Md Sayed Tanveer, Md Zabirul Islam, Parisa Kaviani, Qing Lyu, Mannudeep K. Kalra, Christopher T. Whitlow, Ge Wang

专题命中 评测与基准 ：large language model(title)；language model(title)；LLM(abstract)；分类 cs.CL、cs.AI

TrustJudge: Inconsistencies of LLM-as-a-Judge and How to Alleviate Them

Yidong Wang, Yunze Song, Tingyuan Zhu, Xuanwang Zhang, Zhuohao Yu, Hao Chen, Chiyu Song, Qiufeng Wang, Cunxiang Wang, Zhen Wu, Xinyu Dai, Yue Zhang, Wei Ye, Shikun Zhang

机构 * Peking University（北京大学） ； National University of Singapore（新加坡国立大学） ； Institute of Science Tokyo（东京科学研究院） ； Nanjing University（南京大学） ； Westlake University（西湖大学） ； Southeast University（东南大学） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 22 pages, 9 figures, 6 tables

EmoPrefer: Can Large Language Models Understand Human Emotion Preferences?

Zheng Lian, Licai Sun, Lan Chen, Haoyu Chen, Zebang Cheng, Fan Zhang, Ziyu Jia, Ziyang Ma, Fei Ma, Xiaojiang Peng, Jianhua Tao

专题命中 评测与基准 ：large language model(title)；language model(title)；prompting(abstract)

Do LLM Agents Know How to Ground, Recover, and Assess? A Benchmark for Epistemic Competence in Information-Seeking Agents

Jiaqi Shao, Yuxiang Lin, Munish Prasad Lohani, Yufeng Miao, Bing Luo

机构 * Duke Kunshan University（杜克昆山大学） ； Microsoft AI（微软人工智能） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

FLEXI: Benchmarking Full-duplex Human-LLM Speech Interaction

Yuan Ge, Saihan Chen, Jingqi Xiao, Xiaoqian Liu, Tong Xiao, Yan Xiang, Zhengtao Yu, Jingbo Zhu

机构 * School of Computer Science and Engineering, Northeastern University, Shenyang, China（计算机科学与工程学院，东北大学，沈阳，中国） ； NiuTrans Research（NiuTrans研究） ； Kunming University of Science and Technology（昆明理工大学） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

MotivGraph-SoIQ: Integrating Motivational Knowledge Graphs and Socratic Dialogue for Enhanced LLM Ideation

Xinping Lei, Tong Zhou, Yubo Chen, Kang Liu, Jun Zhao

机构 * The Key Laboratory of Cognition and Decision Intelligence for Complex Systems（认知与决策智能复杂系统重点实验室） ； School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院） ； Hunan Provincial Key Laboratory of Philosophy and Social Sciences of Artificial Intelligence and Precision International, Hunan Normal University（湖南省人工智能与精准哲学社会科学省级重点实验室，湖南师范大学） ； Beijing University of Posts and Telecommunications（北京邮电大学） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments EMNLP2025 Findings

Improving LLM-as-a-Judge Inference with the Judgment Distribution

Victor Wang, Michael J. Q. Zhang, Eunsol Choi

机构 * Department of Computer Science（计算机科学系） ； The University of Texas at Austin（德克萨斯大学奥斯汀分校） ； New York University（纽约大学） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；language model(abstract)；prompting(abstract)；分类 cs.CL

Comments EMNLP 2025 Findings

A Survey on LLM-based Code Generation for Low-Resource and Domain-Specific Programming Languages

Sathvik Joel, Jie JW Wu, Fatemeh H. Fard

机构 * Indian Institute of Technology Madras, Chennai（印度理工学院马德拉斯分校，钦奈） ； University of British Columbia, Kelowna（不列颠哥伦比亚大学，克洛维纳） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

Comments 64 pages, 3 figures, 15 tables. Accepted in ACM Transactions on Software Engineering and Methodology (TOSEM)

SK2Decompile: LLM-based Two-Phase Binary Decompilation from Skeleton to Skin

Hanzhuo Tan, Weihao Li, Xiaolong Tian, Siyi Wang, Jiaming Liu, Jing Li, Yuqun Zhang

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)

Beyond A Single AI Cluster: A Survey of Decentralized LLM Training

Haotian Dong, Jingyan Jiang, Rongwei Lu, Jiajun Luo, Jiajun Song, Bowen Li, Ying Shen, Zhi Wang

专题命中 评测与基准 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)

Comments EMNLP 2025

LLM-Based Support for Diabetes Diagnosis: Opportunities, Scenarios, and Challenges with GPT-5

Gaurav Kumar Gupta, Nirajan Acharya, Pranal Pande

机构 * Youngstown State University（扬斯敦州立大学） 纠错

专题命中 评测与基准 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

Texture or Semantics? Vision-Language Models Get Lost in Font Recognition

Zhecheng Li, Guoxian Song, Yujun Cai, Zhen Xiong, Junsong Yuan, Yiwei Wang

机构 * University of California, San Diego（加州大学圣地亚哥分校） ； ByteDance（字节跳动） ； The University of Queensland（昆士兰大学） ； University of Southern California（南加州大学） ； University at Buffalo（布法罗大学） ； University of California, Merced（加州大学默塞德分校） 纠错

专题命中 评测与基准 ：language model(title,abstract)；prompting(abstract)；分类 cs.CL

Comments Accepted to COLM 2025

Retrieval-Augmented Guardrails for AI-Drafted Patient-Portal Messages: Error Taxonomy Construction and Large-Scale Evaluation

Wenyuan Chen, Fateme Nateghi Haredasht, Kameron C. Black, Francois Grolleau, Emily Alsentzer, Jonathan H. Chen, Stephen P. Ma

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Persona-Augmented Benchmarking: Evaluating LLMs Across Diverse Writing Styles

Kimberly Le Truong, Riccardo Fogliato, Hoda Heidari, Zhiwei Steven Wu

机构 * Carnegie Mellon University（卡内基梅隆大学） ； Amazon AWS（亚马逊AWS） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments Accepted to EMNLP 2025

EgoInstruct: An Egocentric Video Dataset of Face-to-face Instructional Interactions with Multi-modal LLM Benchmarking

Yuki Sakai, Ryosuke Furuta, Juichun Yen, Yoichi Sato

机构 * The University of Tokyo（东京大学） 纠错

专题命中 评测与基准 ：LLM(title)；large language model(abstract)；language model(abstract)

Comments Accepted to the I-HFM Workshop at ICCV 2025

Towards Transparent AI: A Survey on Explainable Language Models

Avash Palikhe, Zichong Wang, Zhipeng Yin, Rui Guo, Qiang Duan, Jie Yang, Wenbin Zhang

机构 * Florida International University（佛罗里达国际大学） ； University of Florida（佛罗里达大学） ； Pennsylvania State University（宾夕法尼亚州立大学） ； University of Wollongong（沃林根大学） 纠错

专题命中 评测与基准 ：language model(title,abstract)；分类 cs.CL

LLM Agent Meets Agentic AI: Can LLM Agents Simulate Customers to Evaluate Agentic-AI-based Shopping Assistants?

Lu Sun, Shihan Fu, Bingsheng Yao, Yuxuan Lu, Wenbo Li, Hansu Gu, Jiri Gesi, Jing Huang, Chen Luo, Dakuo Wang

机构 * University of California San Diego La Jolla California United States ； Northeastern University Boston Massachusetts United States ； North Carolina State University Raleigh North Carolina United States ； Independent Researcher Seattle Washington United States ； Independent Researcher Palo Alto California United States ； University of California San Diego ； Northeastern University ； North Carolina State University ； Independent Researcher 纠错

专题命中 评测与基准 ：LLM(title,abstract)；分类 cs.CL

VISION: Prompting Ocean Vertical Velocity Reconstruction from Incomplete Observations

Yuan Gao, Hao Wu, Qingsong Wen, Kun Wang, Xian Wu, Xiaomeng Huang

机构 * Tsinghua University（清华大学） ； Squirrel Ai Learning ； Nanyang Technological University（南洋理工大学） ； Tencent（腾讯） 纠错

专题命中 评测与基准 ：prompting(title,abstract)；分类 cs.LG

DialSim: A Dialogue Simulator for Evaluating Long-Term Multi-Party Dialogue Understanding of Conversational Agents

Jiho Kim, Woosog Chay, Hyeonji Hwang, Daeun Kyung, Hyunseung Chung, Eunbyeol Cho, Yeonsu Kwon, Yohan Jo, Edward Choi

机构 * KAIST（韩国科学技术院） ； SNU（釜山大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Color Names in Vision-Language Models

Alexandra Gomez-Villa, Pablo Hernández-Cámara, Muhammad Atif Butt, Valero Laparra, Jesus Malo, Javier Vazquez-Corral

专题命中 评测与基准 ：language model(title,abstract)

StockGenChaR: A Study on the Evaluation of Large Vision-Language Models on Stock Chart Captioning

Le Qiu, Emmanuele Chersoni

专题命中 评测与基准 ：language model(title,abstract)

VideoJudge: Bootstrapping Enables Scalable Supervision of MLLM-as-a-Judge for Video Understanding

Abdul Waheed, Zhen Wu, Dareen Alharthi, Seungone Kim, Bhiksha Raj

机构 * Carnegie Mellon University（卡内基梅隆大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Work in progress

"She was useful, but a bit too optimistic": Augmenting Design with Interactive Virtual Personas

Paluck Deep, Monica Bharadhidasan, A. Baki Kocaballi

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments The version accepted for publication at International Journal of Human-Computer Studies

Journal ref International Journal of Human-Computer Studies (2025)