语言大模型 / LLM

SafeMT: Multi-turn Safety for Multimodal Language Models

Han Zhu, Juntao Dai, Jiaming Ji, Haoran Li, Chengkun Cai, Pengcheng Wen, Chi-Min Chan, Boyuan Chen, Yaodong Yang, Sirui Han, Yike Guo

机构 * Hong Kong University of Science and Technology（香港理工大学） ； Peking University（北京大学） ； University of Edinburgh（爱丁堡大学） 纠错

专题命中 评测与基准 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Oyster-I: Beyond Refusal -- Constructive Safety Alignment for Responsible Language Models

Ranjie Duan, Jiexi Liu, Xiaojun Jia, Shiji Zhao, Ruoxi Cheng, Fengxiang Wang, Cheng Wei, Yong Xie, Chang Liu, Defeng Li, Yinpeng Dong, Yichi Zhang, Yuefeng Chen, Chongwen Wang, Xingjun Ma, Xingxing Wei, Yang Liu, Hang Su, Jun Zhu, Xinfeng Li, Yitong Sun, Jie Zhang, Jinzhao Hu, Sha Xu, Wenchao Yang, Yitong Yang, Xingyao Zhang, Yingshui Tan, Jialing Tao, Hui Xue

机构 * Alibaba AAIG（阿里巴巴AAIG） 纠错

专题命中 评测与基准 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments Technical Report Code & Model weights available: https://github.com/Alibaba-AAIG/Oyster

Evaluating Open-Source Vision-Language Models for Multimodal Sarcasm Detection

Saroj Basnet, Shafkat Farabi, Tharindu Ranasinghe, Diptesh Kanoji, Marcos Zampieri

机构 * George Mason University（乔治·马歇尔大学） ； Lancaster University（兰卡斯特大学） ； University of Surrey（萨里大学） 纠错

专题命中 评测与基准 ：language model(title,abstract)；prompting(abstract)；分类 cs.LG

Comments Accepted to ICDMW 2025 Workshop on Multimodal AI (MMAI). Full workshop info: https://icdmw25mmai.github.io/

Journal ref Proc. IEEE International Conference on Data Mining Workshops (ICDMW 2025), Workshop on Multimodal AI (MMAI 2025), Los Angeles, USA, December 2025

Time Travel is Cheating: Going Live with DeepFund for Real-Time Fund Investment Benchmarking

Changlun Li, Yao Shi, Chen Wang, Qiqi Duan, Runke Ruan, Weijie Huang, Haonan Long, Lijun Huang, Nan Tang, Yuyu Luo

机构 * The Hong Kong University of Science and Technology (Guangzhou)（香港理工大学（广州）） ； Paradoox AI Research（Paradoox AI研究院） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments NeurIPS 2025 Datasets and Benchmarks Track

iCodeReviewer: Improving Secure Code Review with Mixture of Prompts

Yun Peng, Kisub Kim, Linghan Meng, Kui Liu

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Language Modeling for the Future of Finance: A Survey into Metrics, Tasks, and Data Opportunities

Nikita Tatarinov, Siddhant Sukhani, Agam Shah, Sudheer Chava

机构 * Georgia Institute of Technology（佐治亚理工学院） 纠错

专题命中 评测与基准 ：language model(title,abstract)；分类 cs.CL

A Survey of Multilingual Reasoning in Language Models

Akash Ghosh, Debayan Datta, Sriparna Saha, Chirag Agarwal

机构 * Indian Institute of Technology Patna（印度理工学院帕纳巴分校） ； University of Virginia（弗吉尼亚大学） 纠错

专题命中 评测与基准 ：language model(title,abstract)；分类 cs.CL

Comments EMNLP Findings 2025

The potential of LLM-generated reports in DevSecOps

Nikolaos Lykousas, Vasileios Argyropoulos, Fran Casino

机构 * Data Centric, Romania（数据导向机构，罗马尼亚） ； Department of Computer Engineering and Mathematics（计算机工程与数学系） 纠错

专题命中 评测与基准 ：LLM(title,abstract)；分类 cs.AI

Comments Published in AIESE 2024 (International Conference on AI empowered Software Engineering)

PHANTOM RECALL: When Familiar Puzzles Fool Smart Models

Souradeep Mukhopadhyay, Rishabh Baral, Nimeesh Mahajan, Samhitha Harish, Aswin RRV, Mihir Parmar, Mutsumi Nakamura, Chitta Baral

机构 * School of Computing and Augmented Intelligence, Arizona State University（计算与增强智能学院，亚利桑那州立大学） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；prompting(abstract)；分类 cs.CL、cs.AI

Comments 22 Pages

Towards General Urban Monitoring with Vision-Language Models: A Review, Evaluation, and a Research Agenda

André Torneiro, Diogo Monteiro, Paulo Novais, Pedro Rangel Henriques, Nuno F. Rodrigues

机构 * ALGORITMI Research Centre/LASI（ALGORITMI研究机构/LASI） ； University of Minho（明霍大学） ； Logimade（Logimade公司） ； INESC TEC（INESC TEC研究机构） ； Ai, School of Technology（2Ai技术学院） ； IPCA 纠错

专题命中 评测与基准 ：language model(title,abstract)

Comments 44 pages

Benchmarking foundation models for hyperspectral image classification: Application to cereal crop type mapping

Walid Elbarz, Mohamed Bourriz, Hicham Hajji, Hamd Ait Abdelali, François Bourzeix

机构 * Analytics Lab (A-Lab), UM6P, Rabat 11103, Morocco（安利实验室（A-Lab）） ； Center for Remote Sensing Applications (CRSA), UM6P, Ben Guerir 43150, Morocco（远程传感应用中心（CRSA）） ； Friedrich Schiller University Jena, Department of Geography, Jena 07743, Germany（耶拿弗里德里希·施勒格尔大学） ； Dept. of Cartography（制图部门） 纠错

专题命中 评测与基准 ：foundation model(title,abstract)

Comments currently being reviewed for WHISPERS conference ( Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing )

ALLSTaR: Automated LLM-Driven Scheduler Generation and Testing for Intent-Based RAN

Maxime Elkael, Michele Polese, Reshma Prasad, Stefano Maxenti, Tommaso Melodia

专题命中 评测与基准 ：LLM(title,abstract)

Comments Under submission to an IEEE journal, copyright may change without notice

Beating Harmful Stereotypes Through Facts: RAG-based Counter-speech Generation

Greta Damo, Elena Cabrio, Serena Villata

机构 * Université Côte d’Azur, CNRS, Inria, I3S, France（法国大学-科蒂-阿祖尔大学、国家科学研究中心、法国国家信息与自动化研究所、I3S研究所） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

DeepMMSearch-R1: Empowering Multimodal LLMs in Multimodal Web Search

Kartik Narayan, Yang Xu, Tian Cao, Kavya Nerella, Vishal M. Patel, Navid Shiee, Peter Grasch, Chao Jia, Yinfei Yang, Zhe Gan

机构 * Johns Hopkins University（约翰霍普金斯大学） ； Apple（苹果公司） 纠错

专题命中 评测与基准 ：LLM(abstract)；large language model(abstract)；language model(abstract)

VQArt-Bench: A semantically rich VQA Benchmark for Art and Cultural Heritage

A. Alfarano, L. Venturoli, D. Negueruela del Castillo

机构 * University of Zurich, Max Planck Society（苏黎世大学、马克斯·普朗克研究所） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Shallow Robustness, Deep Vulnerabilities: Multi-Turn Evaluation of Medical LLMs

Blazej Manczak, Eric Lin, Francisco Eiras, James O' Neill, Vaikkunth Mugunthan

机构 * Dynamo AI 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Dataset and code: https://huggingface.co/datasets/dynamoai-ml/MedQA-USMLE-4-MultiTurnRobust ; https://github.com/bmanczak/MedQA-MultiTurnRobustness Accepted as a poster at NeurIPS 2025 Workshop on GenAI for Health: Potential, Trust, and Policy Compliance

An AI-Based Behavioral Health Safety Filter and Dataset for Identifying Mental Health Crises in Text-Based Conversations

Benjamin W. Nelson, Celeste Wong, Matthew T. Silvestrini, Sooyoon Shin, Alanna Robinson, Jessica Lee, Eric Yang, John Torous, Andrew Trister

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Main Text: 2943; Abstract: 256; Tables and Figures: 5

Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation

Sayash Kapoor, Benedikt Stroebl, Peter Kirgis, Nitya Nadgir, Zachary S Siegel, Boyi Wei, Tianci Xue, Ziru Chen, Felix Chen, Saiteja Utpala, Franck Ndzomga, Dheeraj Oruganty, Sophie Luskin, Kangheng Liu, Botao Yu, Amit Arora, Dongyoon Hahm, Harsh Trivedi, Huan Sun, Juyong Lee, Tengjun Jin, Yifan Mai, Yifei Zhou, Yuxuan Zhu, Rishi Bommasani, Daniel Kang, Dawn Song, Peter Henderson, Yu Su, Percy Liang, Arvind Narayanan

专题命中 评测与基准 ：LLM(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Can ChatGPT support software verification?

Christian Janßen, Cedric Richter, Heike Wehrheim

机构 * Carl-von-Ossietzky Universität Oldenburg（奥尔登堡卡尔·冯·奥西特齐大学） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments accepted at Fundamental Approaches to Software Engineering 2024

MedKGEval: A Knowledge Graph-Based Multi-Turn Evaluation Framework for Open-Ended Patient Interactions with Clinical LLMs

Yuechun Yu, Han Ying, Haoan Jin, Wenjian Jiang, Dong Xian, Binghao Wang, Zhou Yang, Mengyue Wu

机构 * Shanghai Jiao Tong University（上海交通大学） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.AI

AwareCompiler: Agentic Context-Aware Compiler Optimization via a Synergistic Knowledge-Data Driven Framework

Hongyu Lin, Haolin Pan, Haoran Luo, Yuchen Li, Kaichun Yao, Libo Zhang, Mingjie Xing, Yanjun Wu

机构 * University of Chinese Academy of Sciences（中国科学院大学） ； Institute of Software Chinese Academy of Sciences（中国科学院软件研究所） ； Hangzhou Institute for Advanced Study at UCAS（UCAS杭州高等研究 institute） ； Nanyang Technological University（南洋理工大学） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.AI

AFRIDOC-MT: Document-level MT Corpus for African Languages

Jesujoba O. Alabi, Israel Abebe Azime, Miaoran Zhang, Cristina España-Bonet, Rachel Bawden, Dawei Zhu, David Ifeoluwa Adelani, Clement Oyeleke Odoje, Idris Akinade, Iffat Maab, Davis David, Shamsuddeen Hassan Muhammad, Neo Putini, David O. Ademuyiwa, Andrew Caines, Dietrich Klakow

机构 * Masakhane NLP ； Saarland University（萨尔兰大学） ； DFKI GmbH ； Barcelona Supercomputing Center（巴塞罗那超级计算中心） ； Inria（法国国家信息与自动化技术研究院） ； Mila, McGill University & Canada CIFAR AI Chair（Mila，麦吉尔大学及加拿大CIFAR人工智能主席） ； University of Ibadan（伊巴丹大学） ； National Institute of Informatics, Japan（日本国立信息研究所） ； Selcom Tanzania（坦桑尼亚Selcom） ； Imperial College London（伦敦帝国理工学院） ； University of KwaZulu-Natal（南非夸祖鲁-纳塔尔大学） ； Loughborough University（洛桑大学） ； University of Cambridge（剑桥大学） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)；分类 cs.CL

Comments EMNLP 2025

See, Point, Fly: A Learning-Free VLM Framework for Universal Unmanned Aerial Navigation

Chih Yao Hu, Yang-Sen Lin, Yuna Lee, Chih-Hai Su, Jie-Ying Lee, Shr-Ruei Tsai, Chin-Yang Lin, Kuan-Wen Chen, Tsung-Wei Ke, Yu-Lun Liu

机构 * National Yang Ming Chiao Tung University（国家阳明交通大学） ； National Taiwan University（国立台湾大学） 纠错

专题命中 评测与基准 ：language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments CoRL 2025. Project page: https://spf-web.pages.dev

Journal ref Proceedings of The 9th Conference on Robot Learning, PMLR 305:4697-4708, 2025

Towards Engineering Multi-Agent LLMs: A Protocol-Driven Approach

Zhenyu Mao, Jacky Keung, Fengji Zhang, Shuo Liu, Yifei Wang, Jialong Li

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)

A Longitudinal Study on Different Annotator Feedback Loops in Complex RAG Tasks

Sara Rosenthal, Maeda Hanafi, Yannis Katsis, Lucian Popa, Marina Danilevsky

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)

Comments 26 pages

OST-Bench: Evaluating the Capabilities of MLLMs in Online Spatio-temporal Scene Understanding

Jingli Lin, Chenming Zhu, Runsen Xu, Xiaohan Mao, Xihui Liu, Tai Wang, Jiangmiao Pang

机构 * Shanghai AI Laboratory（上海人工智能实验室） ； Shanghai Jiao Tong University（上海交通大学） ； The University of Hong Kong（香港大学） ； The Chinese University of Hong Kong（香港中文大学） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)

Comments 30 pages, a benchmark designed to evaluate Online Spatio-Temporal understanding from the perspective of an agent actively exploring a scene. Project Page: https://rbler1234.github.io/OSTBench.github.io/

CoVLA: Comprehensive Vision-Language-Action Dataset for Autonomous Driving

Hidehisa Arai, Keita Miwa, Kento Sasaki, Yu Yamaguchi, Kohei Watanabe, Shunsuke Aoki, Issei Yamamoto

机构 * Turing Inc.（图灵公司） 纠错

专题命中 评测与基准 ：large language model(abstract)；language model(abstract)

Comments WACV 2025, Project Page: https://turingmotors.github.io/covla-ad/

BoN Appetit Team at LeWiDi-2025: Best-of-N Test-time Scaling Can Not Stomach Annotation Disagreements (Yet)

Tomas Ruiz, Siyao Peng, Barbara Plank, Carsten Schwemmer

机构 * Ludwig Maximilian University of Munich（慕尼黑路德维希-马克西米利安大学） 纠错

专题命中 评测与基准 ：LLM(abstract)；分类 cs.CL、cs.AI

SPADE: Spatial Transcriptomics and Pathology Alignment Using a Mixture of Data Experts for an Expressive Latent Space

Ekaterina Redekop, Mara Pleasure, Zichen Wang, Kimberly Flores, Anthony Sisk, William Speier, Corey W. Arnold

机构 * Biomedical AI Research Lab, University of California, Los Angeles（生物医学人工智能研究实验室，加州大学洛杉矶分校） ； Department of Pathology, University of California, Los Angeles（病理学系，加州大学洛杉矶分校） ； Department of Radiology, University of California, Los Angeles（放射学系，加州大学洛杉矶分校） ； Department of Bioengineering, University of California, Los Angeles（生物工程系，加州大学洛杉矶分校） ； UCLA Medical Informatics Home Area, University of California, Los Angeles（UCLA医学信息学家庭区域，加州大学洛杉矶分校） ； Department of Computational Medicine, University of California, Los Angeles（计算医学系，加州大学洛杉矶分校） 纠错

专题命中 评测与基准 ：foundation model(abstract)；分类 cs.AI、cs.LG

GraphRAG under Fire

Jiacheng Liang, Yuhui Wang, Changjiang Li, Rongyi Zhu, Tanqiu Jiang, Neil Gong, Ting Wang

机构 * Stony Brook University（石溪大学） ； Duke University（杜克大学） 纠错

专题命中 评测与基准 ：language model(abstract)；分类 cs.AI、cs.LG

Comments 13 pages. Accepted by IEEE Symposium on Security and Privacy 2026 (S&P 2026)