大模型推理能力

Turbulence: Systematically and Automatically Testing Instruction-Tuned Large Language Models for Code

Shahin Honarvar, Mark van der Wilk, Alastair Donaldson

机构 * Imperial College London（伦敦帝国学院） ； University of Oxford（牛津大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments Updated to the ICST2025 conference version

Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generation

Satyapriya Krishna, Kalpesh Krishna, Anhad Mohananey, Steven Schwarcz, Adam Stambler, Shyam Upadhyay, Manaal Faruqui

机构 * Harvard University（哈佛大学） ； Google DeepMind（谷歌DeepMind） ； Meta 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics (NAACL), 2025

Black-Box Adversarial Attack on Vision Language Models for Autonomous Driving

Lu Wang, Tianyuan Zhang, Yang Qu, Siyuan Liang, Yuwei Chen, Aishan Liu, Xianglong Liu, Dacheng Tao

机构 * Beihang University（北京航空航天大学） ； National University of Singapore（新加坡国立大学） ； Aviation Industry Development Research Center of China（中国航空工业发展研究中心） ； Nanyang Technological University（南洋理工大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

GLAM: Global-Local Variation Awareness in Mamba-based World Model

Qian He, Wenqi Liang, Chunhui Hao, Gan Sun, Jiandong Tian

专题命中 推理评测 ：reasoning(abstract)；分类 cs.LG

Customizing Language Models with Instance-wise LoRA for Sequential Recommendation

Xiaoyu Kong, Jiancan Wu, An Zhang, Leheng Sheng, Hui Lin, Xiang Wang, Xiangnan He

机构 * University of Science and Technology of China（中国科学技术大学） ； National University of Singapore（新加坡国立大学） ； Electronic Science Research Institute of China Electronics Technology Group Corporation（中国电子科技集团公司电子科学研究院） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments NeurIPS 2024 poster

Journal ref 38th Conference on Neural Information Processing Systems (NeurIPS 2024)

Learning to Balance Altruism and Self-interest Based on Empathy in Mixed-Motive Games

Fanqi Kong, Yizhe Huang, Song-Chun Zhu, Siyuan Qi, Xue Feng

机构 * BIGAI（北京通用人工智能研究院） ； Peking University（北京大学） ； Tsinghua University（清华大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Advice for Diabetes Self-Management by ChatGPT Models: Challenges and Recommendations

Waqar Hussain, John Grundy

机构 * CSIRO’s Data61（联邦科学与工业研究组织数据61） ； Monash University（莫纳什大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Bilingual Evaluation of Language Models on General Knowledge in University Entrance Exams with Minimal Contamination

Eva Sánchez Salido, Roser Morante, Julio Gonzalo, Guillermo Marco, Jorge Carrillo-de-Albornoz, Laura Plaza, Enrique Amigó, Andrés Fernández, Alejandro Benito-Santos, Adrián Ghajari Espinosa, Victor Fresno

机构 * UNED（国立远程教育大学） ； UNED Research Center in Natural Language Processing and Information Retrieval（国立远程教育大学自然语言处理与信息检索研究中心） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Evaluating Agent-based Program Repair at Google

Pat Rondon, Renyao Wei, José Cambronero, Jürgen Cito, Aaron Sun, Siddhant Sanyam, Michele Tufano, Satish Chandra

机构 * Google（谷歌） 纠错

专题命中 推理评测 ：planning(abstract)；分类 cs.AI

MAG-V: A Multi-Agent Framework for Synthetic Data Generation and Verification

Saptarshi Sengupta, Harsh Vashistha, Kristal Curtis, Akshay Mallipeddi, Abhinav Mathur, Joseph Ross, Liang Gou

机构 * Splunk Inc（Splunk公司） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

FineCops-Ref: A new Dataset and Task for Fine-Grained Compositional Referring Expression Comprehension

Junzhuo Liu, Xuzheng Yang, Weiwei Li, Peng Wang

机构 * University of Electronic Science and Technology of China（电子科技大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 18 pages, EMNLP 2024 main

JourneyBench: A Challenging One-Stop Vision-Language Understanding Benchmark of Generated Images

Zhecan Wang, Junzhang Liu, Chia-Wei Tang, Hani Alomari, Anushka Sivakumar, Rui Sun, Wenhao Li, Md. Atabuzzaman, Hammad Ayyubi, Haoxuan You, Alvi Ishmam, Kai-Wei Chang, Shih-Fu Chang, Chris Thomas

机构 * Columbia University（哥伦比亚大学） ； UCLA（加州大学洛杉矶分校） ； Virginia Tech（弗吉尼亚理工大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

CoMAL: Collaborative Multi-Agent Large Language Models for Mixed-Autonomy Traffic

Huaiyuan Yao, Longchao Da, Vishnu Nandam, Justin Turnau, Zhiwei Liu, Linsey Pang, Hua Wei

机构 * Arizona State University（亚利桑那州立大学） ； Salesforce（赛富时） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments 8 pages, 4 figures, accepted to SDM25

ARC Prize 2024: Technical Report

Francois Chollet, Mike Knoop, Gregory Kamradt, Bryan Landers

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Online Reinforcement Learning-Based Dynamic Adaptive Evaluation Function for Real-Time Strategy Tasks

Weilong Yang, Jie Zhang, Xunyun Liu, Yanqing Ye

专题命中 推理评测 ：planning(abstract)；分类 cs.AI

Comments 22 pages, 9 figures

Tree-based RAG-Agent Recommendation System: A Case Study in Medical Test Data

Yahe Yang, Chengyue Huang

机构 * George Washington University（乔治·华盛顿大学） ； University of Iowa（爱荷华大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

AVTrustBench: Assessing and Enhancing Reliability and Robustness in Audio-Visual LLMs

Sanjoy Chowdhury, Sayan Nag, Subhrajyoti Dasgupta, Yaoting Wang, Mohamed Elhoseiny, Ruohan Gao, Dinesh Manocha

机构 * University of Maryland（马里兰大学） ； University of Toronto（多伦多大学） ； Mila（米拉研究所） ； Université de Montréal（蒙特利尔大学） ； KAUST（阿卜杜拉国王科技大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Multimodal Urban Areas of Interest Generation via Remote Sensing Imagery and Geographical Prior

Chuanji Shi, Yingying Zhang, Jiaotuan Wang, Xin Guo, Qiqi Zhu

专题命中 推理评测 ：planning(abstract)；分类 cs.AI

Comments 9 pages, 9 figures

Journal ref International Journal of Applied Earth Observation and Geoinformation, 136(2025)

Evaluating the Capabilities of Large Language Models for Multi-label Emotion Understanding

Tadesse Destaw Belay, Israel Abebe Azime, Abinew Ali Ayele, Grigori Sidorov, Dietrich Klakow, Philipp Slusallek, Olga Kolesnikova, Seid Muhie Yimam

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments COLING 2025, main conference, long

EventSum: A Large-Scale Event-Centric Summarization Dataset for Chinese Multi-News Documents

Mengna Zhu, Kaisheng Zeng, Mao Wang, Kaiming Xiao, Lei Hou, Hongbin Huang, Juanzi Li

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Extended version for paper accepted to AAAI 2025

Reflections from the 2024 Large Language Model (LLM) Hackathon for Applications in Materials Science and Chemistry

Yoel Zimmermann, Adib Bazgir, Zartashia Afzal, Fariha Agbere, Qianxiang Ai, Nawaf Alampara, Alexander Al-Feghali, Mehrad Ansari, Dmytro Antypov, Amro Aswad, Jiaru Bai, Viktoriia Baibakova, Devi Dutta Biswajeet, Erik Bitzek, Joshua D. Bocarsly, Anna Borisova, Andres M Bran, L. Catherine Brinson, Marcel Moran Calderon, Alessandro Canalicchio, Victor Chen, Yuan Chiang, Defne Circi, Benjamin Charmes, Vikrant Chaudhary, Zizhang Chen, Min-Hsueh Chiu, Judith Clymo, Kedar Dabhadkar, Nathan Daelman, Archit Datar, Wibe A. de Jong, Matthew L. Evans, Maryam Ghazizade Fard, Giuseppe Fisicaro, Abhijeet Sadashiv Gangan, Janine George, Jose D. Cojal Gonzalez, Michael Götte, Ankur K. Gupta, Hassan Harb, Pengyu Hong, Abdelrahman Ibrahim, Ahmed Ilyas, Alishba Imran, Kevin Ishimwe, Ramsey Issa, Kevin Maik Jablonka, Colin Jones, Tyler R. Josephson, Greg Juhasz, Sarthak Kapoor, Rongda Kang, Ghazal Khalighinejad, Sartaaj Khan, Sascha Klawohn, Suneel Kuman, Alvin Noe Ladines, Sarom Leang, Magdalena Lederbauer, Sheng-Lun, Liao, Hao Liu, Xuefeng Liu, Stanley Lo, Sandeep Madireddy, Piyush Ranjan Maharana, Shagun Maheshwari, Soroush Mahjoubi, José A. Márquez, Rob Mills, Trupti Mohanty, Bernadette Mohr, Seyed Mohamad Moosavi, Alexander Moßhammer, Amirhossein D. Naghdi, Aakash Naik, Oleksandr Narykov, Hampus Näsström, Xuan Vu Nguyen, Xinyi Ni, Dana O'Connor, Teslim Olayiwola, Federico Ottomano, Aleyna Beste Ozhan, Sebastian Pagel, Chiku Parida, Jaehee Park, Vraj Patel, Elena Patyukova, Martin Hoffmann Petersen, Luis Pinto, José M. Pizarro, Dieter Plessers, Tapashree Pradhan, Utkarsh Pratiush, Charishma Puli, Andrew Qin, Mahyar Rajabi, Francesco Ricci, Elliot Risch, Martiño Ríos-García, Aritra Roy, Tehseen Rug, Hasan M Sayeed, Markus Scheidgen, Mara Schilling-Wilhelmi, Marcel Schloz, Fabian Schöppach, Julia Schumann, Philippe Schwaller, Marcus Schwarting, Samiha Sharlin, Kevin Shen, Jiale Shi, Pradip Si, Jennifer D'Souza, Taylor Sparks, Suraj Sudhakar, Leopold Talirz, Dandan Tang, Olga Taran, Carla Terboven, Mark Tropin, Anastasiia Tsymbal, Katharina Ueltzen, Pablo Andres Unzueta, Archit Vasan, Tirtha Vinchurkar, Trung Vo, Gabriel Vogel, Christoph Völker, Jan Weinreich, Faradawn Yang, Mohd Zaki, Chi Zhang, Sylvester Zhang, Weijie Zhang, Ruijie Zhu, Shang Zhu, Jan Janssen, Calvin Li, Ian Foster, Ben Blaiszik

专题命中 推理评测 ：reasoning(abstract)；分类 cs.LG

Comments Updating author information, the submission remains largely unchanged. 98 pages total

Decoding the Flow: CauseMotion for Emotional Causality Analysis in Long-form Conversations

Yuxuan Zhang, Yulong Li, Zichen Yu, Feilong Tang, Zhixiang Lu, Chong Li, Kang Dang, Jionglong Su

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 7pages

Audiopedia: Audio QA with Knowledge

Abhirama Subramanyam Penamakuri, Kiran Chhatre, Akshat Jain

专题命中 推理评测 ：reasoning(abstract)；分类 cs.LG

Comments Accepted to ICASSP 2025

A Comprehensive Survey of Large Language Models and Multimodal Large Language Models in Medicine

Hanguang Xiao, Feizhong Zhou, Xingyue Liu, Tianqi Liu, Zhipeng Li, Xin Liu, Xiaoxuan Huang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Journal ref Information Fusion, 117 (2025) 102888

Expand VSR Benchmark for VLLM to Expertize in Spatial Rules

Peijin Xie, Lin Sun, Bingquan Liu, Dexin Wang, Xiangzheng Zhang, Chengjie Sun, Jiajia Zhang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Molly: Making Large Language Model Agents Solve Python Problem More Logically

Rui Xiao, Jiong Wang, Lu Han, Na Zong, Han Wu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments arXiv admin note: text overlap with arXiv:2402.07913

Enhancing Knowledge Distillation for LLMs with Response-Priming Prompting

Vijay Goyal, Mustafa Khan, Aprameya Tirupati, Harveer Saini, Michael Lam, Kevin Zhu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted to SoCal NLP Symposium 2024

Counting-Stars: A Multi-evidence, Position-aware, and Scalable Benchmark for Evaluating Long-Context Large Language Models

Mingyang Song, Mao Zheng, Xuan Luo

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted by COLING 2025

Cross-Lingual Text-Rich Visual Comprehension: An Information Theory Perspective

Xinmiao Yu, Xiaocheng Feng, Yun Li, Minghui Liao, Ya-Qi Yu, Xiachong Feng, Weihong Zhong, Ruihan Chen, Mengkang Hu, Jihao Wu, Dandan Tu, Duyu Tang, Bing Qin

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

SCBench: A Sports Commentary Benchmark for Video LLMs

Kuangzhi Ge, Lingjun Chen, Kevin Zhang, Yulin Luo, Tianyu Shi, Liaoyuan Fan, Xiang Li, Guanqun Wang, Shanghang Zhang

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.AI