大模型推理能力

BigDocs: An Open Dataset for Training Multimodal Models on Document and Code Tasks

Juan Rodriguez, Xiangru Jian, Siba Smarak Panigrahi, Tianyu Zhang, Aarash Feizi, Abhay Puri, Akshay Kalkunte, François Savard, Ahmed Masry, Shravan Nayak, Rabiul Awal, Mahsa Massoud, Amirhossein Abaskohi, Zichao Li, Suyuchen Wang, Pierre-André Noël, Mats Leon Richter, Saverio Vadacchino, Shubham Agarwal, Sanket Biswas, Sara Shanian, Ying Zhang, Noah Bolger, Kurt MacDonald, Simon Fauvel, Sathwik Tejaswi, Srinivas Sunkara, Joao Monteiro, Krishnamurthy DJ Dvijotham, Torsten Scholak, Nicolas Chapados, Sepideh Kharagani, Sean Hughes, M. Özsu, Siva Reddy, Marco Pedersoli, Yoshua Bengio, Christopher Pal, Issam Laradji, Spandana Gella, Perouz Taslakian, David Vazquez, Sai Rajeswar

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

Comments The project is hosted at https://bigdocs.github.io

Journal ref ICLR 2025 https://openreview.net/forum?id=UTgNFcpk0j

To Trust or Not to Trust? Enhancing Large Language Models' Situated Faithfulness to External Contexts

Yukun Huang, Sanxing Chen, Hongyi Cai, Bhuwan Dhingra

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

DeepReview: Improving LLM-based Paper Review with Human-like Deep Thinking Process

Minjun Zhu, Yixuan Weng, Linyi Yang, Yue Zhang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

SplitQuantV2: Enhancing Low-Bit Quantization of LLMs Without GPUs

Jaewoo Song, Fangzhen Lin

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

ProJudge: A Multi-Modal Multi-Discipline Benchmark and Instruction-Tuning Dataset for MLLM-based Process Judges

Jiaxin Ai, Pengfei Zhou, Zhaopan Xu, Ming Li, Fanrui Zhang, Zizhen Li, Jianwen Sun, Yukang Feng, Baojin Huang, Zhongyuan Wang, Kaipeng Zhang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Chart-HQA: A Benchmark for Hypothetical Question Answering in Charts

Xiangnan Chen, Yuancheng Fang, Qian Xiao, Juncheng Li, Jun Lin, Siliang Tang, Yi Yang, Yueting Zhuang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Under review

Matrix Factorization for Inferring Associations and Missing Links

Ryan Barron, Maksim E. Eren, Duc P. Truong, Cynthia Matuszek, James Wendelberger, Mary F. Dorn, Boian Alexandrov

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Comments 35 pages, 14 figures, 3 tables, 1 algorithm

HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughly

Howard Yen, Tianyu Gao, Minmin Hou, Ke Ding, Daniel Fleischer, Peter Izsak, Moshe Wasserblat, Danqi Chen

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments ICLR 2025. Project page: https://princeton-nlp.github.io/HELMET/

Keeping Yourself is Important in Downstream Tuning Multimodal Large Language Model

Wenke Huang, Jian Liang, Xianda Guo, Yiyang Fang, Guancheng Wan, Xuankun Rong, Chi Wen, Zekun Shi, Qingyun Li, Didi Zhu, Yanbiao Ma, Ke Liang, Bin Yang, He Li, Jiawei Shao, Mang Ye, Bo Du

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMs

Ved Sirdeshmukh, Kaustubh Deshpande, Johannes Mols, Lifeng Jin, Ed-Yeremai Cardona, Dean Lee, Jeremy Kritz, Willow Primack, Summer Yue, Chen Xing

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Structured Outputs Enable General-Purpose LLMs to be Medical Experts

Guangfu Guo, Kai Zhang, Bryan Hoo, Yujun Cai, Xiaoqian Lu, Nanyun Peng, Yiwei Wang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Exploring Advanced Techniques for Visual Question Answering: A Comprehensive Comparison

Aiswarya Baby, Tintu Thankom Koshy

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Comments 8 pages, No figures

AnnoCaseLaw: A Richly-Annotated Dataset For Benchmarking Explainable Legal Judgment Prediction

Magnus Sesodia, Alina Petrova, John Armour, Thomas Lukasiewicz, Oana-Maria Camburu, Puneet K. Dokania, Philip Torr, Christian Schroeder de Witt

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Teach-to-Reason with Scoring: Self-Explainable Rationale-Driven Multi-Trait Essay Scoring

Heejin Do, Sangwon Ryu, Gary Geunbae Lee

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Expertise Is What We Want

Alan Ashworth, Munir Al-Dajani, Keegan Duchicela, Kiril Kafadarov, Allison Kurian, Othman Laraki, Amina Lazrak, Divneet Mandair, Wendy McKennon, Rebecca Miksad, Jayodita Sanghvi, Travis Zack

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 18 pages, 7 figures, 5 tables

Do Large Language Models Align with Core Mental Health Counseling Competencies?

Viet Cuong Nguyen, Mohammad Taher, Dongwan Hong, Vinicius Konkolics Possobom, Vibha Thirunellayi Gopalakrishnan, Ekta Raj, Zihang Li, Heather J. Soled, Michael L. Birnbaum, Srijan Kumar, Munmun De Choudhury

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 10 Pages, Accepted to Findings of NAACL 2025

When Graph meets Multimodal: Benchmarking and Meditating on Multimodal Attributed Graphs Learning

Hao Yan, Chaozhuo Li, Jun Yin, Zhigang Yu, Weihao Han, Mingzheng Li, Zhengxin Zeng, Hao Sun, Senzhang Wang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

MMEvalPro: Calibrating Multimodal Benchmarks Towards Trustworthy and Efficient Evaluation

Jinsheng Huang, Liang Chen, Taian Guo, Fu Zeng, Yusheng Zhao, Bohan Wu, Ye Yuan, Haozhe Zhao, Zhihui Guo, Yichi Zhang, Jingyang Yuan, Wei Ju, Luchen Liu, Tianyu Liu, Baobao Chang, Ming Zhang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 18 pages, code released at https://github.com/chenllliang/MMEvalPro, Homepage at https://mmevalpro.github.io/

HealthQ: Unveiling Questioning Capabilities of LLM Chains in Healthcare Conversations

Ziyu Wang, Hao Li, Di Huang, Hye-Sung Kim, Chae-Won Shin, Amir M. Rahmani

专题命中 推理评测 ：CoT(abstract)；分类 cs.CL、cs.LG

DeltaLLM: Compress LLMs with Low-Rank Deltas between Shared Weights

Liana Mikaelyan, Ayyoob Imani, Mathew Salvaris, Parth Pathak, Mohsen Fayyaz

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

FinLLM-B: When Large Language Models Meet Financial Breakout Trading

Kang Zhang, Osamu Yoshie, Lichao Sun, Weiran Huang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted by NAACL 2025

Generalizing From Short to Long: Effective Data Synthesis for Long-Context Instruction Tuning

Wenhao Zhu, Pinzhen Chen, Hanxu Hu, Shujian Huang, Fei Yuan, Jiajun Chen, Alexandra Birch

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

M-MAD: Multidimensional Multi-Agent Debate for Advanced Machine Translation Evaluation

Zhaopeng Feng, Jiayuan Su, Jiamei Zheng, Jiahan Ren, Yan Zhang, Jian Wu, Hongwei Wang, Zuozhu Liu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Code and data are available at https://github.com/SU-JIAYUAN/M-MAD

NYU CTF Bench: A Scalable Open-Source Benchmark Dataset for Evaluating LLMs in Offensive Security

Minghao Shao, Sofija Jancheska, Meet Udeshi, Brendan Dolan-Gavitt, Haoran Xi, Kimberly Milner, Boyuan Chen, Max Yin, Siddharth Garg, Prashanth Krishnamurthy, Farshad Khorrami, Ramesh Karri, Muhammad Shafique

专题命中 推理评测 ：planning(abstract)；分类 cs.AI、cs.LG

Chinese Spelling Correction: A Comprehensive Survey of Progress, Challenges, and Opportunities

Changchun Liu, Kai Zhang, Junzhe Jiang, Zixiao Kong, Qi Liu, Enhong Chen

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

SnipGen: A Mining Repository Framework for Evaluating LLMs for Code

Daniel Rodriguez-Cardenas, Alejandro Velasco, Denys Poshyvanyk

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.AI、cs.LG

Comments 5 pages, 3 figures, 2 tables

Understanding Figurative Meaning through Explainable Visual Entailment

Arkadiy Saakyan, Shreyas Kulkarni, Tuhin Chakrabarty, Smaranda Muresan

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments NAACL 2025 Main Conference

KernelBench: Can LLMs Write Efficient GPU Kernels?

Anne Ouyang, Simon Guo, Simran Arora, Alex L. Zhang, William Hu, Christopher Ré, Azalia Mirhoseini

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Self-Evaluation for Job-Shop Scheduling

Imanol Echeverria, Maialen Murua, Roberto Santana

专题命中 推理评测 ：planning(abstract)；分类 cs.AI、cs.LG

LawGPT: Knowledge-Guided Data Generation and Its Application to Legal LLM

Zhi Zhou, Kun-Yang Yu, Shi-Yu Tian, Xiao-Wen Yang, Jiang-Xin Shi, Pengxiao Song, Yi-Xuan Jin, Lan-Zhe Guo, Yu-Feng Li

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Preprint