大模型推理能力

Attention Heads of Large Language Models: A Survey

Zifan Zheng, Yezhaohui Wang, Yuxin Huang, Shichao Song, Mingchuan Yang, Bo Tang, Feiyu Xiong, Zhiyu Li

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 33 pages, 11 figures, 7 tables, 7 equations

LegalAgentBench: Evaluating LLM Agents in Legal Domain

Haitao Li, Junjie Chen, Jingli Yang, Qingyao Ai, Wei Jia, Youfeng Liu, Kai Lin, Yueyue Wu, Guozhi Yuan, Yiran Hu, Wuyue Wang, Yiqun Liu, Minlie Huang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 23 pages

M3T: Multi-Modal Medical Transformer to bridge Clinical Context with Visual Insights for Retinal Image Medical Description Generation

Nagur Shareef Shaik, Teja Krishna Cherukuri, Dong Hye Ye

专题命中 推理评测 ：planning(abstract)；分类 cs.LG

Comments This paper has been accepted for presentation at the IEEE International Conference on Image Processing (ICIP 2024)

AI-generated Image Quality Assessment in Visual Communication

Yu Tian, Yixuan Li, Baoliang Chen, Hanwei Zhu, Shiqi Wang, Sam Kwong

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments AAAI-2025; Project page: https://github.com/ytian73/AIGI-VC

Automatic Extraction of Metaphoric Analogies from Literary Texts: Task Formulation, Dataset Construction, and Evaluation

Joanne Boisson, Zara Siddique, Hsuvas Borkakoty, Dimosthenis Antypas, Luis Espinosa Anke, Jose Camacho-Collados

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted to COLING 2025, long paper

COLUMBUS: Evaluating COgnitive Lateral Understanding through Multiple-choice reBUSes

Koen Kraaijveld, Yifan Jiang, Kaixin Ma, Filip Ilievski

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments 15 pages, 10 figures, accepted to AAAI-25

ArtAug: Enhancing Text-to-Image Generation through Synthesis-Understanding Interaction

Zhongjie Duan, Qianyi Zhao, Cen Chen, Daoyuan Chen, Wenmeng Zhou, Yaliang Li, Yingda Chen

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments 18 pages, 8 figures

Tangram: Benchmark for Evaluating Geometric Element Recognition in Large Multimodal Models

Chao Zhang, Jiamin Tang, Jing Xiao

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments 12 pages, 8 figures

Benchmarking Table Comprehension In The Wild

Yikang Pan, Yi Zhu, Rand Xie, Yizhi Liu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted at TRL Workshop@Neurips 2024. Link to data https://github.com/boson-ai/Table_eval_public

Evaluation of Code LLMs on Geospatial Code Generation

Piotr Gramacki, Bruno Martins, Piotr Szymański

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 7th ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery (GeoAI'24)

Enhancing Temporal Understanding in Audio Question Answering for Large Audio Language Models

Arvind Krishna Sridhar, Yinyi Guo, Erik Visser

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 9 pages, 6 figures

Hypothetical Minds: Scaffolding Theory of Mind for Multi-Agent Tasks with Large Language Models

Logan Cross, Violet Xiang, Agam Bhatia, Daniel LK Yamins, Nick Haber

专题命中 推理评测 ：planning(abstract)；分类 cs.AI

KULTURE Bench: A Benchmark for Assessing Language Model in Korean Cultural Context

Xiaonan Wang, Jinyoung Yeo, Joon-Ho Lim, Hansaem Kim

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted by the 38th Pacific Asia Conference on Language, Information and Computation

Improving the Natural Language Inference robustness to hard dataset by data augmentation and preprocessing

Zijiang Yang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

From Concept to Manufacturing: Evaluating Vision-Language Models for Engineering Design

Cyril Picard, Kristen M. Edwards, Anna C. Doris, Brandon Man, Giorgio Giannone, Md Ferdous Alam, Faez Ahmed

专题命中 推理评测 ：planning(abstract)；分类 cs.AI

Verbosity $\neq$ Veracity: Demystify Verbosity Compensation Behavior of Large Language Models

Yusen Zhang, Sarkar Snigdha Sarathi Das, Rui Zhang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 22 pages, 7 figures

Text-Tuple-Table: Towards Information Integration in Text-to-Table Generation via Global Tuple Extraction

Zheye Deng, Chunkit Chan, Weiqi Wang, Yuxi Sun, Wei Fan, Tianshi Zheng, Yauwai Yim, Yangqiu Song

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted to EMNLP 2024

INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledge

Angelika Romanou, Negar Foroutan, Anna Sotnikova, Zeming Chen, Sree Harsha Nelaturu, Shivalika Singh, Rishabh Maheshwary, Micol Altomare, Mohamed A. Haggag, Snegha A, Alfonso Amayuelas, Azril Hafizi Amirudin, Viraat Aryabumi, Danylo Boiko, Michael Chang, Jenny Chim, Gal Cohen, Aditya Kumar Dalmia, Abraham Diress, Sharad Duwal, Daniil Dzenhaliou, Daniel Fernando Erazo Florez, Fabian Farestam, Joseph Marvin Imperial, Shayekh Bin Islam, Perttu Isotalo, Maral Jabbarishiviari, Börje F. Karlsson, Eldar Khalilov, Christopher Klamm, Fajri Koto, Dominik Krzemiński, Gabriel Adriano de Melo, Syrielle Montariol, Yiyang Nan, Joel Niklaus, Jekaterina Novikova, Johan Samir Obando Ceron, Debjit Paul, Esther Ploeger, Jebish Purbey, Swati Rajwal, Selvan Sunitha Ravi, Sara Rydell, Roshan Santhosh, Drishti Sharma, Marjana Prifti Skenduli, Arshia Soltani Moakhar, Bardia Soltani Moakhar, Ran Tamir, Ayush Kumar Tarun, Azmine Toushik Wasi, Thenuka Ovin Weerasinghe, Serhan Yilmaz, Mike Zhang, Imanol Schlag, Marzieh Fadaee, Sara Hooker, Antoine Bosselut

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

MaterialBENCH: Evaluating College-Level Materials Science Problem-Solving Abilities of Large Language Models

Michiko Yoshitake, Yuta Suzuki, Ryo Igarashi, Yoshitaka Ushiku, Keisuke Nagato

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Neutralizing Backdoors through Information Conflicts for Large Language Models

Chen Chen, Yuchen Sun, Xueluan Gong, Jiaxin Gao, Kwok-Yan Lam

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

EFSA: Towards Event-Level Financial Sentiment Analysis

Tianyu Chen, Yiming Zhang, Guoxin Yu, Dapeng Zhang, Li Zeng, Qing He, Xiang Ao

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.CL

Synthetic Data Generation with LLM for Improved Depression Prediction

Andrea Kang, Jun Yu Chen, Zoe Lee-Youngzie, Shuhao Fu

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.LG

Comments 6 pages excluding references and appendix

Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey

Jiayi Kuang, Jingyou Xie, Haohao Luo, Ronghao Li, Zhe Xu, Xianfeng Cheng, Yinghui Li, Xika Lin, Ying Shen

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

CoverBench: A Challenging Benchmark for Complex Claim Verification

Alon Jacovi, Moran Ambar, Eyal Ben-David, Uri Shaham, Amir Feder, Mor Geva, Dror Marcus, Avi Caciularu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Huggingface Datasets link: https://huggingface.co/datasets/google/coverbench

Finding Blind Spots in Evaluator LLMs with Interpretable Checklists

Sumanth Doddapaneni, Mohammed Safi Ur Rahman Khan, Sshubam Verma, Mitesh M. Khapra

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments EMNLP 2024

Generating Out-Of-Distribution Scenarios Using Language Models

Erfan Aasi, Phat Nguyen, Shiva Sreeram, Guy Rosman, Sertac Karaman, Daniela Rus

专题命中 推理评测 ：reasoning(abstract)；分类 cs.LG

Multi-ToM: Evaluating Multilingual Theory of Mind Capabilities in Large Language Models

Jayanta Sadhu, Ayan Antik Khan, Noshin Nawal, Sanju Basak, Abhik Bhattacharjee, Rifat Shahriyar

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Enhancing Large Vision Language Models with Self-Training on Image Comprehension

Yihe Deng, Pan Lu, Fan Yin, Ziniu Hu, Sheng Shen, Quanquan Gu, James Zou, Kai-Wei Chang, Wei Wang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 22 pages, 14 figures, 9 tables

FFAA: Multimodal Large Language Model based Explainable Open-World Face Forgery Analysis Assistant

Zhengchao Huang, Bin Xia, Zicheng Lin, Zhun Mou, Wenming Yang, Jiaya Jia

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments 23 pages, 21 figures; project page: https://ffaa-vl.github.io

Multilingual Large Language Models: A Systematic Survey

Shaolin Zhu, Supryadi, Shaoyang Xu, Haoran Sun, Leiyu Pan, Menglong Cui, Jiangcun Du, Renren Jin, António Branco, Deyi Xiong

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL