大模型推理能力

MAIRA-Seg: Enhancing Radiology Report Generation with Segmentation-Aware Multimodal Large Language Models

Harshita Sharma, Valentina Salvatelli, Shaury Srivastav, Kenza Bouzid, Shruthi Bannur, Daniel C. Castro, Maximilian Ilse, Sam Bond-Taylor, Mercy Prasanna Ranjit, Fabian Falck, Fernando Pérez-García, Anton Schwaighofer, Hannah Richardson, Maria Teodora Wetscherek, Stephanie L. Hyland, Javier Alvarez-Valle

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted as Proceedings Paper at ML4H 2024

Information Anxiety in Large Language Models

Prasoon Bajpai, Sarah Masud, Tanmoy Chakraborty

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Improving LLM Classification of Logical Errors by Integrating Error Relationship into Prompts

Yanggyu Lee, Suchae Jeong, Jihie Kim

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.AI

Comments Published in ITS 2024 (Best Paper Award)

SciInstruct: a Self-Reflective Instruction Annotated Dataset for Training Scientific Language Models

Dan Zhang, Ziniu Hu, Sining Zhoubian, Zhengxiao Du, Kaiyu Yang, Zihan Wang, Yisong Yue, Yuxiao Dong, Jie Tang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted to NeurIPS D&B Track 2024

Responsible AI in Construction Safety: Systematic Evaluation of Large Language Models and Prompt Engineering

Farouq Sammour, Jia Xu, Xi Wang, Mo Hu, Zhenyu Zhang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments 29 pages, 5 figures

General LLMs as Instructors for Domain-Specific LLMs: A Sequential Fusion Method to Integrate Extraction and Editing

Xin Zhang, Tianjie Ju, Huijia Liang, Ying Fu, Qin Zhang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Working in progress

SetLexSem Challenge: Using Set Operations to Evaluate the Lexical and Semantic Robustness of Language Models

Bardiya Akhbari, Manish Gawali, Nicholas A. Dronen

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 10 pages, 8 figures, NeurIPS 2024 Datasets and Benchmarks track

HarmLevelBench: Evaluating Harm-Level Compliance and the Impact of Quantization on Model Alignment

Yannis Belkhiter, Giulio Zizzo, Sergio Maffeis

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments NeurIPS 2024 Workshop on Safe Generative Artificial Intelligence (SafeGenAI)

DetectBench: Can Large Language Model Detect and Piece Together Implicit Evidence?

Zhouhong Gu, Lin Zhang, Xiaoxuan Zhu, Jiangjie Chen, Wenhao Huang, Yikai Zhang, Shusen Wang, Zheyu Ye, Yan Gao, Hongwei Feng, Yanghua Xiao

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments EMNLP Findings 2024

GenAI Arena: An Open Evaluation Platform for Generative Models

Dongfu Jiang, Max Ku, Tianle Li, Yuansheng Ni, Shizhuo Sun, Rongqi Fan, Wenhu Chen

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments 9 pages,7 figures

Journal ref NeurIPS 2024

SciDQA: A Deep Reading Comprehension Dataset over Scientific Papers

Shruti Singh, Nandan Sarkar, Arman Cohan

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 18 pages, Accepted to EMNLP 2024

Multilingual Pretraining Using a Large Corpus Machine-Translated from a Single Source Language

Jiayi Wang, Yao Lu, Maurice Weber, Max Ryabinin, Yihong Chen, Raphael Tang, Pontus Stenetorp

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

CIPHER: Cybersecurity Intelligent Penetration-testing Helper for Ethical Researcher

Derry Pratama, Naufal Suryanto, Andro Aprila Adiputra, Thi-Thu-Huong Le, Ahmada Yusril Kadiptya, Muhammad Iqbal, Howon Kim

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments 28 pages, github available

Journal ref Sensors 2024, 24(21), 6878;

First Multi-Dimensional Evaluation of Flowchart Comprehension for Multimodal Large Language Models

Enming Zhang, Ruobing Yao, Huanyong Liu, Junhui Yu, Jiale Wang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

LifelongMemory: Leveraging LLMs for Answering Queries in Long-form Egocentric Videos

Ying Wang, Yanlai Yang, Mengye Ren

专题命中 推理评测 ：reasoning(abstract)；分类 cs.LG

Internalizing ASR with Implicit Chain of Thought for Efficient Speech-to-Speech Conversational LLM

Robin Shing-Hei Yuen, Timothy Tin-Long Tse, Jian Zhu

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.CL

Comments Updated for reviewer comments

The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale

Guilherme Penedo, Hynek Kydlíček, Loubna Ben allal, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro Von Werra, Thomas Wolf

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

ML Research Benchmark

Matthew Kenney

专题命中 推理评测 ：planning(abstract)；分类 cs.AI

SECURE: Benchmarking Large Language Models for Cybersecurity

Dipkamal Bhusal, Md Tanvirul Alam, Le Nguyen, Ashim Mahara, Zachary Lightcap, Rodney Frazier, Romy Fieblinger, Grace Long Torales, Benjamin A. Blakely, Nidhi Rastogi

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

SG-Bench: Evaluating LLM Safety Generalization Across Diverse Tasks and Prompt Types

Yutao Mou, Shikun Zhang, Wei Ye

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.CL

Comments Accepted by NeurIPS2024 (Dataset and Benchmark Track)

MMDocBench: Benchmarking Large Vision-Language Models for Fine-Grained Visual Document Understanding

Fengbin Zhu, Ziyang Liu, Xiang Yao Ng, Haohui Wu, Wenjie Wang, Fuli Feng, Chao Wang, Huanbo Luan, Tat Seng Chua

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments Under review

Paved or unpaved? A Deep Learning derived Road Surface Global Dataset from Mapillary Street-View Imagery

Sukanya Randhawa, Eren Aygun, Guntaj Randhawa, Benjamin Herfort, Sven Lautenbach, Alexander Zipf

专题命中 推理评测 ：planning(abstract)；分类 cs.AI

RepairAgent: An Autonomous, LLM-Based Agent for Program Repair

Islem Bouzenia, Premkumar Devanbu, Michael Pradel

专题命中 推理评测 ：planning(abstract)；分类 cs.AI

GUIDE: Graphical User Interface Data for Execution

Rajat Chawla, Adarsh Jha, Muskaan Kumar, Mukunda NS, Ishaan Bhola

专题命中 推理评测 ：CoT(abstract)；分类 cs.AI

Comments 11 pages, 8 figures, 3 Tables and 1 Algorithm

SQAPlanner: Generating Data-Informed Software Quality Improvement Plans

Dilini Rajapaksha, Chakkrit Tantithamthavorn, Jirayus Jiarpakdee, Christoph Bergmeir, John Grundy, Wray Buntine

专题命中 推理评测 ：planning(abstract)；分类 cs.LG

Comments This work has been Accepted by the IEEE Transactions on Software Engineering 24 pages

A Debate-Driven Experiment on LLM Hallucinations and Accuracy

Ray Li, Tanishka Bagade, Kevin Martinez, Flora Yasmin, Grant Ayala, Michael Lam, Kevin Zhu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Knowledge Distillation Using Frontier Open-source LLMs: Generalizability and the Role of Synthetic Data

Anup Shirgaonkar, Nikhil Pandey, Nazmiye Ceren Abay, Tolga Aktas, Vijay Aski

专题命中 推理评测 ：reasoning(abstract)；分类 cs.LG

Comments 25 pages, 4 figures

VHELM: A Holistic Evaluation of Vision Language Models

Tony Lee, Haoqin Tu, Chi Heem Wong, Wenhao Zheng, Yiyang Zhou, Yifan Mai, Josselin Somerville Roberts, Michihiro Yasunaga, Huaxiu Yao, Cihang Xie, Percy Liang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments NeurIPS 2024. First three authors contributed equally

Can visual language models resolve textual ambiguity with visual cues? Let visual puns tell you!

Jiwan Chung, Seungwon Lim, Jaehyun Jeon, Seungbeen Lee, Youngjae Yu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments Accepted as main paper in EMNLP 2024

A Statistical Analysis of LLMs' Self-Evaluation Using Proverbs

Ryosuke Sonoda, Ramya Srinivasan

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL