大模型推理能力

InternalInspector $I^2$: Robust Confidence Estimation in LLMs through Internal States

Mohammad Beigi, Ying Shen, Runing Yang, Zihao Lin, Qifan Wang, Ankith Mohan, Jianfeng He, Ming Jin, Chang-Tien Lu, Lifu Huang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 8 pages

A Two-dimensional Zero-shot Dialogue State Tracking Evaluation Method using GPT-4

Ming Gu, Yan Yang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

MEMLA: Enhancing Multilingual Knowledge Editing with Neuron-Masked Low-Rank Adaptation

Jiakuan Xie, Pengfei Cao, Yuheng Chen, Yubo Chen, Kang Liu, Jun Zhao

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Precision Empowers, Excess Distracts: Visual Question Answering With Dynamically Infused Knowledge In Language Models

Manas Jhalani, Annervaz K M, Pushpak Bhattacharyya

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 16 pages, 12 figures

A Survey of Video Datasets for Grounded Event Understanding

Kate Sanders, Benjamin Van Durme

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

EX-FEVER: A Dataset for Multi-hop Explainable Fact Verification

Huanhuan Ma, Weizhi Xu, Yifan Wei, Liuji Chen, Liang Wang, Qiang Liu, Shu Wu, Liang Wang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments ACL 2024 findings paper

Aligning Vision Models with Human Aesthetics in Retrieval: Benchmarks and Algorithms

Miaosen Zhang, Yixuan Wei, Zhen Xing, Yifei Ma, Zuxuan Wu, Ji Li, Zheng Zhang, Qi Dai, Chong Luo, Xin Geng, Baining Guo

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments 28 pages, 26 figures, under review

KNVQA: A Benchmark for evaluation knowledge-based VQA

Sirui Cheng, Siyu Zhang, Jiayi Wu, Muchen Lan

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments There was a little error in the method section of the paper

On the Robustness of Document-Level Relation Extraction Models to Entity Name Variations

Shiao Meng, Xuming Hu, Aiwei Liu, Fukun Ma, Yawen Yang, Shuang Li, Lijie Wen

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted to ACL 2024 Findings

Security Vulnerability Detection with Multitask Self-Instructed Fine-Tuning of Large Language Models

Aidan Z. H. Yang, Haoye Tian, He Ye, Ruben Martins, Claire Le Goues

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.LG

RewardBench: Evaluating Reward Models for Language Modeling

Nathan Lambert, Valentina Pyatkin, Jacob Morrison, LJ Miranda, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, Noah A. Smith, Hannaneh Hajishirzi

专题命中 推理评测 ：reasoning(abstract)；分类 cs.LG

Comments 44 pages, 19 figures, 12 tables

Pearl: A Review-driven Persona-Knowledge Grounded Conversational Recommendation Dataset

Minjin Kim, Minju Kim, Hana Kim, Beong-woo Kwak, Soyeon Chun, Hyunseo Kim, SeongKu Kang, Youngjae Yu, Jinyoung Yeo, Dongha Lee

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Published at ACL 2024 Findings

When LLMs Meet Cunning Texts: A Fallacy Understanding Benchmark for Large Language Models

Yinghui Li, Qingyu Zhou, Yuanzhen Luo, Shirong Ma, Yangning Li, Hai-Tao Zheng, Xuming Hu, Philip S. Yu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

BAMO at SemEval-2024 Task 9: BRAINTEASER: A Novel Task Defying Common Sense

Baktash Ansari, Mohammadmostafa Rostamkhani, Sauleh Eetemadi

专题命中 推理评测 ：CoT(abstract)；分类 cs.CL

Comments 9 pages, 8 tables, 5 figures

Large Language Models as Evaluators for Recommendation Explanations

Xiaoyu Zhang, Yishan Li, Jiayin Wang, Bowen Sun, Weizhi Ma, Peijie Sun, Min Zhang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems

Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, Maosong Sun

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted by ACL 2024 (main), update

Missci: Reconstructing Fallacies in Misrepresented Science

Max Glockner, Yufang Hou, Preslav Nakov, Iryna Gurevych

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments ACL 2024 (main)

Chain of Agents: Large Language Models Collaborating on Long-Context Tasks

Yusen Zhang, Ruoxi Sun, Yanfei Chen, Tomas Pfister, Rui Zhang, Sercan Ö. Arik

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 19 pages, 6 figures

PAT-Questions: A Self-Updating Benchmark for Present-Anchored Temporal Question-Answering

Jannat Ara Meem, Muhammad Shihab Rashid, Yue Dong, Vagelis Hristidis

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted to Findings of ACL '24

Multimodal ArXiv: A Dataset for Improving Scientific Comprehension of Large Vision-Language Models

Lei Li, Yuqi Wang, Runxin Xu, Peiyi Wang, Xiachong Feng, Lingpeng Kong, Qi Liu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Project page: https://mm-arxiv.github.io, Camera Ready Version of ACL 2024

Improving Open-Ended Text Generation via Adaptive Decoding

Wenhong Zhu, Hongkun Hao, Zhiwei He, Yiming Ai, Rui Wang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments ICML2024

CMDBench: A Benchmark for Coarse-to-fine Multimodal Data Discovery in Compound AI Systems

Yanlin Feng, Sajjadur Rahman, Aaron Feng, Vincent Chen, Eser Kandogan

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments Governance, Understanding and Integration of Data for Effective and Responsible AI (GUIDE-AI '24), June 14, 2024, Santiago, AA, Chile

Mining experimental data from Materials Science literature with Large Language Models: an evaluation study

Luca Foppiano, Guillaume Lambard, Toshiyuki Amagasa, Masashi Ishii

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 40 pages: 5 figures and 1 table in the body. 32 Tables in the Appendix / Supplementary materials

Journal ref Science and Technology of Advanced Materials: Methods (2024)

SLIDE: A Framework Integrating Small and Large Language Models for Open-Domain Dialogues Evaluation

Kun Zhao, Bohao Yang, Chen Tang, Chenghua Lin, Liang Zhan

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted by ACL2024 Findings

Predicting Parking Availability in Singapore with Cross-Domain Data: A New Dataset and A Data-Driven Approach

Huaiwu Zhang, Yutong Xia, Siru Zhong, Kun Wang, Zekun Tong, Qingsong Wen, Roger Zimmermann, Yuxuan Liang

专题命中 推理评测 ：planning(abstract)；分类 cs.AI

Comments Accepted by IJCAI 2024 (Multi-Year Track On AI And Social Good with ~20% acceptance rate)

TimeChara: Evaluating Point-in-Time Character Hallucination of Role-Playing Large Language Models

Jaewoo Ahn, Taehyun Lee, Junyoung Lim, Jin-Hwa Kim, Sangdoo Yun, Hwaran Lee, Gunhee Kim

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments ACL 2024 Findings. Code and dataset are released at https://ahnjaewoo.github.io/timechara

MinPrompt: Graph-based Minimal Prompt Data Augmentation for Few-shot Question Answering

Xiusi Chen, Jyun-Yu Jiang, Wei-Cheng Chang, Cho-Jui Hsieh, Hsiang-Fu Yu, Wei Wang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments ACL 2024 main conference

Do Language Models Enjoy Their Own Stories? Prompting Large Language Models for Automatic Story Evaluation

Cyril Chhun, Fabian M. Suchanek, Chloé Clavel

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments TACL, pre-MIT Press publication version

MAmmoTH2: Scaling Instructions from the Web

Xiang Yue, Tuney Zheng, Ge Zhang, Wenhu Chen

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

What Makes it Ok to Set a Fire? Iterative Self-distillation of Contexts and Rationales for Disambiguating Defeasible Social and Moral Situations

Kavel Rao, Liwei Jiang, Valentina Pyatkin, Yuling Gu, Niket Tandon, Nouha Dziri, Faeze Brahman, Yejin Choi

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Camera Ready EMNLP Findings 2023. First two authors contributed equally