大模型推理能力

MARPLE: A Benchmark for Long-Horizon Inference

Emily Jin, Zhuoyi Huang, Jan-Philipp Fränken, Weiyu Liu, Hannah Cha, Erik Brockbank, Sarah Wu, Ruohan Zhang, Jiajun Wu, Tobias Gerstenberg

专题命中 推理评测 ：reasoning(abstract)；分类 cs.LG

Comments NeurIPS 2024. First two authors contributed equally. Project page: https://marple-benchmark.github.io/

Quantifying Generalization Complexity for Large Language Models

Zhenting Qi, Hongyin Luo, Xuliang Huang, Zhuokai Zhao, Yibo Jiang, Xiangjun Fan, Himabindu Lakkaraju, James Glass

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

LLM-as-a-Judge & Reward Model: What They Can and Cannot Do

Guijin Son, Hyunwoo Ko, Hoyoung Lee, Yewon Kim, Seunghyeok Hong

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments under review

Just ASR + LLM? A Study on Speech Large Language Models' Ability to Identify and Understand Speaker in Spoken Dialogue

Junkai Wu, Xulin Fan, Bo-Ru Lu, Xilin Jiang, Nima Mesgarani, Mark Hasegawa-Johnson, Mari Ostendorf

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted to IEEE SLT 2024

Towards Interpretable Mental Health Analysis with Large Language Models

Kailai Yang, Shaoxiong Ji, Tianlin Zhang, Qianqian Xie, Ziyan Kuang, Sophia Ananiadou

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted by EMNLP 2023 main conference as a long paper

Insight: A Multi-Modal Diagnostic Pipeline using LLMs for Ocular Surface Disease Diagnosis

Chun-Hsiao Yeh, Jiayun Wang, Andrew D. Graham, Andrea J. Liu, Bo Tan, Yubei Chen, Yi Ma, Meng C. Lin

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted to MICCAI 2024. Project Webpage: https://danielchyeh.github.io/MDPipe/

Adaptable Moral Stances of Large Language Models on Sexist Content: Implications for Society and Gender Discourse

Rongchen Guo, Isar Nejadgholi, Hillary Dawkins, Kathleen C. Fraser, Svetlana Kiritchenko

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments To be published at EMNLP2024

Automated Peer Reviewing in Paper SEA: Standardization, Evaluation, and Analysis

Jianxiang Yu, Zichen Ding, Jiaqi Tan, Kangyang Luo, Zhenmin Weng, Chenghua Gong, Long Zeng, Renjing Cui, Chengcheng Han, Qiushi Sun, Zhiyong Wu, Yunshi Lan, Xiang Li

专题命中 推理评测 ：self-correction(abstract)；分类 cs.CL

Comments Accepted by EMNLP 2024

Women Are Beautiful, Men Are Leaders: Gender Stereotypes in Machine Translation and Language Modeling

Matúš Pikuliak, Andrea Hrckova, Stefan Oresko, Marián Šimko

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments EMNLP 2024 Findings

Revealing Personality Traits: A New Benchmark Dataset for Explainable Personality Recognition on Dialogues

Lei Sun, Jinming Zhao, Qin Jin

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted to EMNLP 2024 Main Conference (Long Paper)

IntentionQA: A Benchmark for Evaluating Purchase Intention Comprehension Abilities of Language Models in E-commerce

Wenxuan Ding, Weiqi Wang, Sze Heng Douglas Kwok, Minghao Liu, Tianqing Fang, Jiaxin Bai, Xin Liu, Changlong Yu, Zheng Li, Chen Luo, Qingyu Yin, Bing Yin, Junxian He, Yangqiu Song

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Findings of EMNLP 2024

Improving Agent Behaviors with RL Fine-tuning for Autonomous Driving

Zhenghao Peng, Wenjie Luo, Yiren Lu, Tianyi Shen, Cole Gulino, Ari Seff, Justin Fu

专题命中 推理评测 ：planning(abstract)；分类 cs.AI

A Survey on Multimodal Benchmarks: In the Era of Large AI Models

Lin Li, Guikun Chen, Hanrong Shi, Jun Xiao, Long Chen

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments Ongoing project

Improving Diversity of Commonsense Generation by Large Language Models via In-Context Learning

Tianhui Zhang, Bei Peng, Danushka Bollegala

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments EMNLP 2024 Findings, Camera-ready version

CausalBench: A Comprehensive Benchmark for Causal Learning Capability of LLMs

Yu Zhou, Xingyu Wu, Beicheng Huang, Jibin Wu, Liang Feng, Kay Chen Tan

专题命中 推理评测 ：reasoning(abstract)；分类 cs.LG

DualCoTs: Dual Chain-of-Thoughts Prompting for Sentiment Lexicon Expansion of Idioms

Fuqiang Niu, Minghuan Tan, Bowen Zhang, Min Yang, Ruifeng Xu

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.CL

MUSE: Integrating Multi-Knowledge for Knowledge Graph Completion

Pengjie Liu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments arXiv admin note: text overlap with arXiv:2408.05283

Cyber Knowledge Completion Using Large Language Models

Braden K Webb, Sumit Purohit, Rounak Meyur

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments 7 pages, 2 figures. Submitted to 2024 IEEE International Conference on Big Data

PALLM: Evaluating and Enhancing PALLiative Care Conversations with Large Language Models

Zhiyuan Wang, Fangxu Yuan, Virginia LeBaron, Tabor Flickinger, Laura E. Barnes

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted by ACM Transactions on Computing for Healthcare, Special Issue on Large Language Models, Conversational Systems, and Generative AI in Health, pending minor revisions

VLM's Eye Examination: Instruct and Inspect Visual Competency of Vision Language Models

Nam Hyeon-Woo, Moon Ye-Bin, Wonseok Choi, Lee Hyun, Tae-Hyun Oh

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Multi-modal Auto-regressive Modeling via Visual Words

Tianshuo Peng, Zuchao Li, Lefei Zhang, Hai Zhao, Ping Wang, Bo Du

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments ACM MM 2024

Can Large Language Models Logically Predict Myocardial Infarction? Evaluation based on UK Biobank Cohort

Yuxing Zhi, Yuan Guo, Kai Yuan, Hesong Wang, Heng Xu, Haina Yao, Albert C Yang, Guangrui Huang, Yuping Duan

专题命中 推理评测 ：CoT(abstract)；分类 cs.AI

The use of GPT-4o and Other Large Language Models for the Improvement and Design of Self-Assessment Scales for Measurement of Interpersonal Communication Skills

Goran Bubaš

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments 41 pages

LegiLM: A Fine-Tuned Legal Language Model for Data Compliance

Linkai Zhu, Lu Yang, Chaofan Li, Shanwen Hu, Lu Liu, Bin Yin

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 9 pages, 2 figures

Can VLMs Play Action Role-Playing Games? Take Black Myth Wukong as a Study Case

Peng Chen, Pi Bu, Jun Song, Yuan Gao, Bo Zheng

专题命中 推理评测 ：planning(abstract)；分类 cs.AI

A User-Centric Multi-Intent Benchmark for Evaluating Large Language Models

Jiayin Wang, Fengran Mo, Weizhi Ma, Peijie Sun, Min Zhang, Jian-Yun Nie

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Enhancing SLM via ChatGPT and Dataset Augmentation

Tom Pieper, Mohamad Ballout, Ulf Krumnack, Gunther Heidemann, Kai-Uwe Kühnberger

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Large Language Model (LLM) for Telecommunications: A Comprehensive Survey on Principles, Key Techniques, and Opportunities

Hao Zhou, Chengming Hu, Ye Yuan, Yufei Cui, Yili Jin, Can Chen, Haolun Wu, Dun Yuan, Li Jiang, Di Wu, Xue Liu, Charlie Zhang, Xianbin Wang, Jiangchuan Liu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.LG

Operationalizing Contextual Integrity in Privacy-Conscious Assistants

Sahra Ghalebikesabi, Eugene Bagdasaryan, Ren Yi, Itay Yona, Ilia Shumailov, Aneesh Pappu, Chongyang Shi, Laura Weidinger, Robert Stanforth, Leonard Berrada, Pushmeet Kohli, Po-Sen Huang, Borja Balle

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Prompting Large Language Models with Audio for General-Purpose Speech Summarization

Wonjune Kang, Deb Roy

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted to Interspeech 2024