大模型推理能力

Cultural Bias and Cultural Alignment of Large Language Models

Yan Tao, Olga Viberg, Ryan S. Baker, Rene F. Kizilcec

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Journal ref PNAS Nexus, Volume 3, Issue 9, September 2024, pgae346

A Knowledge-Enhanced Disease Diagnosis Method Based on Prompt Learning and BERT Integration

Zhang Zheng

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Knowledge Enhancement,Disease Diagnosis,Prompt Learning,BERT,Knowledge Graph

Dialogue You Can Trust: Human and AI Perspectives on Generated Conversations

Ike Ebubechukwu, Johane Takeuchi, Antonello Ceravola, Frank Joublin

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 17 pages, 15 figures

DetoxBench: Benchmarking Large Language Models for Multitask Fraud & Abuse Detection

Joymallya Chakraborty, Wei Xia, Anirban Majumder, Dan Ma, Walid Chaabene, Naveed Janvekar

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

Comments 12 pages

Journal ref Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2024

PanoSent: A Panoptic Sextuple Extraction Benchmark for Multimodal Conversational Aspect-based Sentiment Analysis

Meng Luo, Hao Fei, Bobo Li, Shengqiong Wu, Qian Liu, Soujanya Poria, Erik Cambria, Mong-Li Lee, Wynne Hsu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACM MM 2024 (Oral)

A Survey on Benchmarks of Multimodal Large Language Models

Jian Li, Weiheng Lu, Hao Fei, Meng Luo, Ming Dai, Min Xia, Yizhang Jin, Zhenye Gan, Ding Qi, Chaoyou Fu, Ying Tai, Wankou Yang, Yabiao Wang, Chengjie Wang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

FineMath: A Fine-Grained Mathematical Evaluation Benchmark for Chinese Large Language Models

Yan Liu, Renren Jin, Ling Shi, Zheng Yao, Deyi Xiong

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Tracing Privacy Leakage of Language Models to Training Data via Adjusted Influence Functions

Jinxin Liu, Zao Yang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

WikiCausal: Corpus and Evaluation Framework for Causal Knowledge Graph Construction

Oktie Hassanzadeh

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Extended version; poster paper accepted at ISWC 2024

Hidden flaws behind expert-level accuracy of multimodal GPT-4 vision in medicine

Qiao Jin, Fangyuan Chen, Yiliang Zhou, Ziyang Xu, Justin M. Cheung, Robert Chen, Ronald M. Summers, Justin F. Rousseau, Peiyun Ni, Marc J Landsman, Sally L. Baxter, Subhi J. Al'Aref, Yijia Li, Alex Chen, Josef A. Brejt, Michael F. Chiang, Yifan Peng, Zhiyong Lu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Journal ref npj Digital Medicine, 2024

Follow-up Attention: An Empirical Study of Developer and Neural Model Code Exploration

Matteo Paltenghi, Rahul Pandita, Austin Z. Henley, Albert Ziegler

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Comments Published at IEEE Transactions on Software Engineering

A Survey on Evaluation of Multimodal Large Language Models

Jiaxing Huang, Jingyi Zhang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

GPT-4 Emulates Average-Human Emotional Cognition from a Third-Person Perspective

Ala N. Tak, Jonathan Gratch

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments submitted to 12th International Conference on Affective Computing & Intelligent Interaction, Glasgow, UK, September 15-18, 2024

Implicit Sentiment Analysis Based on Chain of Thought Prompting

Zhihua Duan, Jialin Wang

专题命中 推理评测 ：CoT(abstract)；分类 cs.CL、cs.AI

Diagnosing and Remedying Knowledge Deficiencies in LLMs via Label-free Curricular Meaningful Learning

Kai Xiong, Xiao Ding, Li Du, Jiahao Ying, Ting Liu, Bing Qin, Yixin Cao

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Under Review

Athena: Safe Autonomous Agents with Verbal Contrastive Learning

Tanmana Sadhu, Ali Pesaranghader, Yanan Chen, Dong Hoon Yi

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 9 pages, 2 figures, 4 tables

DELIA: Diversity-Enhanced Learning for Instruction Adaptation in Large Language Models

Yuanhao Zeng, Fei Ren, Xinpeng Zhou, Yihang Wang, Yingxia Shao

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 8 pages, 5 figures

Commonsense-T2I Challenge: Can Text-to-Image Generation Models Understand Commonsense?

Xingyu Fu, Muyu He, Yujie Lu, William Yang Wang, Dan Roth

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments COLM 2024, Project Url: https://zeyofu.github.io/CommonsenseT2I/

Self-Taught Evaluators

Tianlu Wang, Ilia Kulikov, Olga Golovneva, Ping Yu, Weizhe Yuan, Jane Dwivedi-Yu, Richard Yuanzhe Pang, Maryam Fazel-Zarandi, Jason Weston, Xian Li

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Can LLMs Serve As Time Series Anomaly Detectors?

Manqing Dong, Hao Huang, Longbing Cao

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.AI、cs.LG

Unveiling LLMs: The Evolution of Latent Representations in a Dynamic Knowledge Graph

Marco Bronzini, Carlo Nicolini, Bruno Lepri, Jacopo Staiano, Andrea Passerini

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted at COLM 2024

GAOKAO-MM: A Chinese Human-Level Benchmark for Multimodal Models Evaluation

Yi Zong, Xipeng Qiu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

LocalValueBench: A Collaboratively Built and Extensible Benchmark for Evaluating Localized Value Alignment and Ethical Safety in Large Language Models

Gwenyth Isobel Meadows, Nicholas Wai Long Lau, Eva Adelina Susanto, Chi Lok Yu, Aditya Paul

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

WorkBench: a Benchmark Dataset for Agents in a Realistic Workplace Setting

Olly Styles, Sam Miller, Patricio Cerda-Mardini, Tanaya Guha, Victor Sanchez, Bertie Vidgen

专题命中 推理评测 ：planning(abstract)；分类 cs.CL、cs.AI

MuChoMusic: Evaluating Music Understanding in Multimodal Audio-Language Models

Benno Weck, Ilaria Manco, Emmanouil Benetos, Elio Quinton, George Fazekas, Dmitry Bogdanov

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

Comments Accepted at ISMIR 2024. Data: https://doi.org/10.5281/zenodo.12709974 Code: https://github.com/mulab-mir/muchomusic Supplementary material: https://mulab-mir.github.io/muchomusic

Distributed In-Context Learning under Non-IID Among Clients

Siqi Liang, Sumyeong Ahn, Jiayu Zhou

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 12 pages

The Impact of Quantization on Retrieval-Augmented Generation: An Analysis of Small LLMs

Mert Yazan, Suzan Verberne, Frederik Situmeang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted to the IR-RAG Workshop at SIGIR 2024

LAPIS: Language Model-Augmented Police Investigation System

Heedou Kim, Dain Kim, Jiwoo Lee, Chanwoong Yoon, Donghee Choi, Mogan Gim, Jaewoo Kang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

MMWorld: Towards Multi-discipline Multi-faceted World Model Evaluation in Videos

Xuehai He, Weixi Feng, Kaizhi Zheng, Yujie Lu, Wanrong Zhu, Jiachen Li, Yue Fan, Jianfeng Wang, Linjie Li, Zhengyuan Yang, Kevin Lin, William Yang Wang, Lijuan Wang, Xin Eric Wang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Optimizing Numerical Estimation and Operational Efficiency in the Legal Domain through Large Language Models

Jia-Hong Huang, Chao-Chun Yang, Yixian Shen, Alessio M. Pacces, Evangelos Kanoulas

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments The paper has been accepted by the 33rd ACM International Conference on Information and Knowledge Management (CIKM) in 2024