大模型推理能力

Benchmarking Retrieval-Augmented Generation for Medicine

Guangzhi Xiong, Qiao Jin, Zhiyong Lu, Aidong Zhang

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.CL、cs.AI

Comments Homepage: https://teddy-xionggz.github.io/benchmark-medical-rag/

AgentMD: Empowering Language Agents for Risk Prediction with Large-Scale Clinical Tool Learning

Qiao Jin, Zhizheng Wang, Yifan Yang, Qingqing Zhu, Donald Wright, Thomas Huang, W John Wilbur, Zhe He, Andrew Taylor, Qingyu Chen, Zhiyong Lu

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.CL、cs.AI

Comments Work in progress

DolphCoder: Echo-Locating Code Large Language Models with Diverse and Multi-Objective Instruction Tuning

Yejie Wang, Keqing He, Guanting Dong, Pei Wang, Weihao Zeng, Muxi Diao, Yutao Mou, Mengdi Zhang, Jingang Wang, Xunliang Cai, Weiran Xu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 14 pages, 6 figures

Let Your Graph Do the Talking: Encoding Structured Data for LLMs

Bryan Perozzi, Bahare Fatemi, Dustin Zelle, Anton Tsitsulin, Mehran Kazemi, Rami Al-Rfou, Jonathan Halcrow

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

SceMQA: A Scientific College Entrance Level Multimodal Question Answering Benchmark

Zhenwen Liang, Kehan Guo, Gang Liu, Taicheng Guo, Yujun Zhou, Tianyu Yang, Jiajun Jiao, Renjie Pi, Jipeng Zhang, Xiangliang Zhang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Work in progress

Language Models as Inductive Reasoners

Zonglin Yang, Li Dong, Xinya Du, Hao Cheng, Erik Cambria, Xiaodong Liu, Jianfeng Gao, Furu Wei

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted by EACL 2024

"What's my model inside of?": Exploring the role of environments for grounded natural language understanding

Ronen Tamari

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments PhD Thesis

FlexKBQA: A Flexible LLM-Powered Framework for Few-Shot Knowledge Base Question Answering

Zhenyu Li, Sunqi Fan, Yu Gu, Xiuxing Li, Zhichao Duan, Bowen Dong, Ning Liu, Jianyong Wang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted as AAAI-24 Oral paper; Knowledge Base Question Answering; Large Language Model; Data Generation; Few-Shot & Zero-Shot

MicroSegNet: A Deep Learning Approach for Prostate Segmentation on Micro-Ultrasound Images

Hongxu Jiang, Muhammad Imran, Preethika Muralidharan, Anjali Patel, Jake Pensa, Muxuan Liang, Tarik Benidir, Joseph R. Grajo, Jason P. Joseph, Russell Terry, John Michael DiBianco, Li-Ming Su, Yuyin Zhou, Wayne G. Brisbane, Wei Shao

专题命中 推理评测 ：planning(abstract)；分类 cs.AI、cs.LG

Journal ref Computerized Medical Imaging and Graphics (2024): 102326

NormSAGE: Multi-Lingual Multi-Cultural Norm Discovery from Conversations On-the-Fly

Yi R. Fung, Tuhin Chakraborty, Hao Guo, Owen Rambow, Smaranda Muresan, Heng Ji

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

AstroLLaMA-Chat: Scaling AstroLLaMA with Conversational and Diverse Datasets

Ernest Perkowski, Rui Pan, Tuan Dung Nguyen, Yuan-Sen Ting, Sandor Kruk, Tong Zhang, Charlie O'Neill, Maja Jablonska, Zechang Sun, Michael J. Smith, Huiling Liu, Kevin Schawinski, Kartheik Iyer, Ioana Ciucă for UniverseTBD

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

Comments 4 pages, 1 figure, model is available at https://huggingface.co/universeTBD, published in RNAAS

Large Language Models Are Not Strong Abstract Reasoners

Gaël Gendron, Qiming Bao, Michael Witbrock, Gillian Dobbie

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

Comments 50 pages, 14 pages for the main paper and 36 pages for the supplement, 35 figures, 17 tables. V3: performed additional experiments

XAI for In-hospital Mortality Prediction via Multimodal ICU Data

Xingqiao Li, Jindong Gu, Zhiyong Wang, Yancheng Yuan, Bo Du, Fengxiang He

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

A Survey on Evaluation of Large Language Models

Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, Wei Ye, Yue Zhang, Yi Chang, Philip S. Yu, Qiang Yang, Xing Xie

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACM Transactions on Intelligent Systems and Technology (TIST); 45 pages; More recent works; https://llm-eval.github.io/

What can Large Language Models do in chemistry? A comprehensive benchmark on eight tasks

Taicheng Guo, Kehan Guo, Bozhao Nan, Zhenwen Liang, Zhichun Guo, Nitesh V. Chawla, Olaf Wiest, Xiangliang Zhang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments NeurIPS 2023 Datasets and Benchmarks Track camera-ready version

Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena

Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, Ion Stoica

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments NeurIPS 2023 Datasets and Benchmarks Track

Capture the Flag: Uncovering Data Insights with Large Language Models

Issam Laradji, Perouz Taslakian, Sai Rajeswar, Valentina Zantedeschi, Alexandre Lacoste, Nicolas Chapados, David Vazquez, Christopher Pal, Alexandre Drouin

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

Comments 14 pages, 1 figure, Foundation Models for Decision Making Workshop at NeurIPS 2023

MedBench: A Large-Scale Chinese Benchmark for Evaluating Medical Large Language Models

Yan Cai, Linlin Wang, Ye Wang, Gerard de Melo, Ya Zhang, Yanfeng Wang, Liang He

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments accepted by AAAI-24

A Challenger to GPT-4V? Early Explorations of Gemini in Visual Expertise

Chaoyou Fu, Renrui Zhang, Zihan Wang, Yubo Huang, Zhengye Zhang, Longtian Qiu, Gaoxiang Ye, Yunhang Shen, Mengdan Zhang, Peixian Chen, Sirui Zhao, Shaohui Lin, Deqiang Jiang, Di Yin, Peng Gao, Ke Li, Hongsheng Li, Xing Sun

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Total 120 pages. See our project at https://github.com/BradyFU/Awesome-Multimodal-Large-Language-Models

RewriteLM: An Instruction-Tuned Large Language Model for Text Rewriting

Lei Shu, Liangchen Luo, Jayakumar Hoskere, Yun Zhu, Yinxiao Liu, Simon Tong, Jindong Chen, Lei Meng

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.CL、cs.AI

Journal ref AAAI 2024

Implicit Affordance Acquisition via Causal Action-Effect Modeling in the Video Domain

Hsiu-Yu Yang, Carina Silberer

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted at IJCNLP-AACL 2023

Rapid Open-World Adaptation by Adaptation Principles Learning

Cheng Xue, Ekaterina Nikonova, Peng Zhang, Jochen Renz

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

ShuttleSHAP: A Turn-Based Feature Attribution Approach for Analyzing Forecasting Models in Badminton

Wei-Yao Wang, Wen-Chih Peng, Wei Wang, Philip S. Yu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Comments Preprint

Modality Plug-and-Play: Elastic Modality Adaptation in Multimodal LLMs for Embodied AI

Kai Huang, Boyuan Yang, Wei Gao

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

SM70: A Large Language Model for Medical Devices

Anubhav Bhatti, Surajsinh Parmar, San Lee

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 5 Pages, Technical Report

Evidence-based Interpretable Open-domain Fact-checking with Large Language Models

Xin Tan, Bowei Zou, Ai Ti Aw

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Harnessing the Power of Prompt-based Techniques for Generating School-Level Questions using Large Language Models

Subhankar Maity, Aniket Deroy, Sudeshna Sarkar

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMs

Haoqin Tu, Chenhang Cui, Zijun Wang, Yiyang Zhou, Bingchen Zhao, Junlin Han, Wangchunshu Zhou, Huaxiu Yao, Cihang Xie

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

Comments H.T., C.C., and Z.W. contribute equally. Work done during H.T. and Z.W.'s internship at UCSC, and C.C. and Y.Z.'s internship at UNC

GAIA: a benchmark for General AI Assistants

Grégoire Mialon, Clémentine Fourrier, Craig Swift, Thomas Wolf, Yann LeCun, Thomas Scialom

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Do Smaller Language Models Answer Contextualised Questions Through Memorisation Or Generalisation?

Tim Hartill, Joshua Bensemann, Michael Witbrock, Patricia J. Riddle

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI