大模型推理能力

RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shot

Hao-Shu Fang, Hongjie Fang, Zhenyu Tang, Jirong Liu, Chenxi Wang, Junbo Wang, Haoyi Zhu, Cewu Lu

专题命中 推理评测 ：planning(abstract)；分类 cs.AI

Comments RSS 2023 workshop on LTAMP. The project page is at rh20t.github.io

HRoT: Hybrid prompt strategy and Retrieval of Thought for Table-Text Hybrid Question Answering

Tongxu Luo, Fangyu Lei, Jiahe Lei, Weihao Liu, Shihu He, Jun Zhao, Kang Liu

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.CL

MindAgent: Emergent Gaming Interaction

Ran Gong, Qiuyuan Huang, Xiaojian Ma, Hoi Vo, Zane Durante, Yusuke Noda, Zilong Zheng, Song-Chun Zhu, Demetri Terzopoulos, Li Fei-Fei, Jianfeng Gao

专题命中 推理评测 ：planning(abstract)；分类 cs.AI

Comments The first three authors contributed equally. 28 pages

BHASA: A Holistic Southeast Asian Linguistic and Cultural Evaluation Suite for Large Language Models

Wei Qi Leong, Jian Gang Ngui, Yosephine Susanto, Hamsawardhini Rengarajan, Kengatharaiyer Sarveswaran, William Chandra Tjhi

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 86 pages, 7 figures, added link to repository in abstract, minor formatting changes and typo corrections

Multimodal Multi-Hop Question Answering Through a Conversation Between Tools and Efficiently Finetuned Large Language Models

Hossein Rajabzadeh, Suyuchen Wang, Hyock Ju Kwon, Bang Liu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

TouchStone: Evaluating Vision-Language Models by Language Models

Shuai Bai, Shusheng Yang, Jinze Bai, Peng Wang, Xingxuan Zhang, Junyang Lin, Xinggang Wang, Chang Zhou, Jingren Zhou

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments https://github.com/OFA-Sys/TouchStone

The Poison of Alignment

Aibek Bekbayev, Sungbae Chun, Yerzat Dulat, James Yamazaki

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

PMC-LLaMA: Towards Building Open-source Language Models for Medicine

Chaoyi Wu, Weixiong Lin, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, Weidi Xie

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Physics-Based Task Generation through Causal Sequence of Physical Interactions

Chathura Gamage, Vimukthini Pinto, Matthew Stephenson, Jochen Renz

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments The 19th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE-23)

Link-Context Learning for Multimodal LLMs

Yan Tai, Weichen Fan, Zhao Zhang, Feng Zhu, Rui Zhao, Ziwei Liu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 10 pages, 8 figures

PaCo: Preconditions Attributed to Commonsense Knowledge

Ehsan Qasemi, Filip Ilievski, Muhao Chen, Pedro Szekely

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments EMNLP 2022 (Findings)

Unleashing the Strengths of Unlabeled Data in Pan-cancer Abdominal Organ Quantification: the FLARE22 Challenge

Jun Ma, Yao Zhang, Song Gu, Cheng Ge, Shihao Ma, Adamo Young, Cheng Zhu, Kangkang Meng, Xin Yang, Ziyan Huang, Fan Zhang, Wentao Liu, YuanKe Pan, Shoujin Huang, Jiacheng Wang, Mingze Sun, Weixin Xu, Dengqiang Jia, Jae Won Choi, Natália Alves, Bram de Wilde, Gregor Koehler, Yajun Wu, Manuel Wiesenfarth, Qiongjie Zhu, Guoqiang Dong, Jian He, the FLARE Challenge Consortium, Bo Wang

专题命中 推理评测 ：planning(abstract)；分类 cs.AI

Comments MICCAI FLARE22: https://flare22.grand-challenge.org/

GPT-4 Can't Reason

Konstantine Arkoudas

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

AgentSims: An Open-Source Sandbox for Large Language Model Evaluation

Jiaju Lin, Haoran Zhao, Aochi Zhang, Yiting Wu, Huqiuyue Ping, Qin Chen

专题命中 推理评测 ：planning(abstract)；分类 cs.AI

Comments submit to EMNLP2023 demo track

Towards Generalist Biomedical AI

Tao Tu, Shekoofeh Azizi, Danny Driess, Mike Schaekermann, Mohamed Amin, Pi-Chuan Chang, Andrew Carroll, Chuck Lau, Ryutaro Tanno, Ira Ktena, Basil Mustafa, Aakanksha Chowdhery, Yun Liu, Simon Kornblith, David Fleet, Philip Mansfield, Sushant Prakash, Renee Wong, Sunny Virmani, Christopher Semturs, S Sara Mahdavi, Bradley Green, Ewa Dominowska, Blaise Aguera y Arcas, Joelle Barral, Dale Webster, Greg S. Corrado, Yossi Matias, Karan Singhal, Pete Florence, Alan Karthikesalingam, Vivek Natarajan

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Advancing Visual Grounding with Scene Knowledge: Benchmark and Method

Zhihong Chen, Ruifei Zhang, Yibing Song, Xiang Wan, Guanbin Li

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Computer Vision and Natural Language Processing. 21 pages, 14 figures. CVPR-2023

CValues: Measuring the Values of Chinese Large Language Models from Safety to Responsibility

Guohai Xu, Jiayi Liu, Ming Yan, Haotian Xu, Jinghui Si, Zhuoran Zhou, Peng Yi, Xing Gao, Jitao Sang, Rong Zhang, Ji Zhang, Chao Peng, Fei Huang, Jingren Zhou

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Working in Process

CodeQueries: A Dataset of Semantic Queries over Code

Surya Prakash Sahu, Madhurima Mandal, Shikhar Bharadwaj, Aditya Kanade, Petros Maniatis, Shirish Shevade

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

BLUEX: A benchmark based on Brazilian Leading Universities Entrance eXams

Thales Sales Almeida, Thiago Laitz, Giovana K. Bonás, Rodrigo Nogueira

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Explaining Competitive-Level Programming Solutions using LLMs

Jierui Li, Szymon Tworkowski, Yingying Wu, Raymond Mooney

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 14 pages, presented at the 1st NLRSE workshop

From Human Days to Machine Seconds: Automatically Answering and Generating Machine Learning Final Exams

Iddo Drori, Sarah J. Zhang, Reece Shuttleworth, Sarah Zhang, Keith Tyser, Zad Chin, Pedro Lantigua, Saisamrit Surbehera, Gregory Hunter, Derek Austin, Leonard Tang, Yann Hicke, Sage Simhon, Sathwik Karnik, Darnell Granberry, Madeleine Udell

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.LG

Comments 9 pages

Exploring the Potential of AI-Generated Synthetic Datasets: A Case Study on Telematics Data with ChatGPT

Ryan Lingo

专题命中 推理评测 ：planning(abstract)；分类 cs.LG

Comments 11 pages, 5 figures

ClinicalGPT: Large Language Models Finetuned with Diverse Medical Data and Comprehensive Evaluation

Guangyu Wang, Guoxing Yang, Zongxin Du, Longjun Fan, Xiaohu Li

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Models

Peng Xu, Wenqi Shao, Kaipeng Zhang, Peng Gao, Shuo Liu, Meng Lei, Fanqing Meng, Siyuan Huang, Yu Qiao, Ping Luo

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments 28 pages, 10 figures, a comprehensive evaluation of large vision-language models

X-IQE: eXplainable Image Quality Evaluation for Text-to-Image Generation with Visual Large Language Models

Yixiong Chen, Li Liu, Chris Ding

专题命中 推理评测 ：CoT(abstract)；分类 cs.AI

Comments 18 pages, 6 tables, 11 figures, NeurIPS 2023 submission

SocialLight: Distributed Cooperation Learning towards Network-Wide Traffic Signal Control

Harsh Goel, Yifeng Zhang, Mehul Damani, Guillaume Sartoretti

专题命中 推理评测 ：reasoning(abstract)；分类 cs.LG

Comments To appear in the International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2023)

From chocolate bunny to chocolate crocodile: Do Language Models Understand Noun Compounds?

Jordan Coil, Vered Shwartz

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Findings of ACL 2023

SETI: Systematicity Evaluation of Textual Inference

Xiyan Fu, Anette Frank

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted to Findings of ACL2023

AD-KD: Attribution-Driven Knowledge Distillation for Language Model Compression

Siyue Wu, Hongzhan Chen, Xiaojun Quan, Qifan Wang, Rui Wang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted to ACL 2023 Main Conference

Translate to Disambiguate: Zero-shot Multilingual Word Sense Disambiguation with Pretrained Language Models

Haoqiang Kang, Terra Blevins, Luke Zettlemoyer

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL