大模型推理能力

The ConceptARC Benchmark: Evaluating Understanding and Generalization in the ARC Domain

Arseny Moskvichev, Victor Vikram Odouard, Melanie Mitchell

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Journal ref Transactions on Machine Learning Research, 8/2023

AlphaStar Unplugged: Large-Scale Offline Reinforcement Learning

Michaël Mathieu, Sherjil Ozair, Srivatsan Srinivasan, Caglar Gulcehre, Shangtong Zhang, Ray Jiang, Tom Le Paine, Richard Powell, Konrad Żołna, Julian Schrittwieser, David Choi, Petko Georgiev, Daniel Toyama, Aja Huang, Roman Ring, Igor Babuschkin, Timo Ewalds, Mahyar Bordbar, Sarah Henderson, Sergio Gómez Colmenarejo, Aäron van den Oord, Wojciech Marian Czarnecki, Nando de Freitas, Oriol Vinyals

专题命中 推理评测 ：planning(abstract)；分类 cs.AI、cs.LG

Comments 32 pages, 13 figures, previous version published as a NeurIPS 2021 workshop: https://openreview.net/forum?id=Np8Pumfoty

No Fair Lunch: A Causal Perspective on Dataset Bias in Machine Learning for Medical Imaging

Charles Jones, Daniel C. Castro, Fabio De Sousa Ribeiro, Ozan Oktay, Melissa McCradden, Ben Glocker

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control

Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, Pete Florence, Chuyuan Fu, Montse Gonzalez Arenas, Keerthana Gopalakrishnan, Kehang Han, Karol Hausman, Alexander Herzog, Jasmine Hsu, Brian Ichter, Alex Irpan, Nikhil Joshi, Ryan Julian, Dmitry Kalashnikov, Yuheng Kuang, Isabel Leal, Lisa Lee, Tsang-Wei Edward Lee, Sergey Levine, Yao Lu, Henryk Michalewski, Igor Mordatch, Karl Pertsch, Kanishka Rao, Krista Reymann, Michael Ryoo, Grecia Salazar, Pannag Sanketi, Pierre Sermanet, Jaspiar Singh, Anikait Singh, Radu Soricut, Huong Tran, Vincent Vanhoucke, Quan Vuong, Ayzaan Wahid, Stefan Welker, Paul Wohlhart, Jialin Wu, Fei Xia, Ted Xiao, Peng Xu, Sichun Xu, Tianhe Yu, Brianna Zitkovich

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

Comments Website: https://robotics-transformer.github.io/

Embroid: Unsupervised Prediction Smoothing Can Improve Few-Shot Classification

Neel Guha, Mayee F. Chen, Kush Bhatia, Azalia Mirhoseini, Frederic Sala, Christopher Ré

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.CL、cs.LG

Comments 38 pages, 22 figures, 8 tables

I Spy a Metaphor: Large Language Models and Diffusion Models Co-Create Visual Metaphors

Tuhin Chakrabarty, Arkadiy Saakyan, Olivia Winn, Artemis Panagopoulou, Yue Yang, Marianna Apidianaki, Smaranda Muresan

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.CL、cs.AI

Comments ACL 2023 (Findings)

GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarking

Jiayan Guo, Lun Du, Hengyu Liu, Mengyu Zhou, Xinyi He, Shi Han

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Evaluating Shutdown Avoidance of Language Models in Textual Scenarios

Teun van der Weij, Simon Lermen, Leon lang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Zero-shot Approach to Overcome Perturbation Sensitivity of Prompts

Mohna Chakraborty, Adithya Kulkarni, Qi Li

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

CBBQ: A Chinese Bias Benchmark Dataset Curated with Human-AI Collaboration for Large Language Models

Yufei Huang, Deyi Xiong

专题命中 推理评测 ：self-correction(abstract)；分类 cs.CL、cs.AI

Probing neural language models for understanding of words of estimative probability

Damien Sileo, Marie-Francine Moens

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted at *SEM2023

ToolQA: A Dataset for LLM Question Answering with External Tools

Yuchen Zhuang, Yue Yu, Kuan Wang, Haotian Sun, Chao Zhang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Investigating Pre-trained Language Models on Cross-Domain Datasets, a Step Closer to General AI

Mohamad Ballout, Ulf Krumnack, Gunther Heidemann, Kai-Uwe Kühnberger

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

No Strong Feelings One Way or Another: Re-operationalizing Neutrality in Natural Language Inference

Animesh Nighojkar, Antonio Laverghetta, John Licato

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Appearing at the 17th Linguistic Annotation Workshop at ACL 2023

FLamE: Few-shot Learning from Natural Language Explanations

Yangqiaoyu Zhou, Yiming Zhang, Chenhao Tan

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted at ACL 2023 main conference

bgGLUE: A Bulgarian General Language Understanding Evaluation Benchmark

Momchil Hardalov, Pepa Atanasova, Todor Mihaylov, Galia Angelova, Kiril Simov, Petya Osenova, Ves Stoyanov, Ivan Koychev, Preslav Nakov, Dragomir Radev

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

Comments Accepted to ACL 2023 (Main Conference)

Journal ref ACL 2023

Preconditioned Visual Language Inference with Weak Supervision

Ehsan Qasemi, Amani R. Maina-Kilaas, Devadutta Dash, Khalid Alsaggaf, Muhao Chen

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Explanation Graph Generation via Generative Pre-training over Synthetic Graphs

Han Cui, Shangzhan Li, Yu Zhang, Qi Shi

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACL23-Findings

Using Visual Cropping to Enhance Fine-Detail Question Answering of BLIP-Family Models

Jiarui Zhang, Mahyar Khayatkhoei, Prateek Chhikara, Filip Ilievski

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 16 pages, 5 figures, 7 tables

Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Model

Deepanway Ghosal, Navonil Majumder, Ambuj Mehrish, Soujanya Poria

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.CL、cs.AI

Comments https://github.com/declare-lab/tango

An Experimental Investigation into the Evaluation of Explainability Methods

Sédrick Stassin, Alexandre Englebert, Géraldin Nanfack, Julien Albert, Nassim Versbraegen, Gilles Peiffer, Miriam Doh, Nicolas Riche, Benoît Frenay, Christophe De Vleeschouwer

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Comments 16 pages, 7 figures

Text-Augmented Open Knowledge Graph Completion via Pre-Trained Language Models

Pengcheng Jiang, Shivam Agarwal, Bowen Jin, Xuan Wang, Jimeng Sun, Jiawei Han

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 18 pages, 11 figures, 8 tables. Accepted by ACL 23' Findings

EvEval: A Comprehensive Evaluation of Event Semantics for Large Language Models

Zhengwei Tao, Zhi Jin, Xiaoying Bai, Haiyan Zhao, Yanlin Feng, Jia Li, Wenpeng Hu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Language Model Self-improvement by Reinforcement Learning Contemplation

Jing-Cheng Pang, Pengyuan Wang, Kaiyuan Li, Xiong-Hui Chen, Jiacheng Xu, Zongzhang Zhang, Yang Yu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment

Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, Chenguang Zhu

专题命中 推理评测 ：CoT(abstract)；分类 cs.CL、cs.AI

Learning Meta Representations of One-shot Relations for Temporal Knowledge Graph Link Prediction

Zifeng Ding, Bailan He, Yunpu Ma, Zhen Han, Volker Tresp

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Comments IJCNN 2023 oral

The KITMUS Test: Evaluating Knowledge Integration from Multiple Sources in Natural Language Understanding Systems

Akshatha Arodi, Martin Pömsl, Kaheer Suleman, Adam Trischler, Alexandra Olteanu, Jackie Chi Kit Cheung

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

Comments Accepted at ACL 2023. Code available at https://github.com/mpoemsl/kitmus

PubGraph: A Large-Scale Scientific Knowledge Graph

Kian Ahrabian, Xinwei Du, Richard Delwin Myloth, Arun Baalaaji Sankar Ananthan, Jay Pujara

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Comments 17 Pages, 6 Figures, 9 Tables

RHO ($ρ$): Reducing Hallucination in Open-domain Dialogues with Knowledge Grounding

Ziwei Ji, Zihan Liu, Nayeon Lee, Tiezheng Yu, Bryan Wilie, Min Zeng, Pascale Fung

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments accepted by ACL 2023 Findings

Relation-Aware Language-Graph Transformer for Question Answering

Jinyoung Park, Hyeong Kyu Choi, Juyeon Ko, Hyeonjin Park, Ji-Hoon Kim, Jisu Jeong, Kyungmin Kim, Hyunwoo J. Kim

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments AAAI2023 (accepted)