大模型推理能力

Transformer Models for Type Inference in the Simply Typed Lambda Calculus: A Case Study in Deep Learning for Code

Brando Miranda, Avi Shinnar, Vasily Pestun, Barry Trager

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Comments 22 pages

A Minimalist Dataset for Systematic Generalization of Perception, Syntax, and Semantics

Qing Li, Siyuan Huang, Yining Hong, Yixin Zhu, Ying Nian Wu, Song-Chun Zhu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Comments ICLR 2023. website: https://liqing-ustc.github.io/HINT

Knowledge Enhanced Semantic Communication Receiver

Bingyan Wang, Rongpeng Li, Jianhang Zhu, Zhifeng Zhao, Honggang Zhang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.LG

Is Multimodal Vision Supervision Beneficial to Language?

Avinash Madasu, Vasudev Lal

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

LMR: Lane Distance-Based Metric for Trajectory Prediction

Julian Schmidt, Thomas Monninger, Julian Jordan, Klaus Dietmayer

专题命中 推理评测 ：planning(abstract)；分类 cs.AI、cs.LG

Comments Accepted to the 2023 IEEE Intelligent Vehicles Symposium (IV 2023)

Capabilities of GPT-4 on Medical Challenge Problems

Harsha Nori, Nicholas King, Scott Mayer McKinney, Dean Carignan, Eric Horvitz

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 35 pages, 15 figures; added GPT-4-base model results and discussion

Accelerating exploration and representation learning with offline pre-training

Bogdan Mazoure, Jake Bruce, Doina Precup, Rob Fergus, Ankit Anand

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

ConStruct-VL: Data-Free Continual Structured VL Concepts Learning

James Seale Smith, Paola Cascante-Bonilla, Assaf Arbelle, Donghyun Kim, Rameswar Panda, David Cox, Diyi Yang, Zsolt Kira, Rogerio Feris, Leonid Karlinsky

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Comments Accepted by the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2023)

Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks

Tomer Ullman

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments 11 pages, 2 figures

Relational Attention: Generalizing Transformers for Graph-Structured Tasks

Cameron Diao, Ricky Loynd

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Comments The Eleventh International Conference on Learning Representations, ICLR'23

GPT as Knowledge Worker: A Zero-Shot Evaluation of (AI)CPA Capabilities

Jillian Bommarito, Michael Bommarito, Daniel Martin Katz, Jessica Katz

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Source code and data available in online SI at https://github.com/mjbommar/gpt-as-knowledge-worker

Competition-Level Code Generation with AlphaCode

Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, Thomas Hubert, Peter Choy, Cyprien de Masson d'Autume, Igor Babuschkin, Xinyun Chen, Po-Sen Huang, Johannes Welbl, Sven Gowal, Alexey Cherepanov, James Molloy, Daniel J. Mankowitz, Esme Sutherland Robson, Pushmeet Kohli, Nando de Freitas, Koray Kavukcuoglu, Oriol Vinyals

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Comments 74 pages

The Hateful Memes Challenge Next Move

Weijun Jin, Lance Wilhelm

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Despite "super-human" performance, current LLMs are unsuited for decisions about ethics and safety

Joshua Albrecht, Ellie Kitanidis, Abraham J. Fetterman

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments ML Safety Workshop, NeurIPS 2022

Going Beyond XAI: A Systematic Survey for Explanation-Guided Learning

Yuyang Gao, Siyi Gu, Junji Jiang, Sungsoo Ray Hong, Dazhou Yu, Liang Zhao

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Embedding Synthetic Off-Policy Experience for Autonomous Driving via Zero-Shot Curricula

Eli Bronstein, Sirish Srinivasan, Supratik Paul, Aman Sinha, Matthew O'Kelly, Payam Nikdel, Shimon Whiteson

专题命中 推理评测 ：planning(abstract)；分类 cs.AI、cs.LG

Comments Published in CoRL 2022. Main text (8 pages, 3 figures) + acknowledgements and references (3 pages) + appendix (7 pages, 4 figures)

WikiWhy: Answering and Explaining Cause-and-Effect Questions

Matthew Ho, Aditya Sharma, Justin Chang, Michael Saxon, Sharon Levy, Yujie Lu, William Yang Wang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Improving Commonsense in Vision-Language Models via Knowledge Graph Riddles

Shuquan Ye, Yujia Xie, Dongdong Chen, Yichong Xu, Lu Yuan, Chenguang Zhu, Jing Liao

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Comments Code: https://github.com/pleaseconnectwifi/DANCE Project page: shuquanye.com/DANCE_website

ScienceWorld: Is your Agent Smarter than a 5th Grader?

Ruoyao Wang, Peter Jansen, Marc-Alexandre Côté, Prithviraj Ammanabrolu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted to EMNLP 2022

Few-shot Learning with Multilingual Language Models

Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, Ramakanth Pasunuru, Sam Shleifer, Punit Singh Koura, Vishrav Chaudhary, Brian O'Horo, Jeff Wang, Luke Zettlemoyer, Zornitsa Kozareva, Mona Diab, Veselin Stoyanov, Xian Li

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted to EMNLP 2022; 34 pages

Universal and Independent: Multilingual Probing Framework for Exhaustive Model Interpretation and Evaluation

Oleg Serikov, Vitaly Protasov, Ekaterina Voloshina, Viktoria Knyazkova, Tatiana Shavrina

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted to BlackBoxNLP, EMNLP 2022

SafeText: A Benchmark for Exploring Physical Safety in Language Models

Sharon Levy, Emily Allaway, Melanie Subbiah, Lydia Chilton, Desmond Patton, Kathleen McKeown, William Yang Wang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted to EMNLP 2022

WinoGAViL: Gamified Association Benchmark to Challenge Vision-and-Language Models

Yonatan Bitton, Nitzan Bitton Guetta, Ron Yosef, Yuval Elovici, Mohit Bansal, Gabriel Stanovsky, Roy Schwartz

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Accepted to NeurIPS 2022, Datasets and Benchmarks. Website: https://winogavil.github.io/

EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relations

Ahmad Darkhalil, Dandan Shan, Bin Zhu, Jian Ma, Amlan Kar, Richard Higgins, Sanja Fidler, David Fouhey, Dima Damen

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Comments 10 pages main, 38 pages appendix. Accepted at NeurIPS 2022 Track on Datasets and Benchmarks Data, code and leaderboards from: http://epic-kitchens.github.io/VISOR

Use-Case-Grounded Simulations for Explanation Evaluation

Valerie Chen, Nari Johnson, Nicholay Topin, Gregory Plumb, Ameet Talwalkar

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Active entailment encoding for explanation tree construction using parsimonious generation of hard negatives

Alex Bogatu, Zili Zhou, Dónal Landers, André Freitas

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

ValueNet: A New Dataset for Human Value Driven Dialogue System

Liang Qiu, Yizhou Zhao, Jinchao Li, Pan Lu, Baolin Peng, Jianfeng Gao, Song-Chun Zhu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments Paper accepted by AAAI 2022

On the Opportunities and Risks of Foundation Models

Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, Shyamal Buch, Dallas Card, Rodrigo Castellon, Niladri Chatterji, Annie Chen, Kathleen Creel, Jared Quincy Davis, Dora Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy, Kawin Ethayarajh, Li Fei-Fei, Chelsea Finn, Trevor Gale, Lauren Gillespie, Karan Goel, Noah Goodman, Shelby Grossman, Neel Guha, Tatsunori Hashimoto, Peter Henderson, John Hewitt, Daniel E. Ho, Jenny Hong, Kyle Hsu, Jing Huang, Thomas Icard, Saahil Jain, Dan Jurafsky, Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keeling, Fereshte Khani, Omar Khattab, Pang Wei Koh, Mark Krass, Ranjay Krishna, Rohith Kuditipudi, Ananya Kumar, Faisal Ladhak, Mina Lee, Tony Lee, Jure Leskovec, Isabelle Levent, Xiang Lisa Li, Xuechen Li, Tengyu Ma, Ali Malik, Christopher D. Manning, Suvir Mirchandani, Eric Mitchell, Zanele Munyikwa, Suraj Nair, Avanika Narayan, Deepak Narayanan, Ben Newman, Allen Nie, Juan Carlos Niebles, Hamed Nilforoshan, Julian Nyarko, Giray Ogut, Laurel Orr, Isabel Papadimitriou, Joon Sung Park, Chris Piech, Eva Portelance, Christopher Potts, Aditi Raghunathan, Rob Reich, Hongyu Ren, Frieda Rong, Yusuf Roohani, Camilo Ruiz, Jack Ryan, Christopher Ré, Dorsa Sadigh, Shiori Sagawa, Keshav Santhanam, Andy Shih, Krishnan Srinivasan, Alex Tamkin, Rohan Taori, Armin W. Thomas, Florian Tramèr, Rose E. Wang, William Wang, Bohan Wu, Jiajun Wu, Yuhuai Wu, Sang Michael Xie, Michihiro Yasunaga, Jiaxuan You, Matei Zaharia, Michael Zhang, Tianyi Zhang, Xikun Zhang, Yuhui Zhang, Lucia Zheng, Kaitlyn Zhou, Percy Liang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Comments Authored by the Center for Research on Foundation Models (CRFM) at the Stanford Institute for Human-Centered Artificial Intelligence (HAI). Report page with citation guidelines: https://crfm.stanford.edu/report.html

Evaluating Understanding on Conceptual Abstraction Benchmarks

Victor Vikram Odouard, Melanie Mitchell

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI、cs.LG

Comments EBeM'22: AI Evaluation Beyond Metrics, July 24, 2022, Vienna, Austria

Explaining Answers with Entailment Trees

Bhavana Dalvi, Peter Jansen, Oyvind Tafjord, Zhengnan Xie, Hannah Smith, Leighanna Pipatanangkura, Peter Clark

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL、cs.AI

Comments published in EMNLP 2021