Graham Neubig

Language Modeling with Editable External Knowledge

Belinda Z. Li, Emmy Liu, Alexis Ross, Abbas Zeitoun, Graham Neubig, Jacob Andreas

Wav2Gloss: Generating Interlinear Glossed Text from Speech

Taiqi He, Kwanghee Choi, Lindia Tjuatja, Nathaniel R. Robinson, Jiatong Shi, Shinji Watanabe, Graham Neubig, David R. Mortensen, Lori Levin

Comments ACL 2024 camera ready version

VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks

Jing Yu Koh, Robert Lo, Lawrence Jang, Vikram Duvvur, Ming Chong Lim, Po-Yu Huang, Graham Neubig, Shuyan Zhou, Ruslan Salakhutdinov, Daniel Fried

Comments Accepted to ACL 2024. 24 pages. Project page: https://jykoh.com/vwa

Instruction-tuned Language Models are Better Knowledge Learners

Zhengbao Jiang, Zhiqing Sun, Weijia Shi, Pedro Rodriguez, Chunting Zhou, Graham Neubig, Xi Victoria Lin, Wen-tau Yih, Srinivasan Iyer

Comments ACL 2024. The reproduced data for this paper is available at https://github.com/Edward-Sun/PIT

Better Synthetic Data by Retrieving and Transforming Existing Datasets

Saumya Gandhi, Ritu Gala, Vijay Viswanathan, Tongshuang Wu, Graham Neubig

Comments PDF fixed in v3

SOTOPIA-$π$: Interactive Learning of Socially Intelligent Language Agents

Ruiyi Wang, Haofei Yu, Wenxin Zhang, Zhengyang Qi, Maarten Sap, Graham Neubig, Yonatan Bisk, Hao Zhu

Learning Performance-Improving Code Edits

Alexander Shypula, Aman Madaan, Yimeng Zeng, Uri Alon, Jacob Gardner, Milad Hashemi, Graham Neubig, Parthasarathy Ranganathan, Osbert Bastani, Amir Yazdanbakhsh

Comments Published as a conference paper at ICLR 2024 (Spotlight). Project website: https://pie4perf.com/

WebArena: A Realistic Web Environment for Building Autonomous Agents

Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, Graham Neubig

Comments Our code, data, environment reproduction resources, and video demonstrations are publicly available at https://webarena.dev/

VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding?

Junpeng Liu, Yifan Song, Bill Yuchen Lin, Wai Lam, Graham Neubig, Yuanzhi Li, Xiang Yue

CMULAB: An Open-Source Framework for Training and Deployment of Natural Language Processing Models

Zaid Sheikh, Antonios Anastasopoulos, Shruti Rijhwani, Lindia Tjuatja, Robbie Jimerson, Graham Neubig

Comments Live demo at https://cmulab.dev

What Are Tools Anyway? A Survey from the Language Model Perspective

Zhiruo Wang, Zhoujun Cheng, Hao Zhu, Daniel Fried, Graham Neubig

SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents

Xuhui Zhou, Hao Zhu, Leena Mathur, Ruohong Zhang, Haofei Yu, Zhengyang Qi, Louis-Philippe Morency, Yonatan Bisk, Daniel Fried, Graham Neubig, Maarten Sap

Comments Preprint, 43 pages. The first two authors contribute equally

Multitask Learning Can Improve Worst-Group Outcomes

Atharva Kulkarni, Lucio Dery, Amrith Setlur, Aditi Raghunathan, Ameet Talwalkar, Graham Neubig

Comments 20 pages, 7 tables, 6 Figures

Multi-Dimensional Evaluation of Text Summarization with In-Context Learning

Sameer Jain, Vaishakh Keshava, Swarnashree Mysore Sathyendra, Patrick Fernandes, Pengfei Liu, Graham Neubig, Chunting Zhou

Comments ACL Findings '23

Journal ref Findings of the Association for Computational Linguistics - ACL 2023, pp 8487-8495

Do LLMs exhibit human-like response biases? A case study in survey design

Lindia Tjuatja, Valerie Chen, Sherry Tongshuang Wu, Ameet Talwalkar, Graham Neubig

Can Large Language Models be Trusted for Evaluation? Scalable Meta-Evaluation of LLMs as Evaluators via Agent Debate

Steffi Chern, Ethan Chern, Graham Neubig, Pengfei Liu

Solving NLP Problems through Human-System Collaboration: A Discussion-based Approach

Masahiro Kaneko, Graham Neubig, Naoaki Okazaki

Comments EACL2024 Findings

TroVE: Inducing Verifiable and Efficient Toolboxes for Solving Programmatic Tasks

Zhiruo Wang, Daniel Fried, Graham Neubig

An In-depth Look at Gemini's Language Abilities

Syeda Nahida Akter, Zichun Yu, Aashiq Muhamed, Tianyue Ou, Alex Bäuerle, Ángel Alexander Cabrera, Krish Dholakia, Chenyan Xiong, Graham Neubig

Program-Aided Reasoners (better) Know What They Know

Anubha Kabra, Sanketh Rangreji, Yash Mathur, Aman Madaan, Emmy Liu, Graham Neubig

Learning to Filter Context for Retrieval-Augmented Generation

Zhiruo Wang, Jun Araki, Zhengbao Jiang, Md Rizwan Parvez, Graham Neubig

DeMuX: Data-efficient Multilingual Learning

Simran Khanuja, Srinivas Gowriraj, Lucio Dery, Graham Neubig

CodeBERTScore: Evaluating Code Generation with Pretrained Models of Code

Shuyan Zhou, Uri Alon, Sumit Agarwal, Graham Neubig

Unlimiformer: Long-Range Transformers with Unlimited Length Input

Amanda Bertsch, Uri Alon, Graham Neubig, Matthew R. Gormley

Comments NeurIPS 2023

Teacher Perception of Automatically Extracted Grammar Concepts for L2 Language Learning

Aditi Chaudhary, Arun Sampath, Ashwin Sheshadri, Antonios Anastasopoulos, Graham Neubig

Comments Accepted at EMNLP Findings 2023. arXiv admin note: substantial text overlap with arXiv:2206.05154

Crossing the Threshold: Idiomatic Machine Translation through Retrieval Augmentation and Loss Weighting

Emmy Liu, Aditi Chaudhary, Graham Neubig

Comments EMNLP 2023

Active Retrieval Augmented Generation

Zhengbao Jiang, Frank F. Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, Graham Neubig

Comments EMNLP 2023

It's MBR All the Way Down: Modern Generation Techniques Through the Lens of Minimum Bayes Risk

Amanda Bertsch, Alex Xie, Graham Neubig, Matthew R. Gormley

Comments Under submission

Procedures as Programs: Hierarchical Control of Situated Agents through Natural Language

Shuyan Zhou, Pengcheng Yin, Graham Neubig

ChatGPT MT: Competitive for High- (but not Low-) Resource Languages

Nathaniel R. Robinson, Perez Ogayo, David R. Mortensen, Graham Neubig

Comments 27 pages, 9 figures, 14 tables