Diyi Yang

SWE-smith: Scaling Data for Software Engineering Agents

John Yang, Kilian Lieret, Carlos E. Jimenez, Alexander Wettig, Kabir Khandpur, Yanzhe Zhang, Binyuan Hui, Ofir Press, Ludwig Schmidt, Diyi Yang

机构 * Stanford University（斯坦福大学） ； Princeton University（普林斯顿大学） ； Alibaba Qwen（阿里巴巴文言） 纠错

Comments All assets available at https://swesmith.com

AudioJudge: Understanding What Works in Large Audio Model Based Speech Evaluation

Potsawee Manakul, Woody Haosheng Gan, Michael J. Ryan, Ali Sartaz Khan, Warit Sirichotedumrong, Kunat Pipatanakul, William Held, Diyi Yang

机构 * SCB 10X, SCBX Group（SCB 10X、SCBX集团） ； University of Southern California（南加州大学） ； Stanford University（斯坦福大学） 纠错

Personalization of Large Language Models: A Survey

Zhehao Zhang, Ryan A. Rossi, Branislav Kveton, Yijia Shao, Diyi Yang, Hamed Zamani, Franck Dernoncourt, Joe Barrow, Tong Yu, Sungchul Kim, Ruiyi Zhang, Jiuxiang Gu, Tyler Derr, Hongjie Chen, Junda Wu, Xiang Chen, Zichao Wang, Subrata Mitra, Nedim Lipka, Nesreen Ahmed, Yu Wang

机构 * Dartmouth College（达特茅斯学院） ； Adobe Research（Adobe研究） ； Stanford University（斯坦福大学） ； University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校） ； Pattern Data ； Vanderbilt University（范德比尔特大学） ； Dolby Research（Dolby研究） ； University of California San Diego（加州大学圣地亚哥分校） ； Cisco Research（思科研究） ； University of Oregon（俄勒冈大学） 纠错

Comments Accepted at the Transactions on Machine Learning Research (TMLR) journal

The Ideation-Execution Gap: Execution Outcomes of LLM-Generated versus Human Research Ideas

Chenglei Si, Tatsunori Hashimoto, Diyi Yang

机构 * Stanford University（斯坦福大学） 纠错

Comments main paper is 14 pages

EgoNormia: Benchmarking Physical Social Norm Understanding

MohammadHossein Rezaei, Yicheng Fu, Phil Cuvin, Caleb Ziems, Yanzhe Zhang, Hao Zhu, Diyi Yang

Comments V4, fixes to title and formatting

When Models Know More Than They Can Explain: Quantifying Knowledge Transfer in Human-AI Collaboration

Quan Shi, Carlos E. Jimenez, Shunyu Yao, Nick Haber, Diyi Yang, Karthik Narasimhan

机构 * Princeton Language and Intelligence（普林斯顿语言与智能研究所） ； Stanford University（斯坦福大学） ； OpenAI（开放人工智能研究所） 纠错

Comments For code, data, visualizer, visit: https://kite-live.vercel.app

SynthesizeMe! Inducing Persona-Guided Prompts for Personalized Reward Models in LLMs

Michael J Ryan, Omar Shaikh, Aditri Bhagirath, Daniel Frees, William Held, Diyi Yang

机构 * Stanford University（斯坦福大学） ； Georgia Institute of Technology（佐治亚理工学院） 纠错

Comments ACL 2025 Main Conference

Attacking Vision-Language Computer Agents via Pop-ups

Yanzhe Zhang, Tao Yu, Diyi Yang

机构 * Georgia Tech（佐治亚理工学院） ； The University of Hong Kong（香港大学） ； Stanford University（斯坦福大学） 纠错

Comments ACL 2025

Auditing Gender Presentation Differences in Text-to-Image Models

Yanzhe Zhang, Lu Jiang, Greg Turk, Diyi Yang

Comments Preprint, 23 pages, 14 figures. Project page at https://salt-nlp.github.io/GEP/

Journal ref EAAMO '24: Proceedings of the 4th ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization, 2024

No Preference Left Behind: Group Distributional Preference Optimization

Binwei Yao, Zefan Cai, Yun-Shiuan Chuang, Shanglin Yang, Ming Jiang, Diyi Yang, Junjie Hu

机构 * University of Wisconsin-Madison（威斯康星大学麦迪逊分校） ； Indiana University Indianapolis（印第安纳大学印第安纳波利斯分校） ； Stanford University（斯坦福大学） 纠错

Aligning Language Models with Demonstrated Feedback

Omar Shaikh, Michelle S. Lam, Joey Hejna, Yijia Shao, Hyundong Cho, Michael S. Bernstein, Diyi Yang

机构 * Stanford University（斯坦福大学） ； USC（美国南加州大学） 纠错

Comments ICLR 2025; 28 pages, 8 figures

Challenges and Paths Towards AI for Software Engineering

Alex Gu, Naman Jain, Wen-Ding Li, Manish Shetty, Yijia Shao, Ziyang Li, Diyi Yang, Kevin Ellis, Koushik Sen, Armando Solar-Lezama

Comments 75 pages

PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action

Yijia Shao, Tianshi Li, Weiyan Shi, Yanchen Liu, Diyi Yang

Comments NeurIPS 2024 Datasets and Benchmarks Track

Helping the Helper: Supporting Peer Counselors via AI-Empowered Practice and Feedback

Shang-Ling Hsu, Raj Sanjay Shah, Prathik Senthil, Zahra Ashktorab, Casey Dugan, Werner Geyer, Diyi Yang

Comments 45 pages, 14 figures, CSCW 2025

Mind the Gap! Static and Interactive Evaluations of Large Audio Models

Minzhi Li, William Barr Held, Michael J Ryan, Kunat Pipatanakul, Potsawee Manakul, Hao Zhu, Diyi Yang

The Call for Socially Aware Language Technologies

Diyi Yang, Dirk Hovy, David Jurgens, Barbara Plank

Comments pre-MIT Press publication version

Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering

Chenglei Si, Yanzhe Zhang, Ryan Li, Zhengyuan Yang, Ruibo Liu, Diyi Yang

Comments NAACL 2025; The first two authors contributed equally

Dynamic Skill Adaptation for Large Language Models

Jiaao Chen, Diyi Yang

A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaboration

Zijun Liu, Yanzhe Zhang, Peng Li, Yang Liu, Diyi Yang

Comments Published in COLM2024. Code Repo: https://github.com/SALT-NLP/DyLAN

Semi-Truths: A Large-Scale Dataset of AI-Augmented Images for Evaluating Robustness of AI-Generated Image detectors

Anisha Pal, Julia Kruk, Mansi Phute, Manognya Bhattaram, Diyi Yang, Duen Horng Chau, Judy Hoffman

Comments Accepted at NeurIPS 2024 Track Datasets & Benchmarks Track

Sketch2Code: Evaluating Vision-Language Models for Interactive Web Design Prototyping

Ryan Li, Yanzhe Zhang, Diyi Yang

Comments preprint, 9 pages

Benchmarking Machine Translation with Cultural Awareness

Binwei Yao, Ming Jiang, Tara Bobinac, Diyi Yang, Junjie Hu

Decoding Susceptibility: Modeling Misbelief to Misinformation Through a Computational Approach

Yanchen Liu, Mingyu Derek Ma, Wenna Qin, Azure Zhou, Jiaao Chen, Weiyan Shi, Wei Wang, Diyi Yang

Comments EMNLP 2024

SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains?

John Yang, Carlos E. Jimenez, Alex L. Zhang, Kilian Lieret, Joyce Yang, Xindi Wu, Ori Press, Niklas Muennighoff, Gabriel Synnaeve, Karthik R. Narasimhan, Diyi Yang, Sida I. Wang, Ofir Press

Distilling an End-to-End Voice Assistant Without Instruction Training Data

William Held, Ella Li, Michael Ryan, Weiyan Shi, Yanzhe Zhang, Diyi Yang

Are Large Language Models Consistent over Value-laden Questions?

Jared Moore, Tanvi Deshpande, Diyi Yang

Comments 9 pages, 10 figures, In Findings of EMNLP 2024

Towards Social AI: A Survey on Understanding Social Interactions

Sangmin Lee, Minzhi Li, Bolin Lai, Wenqi Jia, Fiona Ryan, Xu Cao, Ozgur Kara, Bikram Boote, Weiyan Shi, Diyi Yang, James M. Rehg

Understanding #vent Channels on Discord

Kayode Oladeji, Tony Wang, Diyi Yang, Amy Bruckman

Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers

Chenglei Si, Diyi Yang, Tatsunori Hashimoto

Comments main paper is 20 pages

Understanding Online Discussion Across Difference: Insights from Gun Discourse on Reddit

Rijul Magu, Nivedhitha Mathan Kumar, Yihe Liu, Xander Koo, Diyi Yang, Amy Bruckman

Comments CSCW 2024