Trevor Darrell

Self-correcting LLM-controlled Diffusion Models

Tsung-Han Wu, Long Lian, Joseph E. Gonzalez, Boyi Li, Trevor Darrell

Comments 16 pages, 10 figures

From Wrong To Right: A Recursive Approach Towards Vision-Language Explanation

Jiaxin Ge, Sanjay Subramanian, Trevor Darrell, Boyi Li

Comments EMNLP 2023 Main

Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game

Sam Toyer, Olivia Watkins, Ethan Adrian Mendes, Justin Svegliato, Luke Bailey, Tiffany Wang, Isaac Ong, Karim Elmaaroufi, Pieter Abbeel, Trevor Darrell, Alan Ritter, Stuart Russell

Diversify Your Vision Datasets with Automatic Diffusion-Based Augmentation

Lisa Dunlap, Alyssa Umino, Han Zhang, Jiezhi Yang, Joseph E. Gonzalez, Trevor Darrell

Comments Update: replaced Planes dataset with Waterbirds & updated results after bug fix

CLAIR: Evaluating Image Captions with Large Language Models

David Chan, Suzanne Petryk, Joseph E. Gonzalez, Trevor Darrell, John Canny

Comments To Appear at EMNLP 2023

Incorporating Structured Representations into Pretrained Vision & Language Models Using Scene Graphs

Roei Herzig, Alon Mendelson, Leonid Karlinsky, Assaf Arbelle, Rogerio Feris, Trevor Darrell, Amir Globerson

Comments EMNLP 2023

Large Language Models are Visual Reasoning Coordinators

Liangyu Chen, Bo Li, Sheng Shen, Jingkang Yang, Chunyuan Li, Kurt Keutzer, Trevor Darrell, Ziwei Liu

Comments Accepted at NeurIPS 2023

QDTrack: Quasi-Dense Similarity Learning for Appearance-Only Multiple Object Tracking

Tobias Fischer, Thomas E. Huang, Jiangmiao Pang, Linlu Qiu, Haofeng Chen, Trevor Darrell, Fisher Yu

Aligning Large Multimodal Models with Factually Augmented RLHF

Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liang-Yan Gui, Yu-Xiong Wang, Yiming Yang, Kurt Keutzer, Trevor Darrell

Comments Preprint

Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learning

Colorado J. Reed, Ritwik Gupta, Shufan Li, Sarah Brockman, Christopher Funk, Brian Clipp, Kurt Keutzer, Salvatore Candido, Matt Uyttendaele, Trevor Darrell

Comments International Conference on Computer Vision 2023

Guiding Pretraining in Reinforcement Learning with Large Language Models

Yuqing Du, Olivia Watkins, Zihan Wang, Cédric Colas, Trevor Darrell, Pieter Abbeel, Abhishek Gupta, Jacob Andreas

Comments ICML 2023

VideoCutLER: Surprisingly Simple Unsupervised Video Instance Segmentation

Xudong Wang, Ishan Misra, Ziyun Zeng, Rohit Girdhar, Trevor Darrell

Comments Preprint. Code: https://github.com/facebookresearch/CutLER

DETReg: Unsupervised Pretraining with Region Priors for Object Detection

Amir Bar, Xin Wang, Vadim Kantorov, Colorado J Reed, Roei Herzig, Gal Chechik, Anna Rohrbach, Trevor Darrell, Amir Globerson

Comments Project page: https://www.amirbar.net/detreg/

TOAST: Transfer Learning via Attention Steering

Baifeng Shi, Siyu Gai, Trevor Darrell, Xin Wang

Comments Code is available at https://github.com/bfshi/TOAST

Mixture-of-Experts Meets Instruction Tuning:A Winning Combination for Large Language Models

Sheng Shen, Le Hou, Yanqi Zhou, Nan Du, Shayne Longpre, Jason Wei, Hyung Won Chung, Barret Zoph, William Fedus, Xinyun Chen, Tu Vu, Yuexin Wu, Wuyang Chen, Albert Webson, Yunxuan Li, Vincent Zhao, Hongkun Yu, Kurt Keutzer, Trevor Darrell, Denny Zhou

Comments Preprint

Back to the Source: Diffusion-Driven Test-Time Adaptation

Jin Gao, Jialing Zhang, Xihui Liu, Trevor Darrell, Evan Shelhamer, Dequan Wang

Comments published at CVPR 2023

Modular Visual Question Answering via Code Generation

Sanjay Subramanian, Medhini Narasimhan, Kushal Khangaonkar, Kevin Yang, Arsha Nagrani, Cordelia Schmid, Andy Zeng, Trevor Darrell, Dan Klein

Comments ACL 2023

Dropout Reduces Underfitting

Zhuang Liu, Zhiqiu Xu, Joseph Jin, Zhiqiang Shen, Trevor Darrell

Comments ICML 2023

Simple Token-Level Confidence Improves Caption Correctness

Suzanne Petryk, Spencer Whitehead, Joseph E. Gonzalez, Trevor Darrell, Anna Rohrbach, Marcus Rohrbach

Using Language to Extend to Unseen Domains

Lisa Dunlap, Clara Mohri, Devin Guillory, Han Zhang, Trevor Darrell, Joseph E. Gonzalez, Aditi Raghunathan, Anja Rohrbach

Top-Down Visual Attention from Analysis by Synthesis

Baifeng Shi, Trevor Darrell, Xin Wang

Comments CVPR2023 highlight; Project page: https://sites.google.com/view/absvit

Learning and Verification of Task Structure in Instructional Videos

Medhini Narasimhan, Licheng Yu, Sean Bell, Ning Zhang, Trevor Darrell

Comments Wesbite at https://medhini.github.io/task_structure

Scaling Vision-Language Models with Sparse Mixture of Experts

Sheng Shen, Zhewei Yao, Chunyuan Li, Trevor Darrell, Kurt Keutzer, Yuxiong He

Comments Preprint

Teachable Reinforcement Learning via Advice Distillation

Olivia Watkins, Trevor Darrell, Pieter Abbeel, Jacob Andreas, Abhishek Gupta

Comments Published at NeurIPS 2021

Refine and Represent: Region-to-Object Representation Learning

Akash Gokul, Konstantinos Kallidromitis, Shufan Li, Yusuke Kato, Kazuki Kozuka, Trevor Darrell, Colorado J Reed

Multitask Vision-Language Prompt Tuning

Sheng Shen, Shijia Yang, Tianjun Zhang, Bohan Zhai, Joseph E. Gonzalez, Kurt Keutzer, Trevor Darrell

Comments Preprint

More Control for Free! Image Synthesis with Semantic Diffusion Guidance

Xihui Liu, Dong Huk Park, Samaneh Azadi, Gong Zhang, Arman Chopikyan, Yuxiao Hu, Humphrey Shi, Anna Rohrbach, Trevor Darrell

Comments WACV 2023. Project page https://xh-liu.github.io/sdg/

Watch Those Words: Video Falsification Detection Using Word-Conditioned Facial Motion

Shruti Agarwal, Liwen Hu, Evonne Ng, Trevor Darrell, Hao Li, Anna Rohrbach

Comments Accepted in WACV 2023

Bringing Image Scene Structure to Video via Frame-Clip Consistency of Object Tokens

Elad Ben-Avraham, Roei Herzig, Karttikeya Mangalam, Amir Bar, Anna Rohrbach, Leonid Karlinsky, Trevor Darrell, Amir Globerson

Comments Tech report

G^3: Geolocation via Guidebook Grounding

Grace Luo, Giscard Biamby, Trevor Darrell, Daniel Fried, Anna Rohrbach

Comments Findings of EMNLP 2022