Jitendra Malik

Sequential Modeling Enables Scalable Learning for Large Vision Models

Yutong Bai, Xinyang Geng, Karttikeya Mangalam, Amir Bar, Alan Yuille, Trevor Darrell, Jitendra Malik, Alexei A Efros

Comments Website: https://yutongbai.com/lvm.html

GOAT: GO to Any Thing

Matthew Chang, Theophile Gervet, Mukul Khanna, Sriram Yenamandra, Dhruv Shah, So Yeon Min, Kavit Shah, Chris Paxton, Saurabh Gupta, Dhruv Batra, Roozbeh Mottaghi, Jitendra Malik, Devendra Singh Chaplot

Conformal Policy Learning for Sensorimotor Control Under Distribution Shifts

Huang Huang, Satvik Sharma, Antonio Loquercio, Anastasios Angelopoulos, Ken Goldberg, Jitendra Malik

Comments Conformal Policy Learning

Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots

Xavier Puig, Eric Undersander, Andrew Szot, Mikael Dallaire Cote, Tsung-Yen Yang, Ruslan Partsey, Ruta Desai, Alexander William Clegg, Michal Hlavac, So Yeon Min, Vladimír Vondruš, Theophile Gervet, Vincent-Pierre Berges, John M. Turner, Oleksandr Maksymets, Zsolt Kira, Mrinal Kalakrishnan, Jitendra Malik, Devendra Singh Chaplot, Unnat Jain, Dhruv Batra, Akshara Rai, Roozbeh Mottaghi

Comments Project page: http://aihabitat.org/habitat3

Speculative Decoding with Big Little Decoder

Sehoon Kim, Karttikeya Mangalam, Suhong Moon, Jitendra Malik, Michael W. Mahoney, Amir Gholami, Kurt Keutzer

Comments NeurIPS 2023

General In-Hand Object Rotation with Vision and Touch

Haozhi Qi, Brent Yi, Sudharshan Suresh, Mike Lambeta, Yi Ma, Roberto Calandra, Jitendra Malik

Comments CoRL 2023; Website: https://haozhi.io/rotateit/

Humans in 4D: Reconstructing and Tracking Humans with Transformers

Shubham Goel, Georgios Pavlakos, Jathushan Rajasegaran, Angjoo Kanazawa, Jitendra Malik

Comments In ICCV 2023. Project Webpage: https://shubham-goel.github.io/4dhumans/

Learning Vision-based Pursuit-Evasion Robot Policies

Andrea Bajcsy, Antonio Loquercio, Ashish Kumar, Jitendra Malik

Comments Includes Supplementary. Project webpage at https://abajcsy.github.io/vision-based-pursuit/

EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understanding

Karttikeya Mangalam, Raiymbek Akshulakov, Jitendra Malik

Comments https://egoschema.github.io/

On the Benefits of 3D Pose and Tracking for Human Action Recognition

Jathushan Rajasegaran, Georgios Pavlakos, Angjoo Kanazawa, Christoph Feichtenhofer, Jitendra Malik

Comments CVPR2023 (project page: https://brjathu.github.io/LART)

MAViL: Masked Audio-Video Learners

Po-Yao Huang, Vasu Sharma, Hu Xu, Chaitanya Ryali, Haoqi Fan, Yanghao Li, Shang-Wen Li, Gargi Ghosh, Jitendra Malik, Christoph Feichtenhofer

Comments Technical report

Learning Space-Time Semantic Correspondences

Du Tran, Jitendra Malik

Hiera: A Hierarchical Vision Transformer without the Bells-and-Whistles

Chaitanya Ryali, Yuan-Ting Hu, Daniel Bolya, Chen Wei, Haoqi Fan, Po-Yao Huang, Vaibhav Aggarwal, Arkabandhu Chowdhury, Omid Poursaeed, Judy Hoffman, Jitendra Malik, Yanghao Li, Christoph Feichtenhofer

Comments ICML 2023 Oral version. Code+Models: https://github.com/facebookresearch/hiera

Learning a Single Near-hover Position Controller for Vastly Different Quadcopters

Dingqi Zhang, Antonio Loquercio, Xiangyu Wu, Ashish Kumar, Jitendra Malik, Mark W. Mueller

Comments Comprehensive video results can be found on the project webpage https://dz298.github.io/universal-drone-controller/

Navigating to Objects Specified by Images

Jacob Krantz, Theophile Gervet, Karmesh Yadav, Austin Wang, Chris Paxton, Roozbeh Mottaghi, Dhruv Batra, Jitendra Malik, Stefan Lee, Devendra Singh Chaplot

CA$^2$T-Net: Category-Agnostic 3D Articulation Transfer from Single Image

Jasmine Collins, Anqi Liang, Jitendra Malik, Hao Zhang, Frédéric Devernay

Comments 8 pages

Decoupling Human and Camera Motion from Videos in the Wild

Vickie Ye, Georgios Pavlakos, Jitendra Malik, Angjoo Kanazawa

Comments Project site: https://vye16.github.io/slahmr. CVPR 2023

Reversible Vision Transformers

Karttikeya Mangalam, Haoqi Fan, Yanghao Li, Chao-Yuan Wu, Bo Xiong, Christoph Feichtenhofer, Jitendra Malik

Comments Oral at CVPR 2022, updated version

Multiview Compressive Coding for 3D Reconstruction

Chao-Yuan Wu, Justin Johnson, Jitendra Malik, Christoph Feichtenhofer, Georgia Gkioxari

Comments Project page: https://mcc3d.github.io/

Navigating to Objects in the Real World

Theophile Gervet, Soumith Chintala, Dhruv Batra, Jitendra Malik, Devendra Singh Chaplot

Comments 39 pages, 19 figures and tables, submitted to Science Robotics

MeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient Long-Term Video Recognition

Chao-Yuan Wu, Yanghao Li, Karttikeya Mangalam, Haoqi Fan, Bo Xiong, Jitendra Malik, Christoph Feichtenhofer

Comments Technical report. arXiv v2: add link to code

Instance-Specific Image Goal Navigation: Training Embodied Agents to Find Object Instances

Jacob Krantz, Stefan Lee, Jitendra Malik, Dhruv Batra, Devendra Singh Chaplot

Learning to Imitate Object Interactions from Internet Videos

Austin Patel, Andrew Wang, Ilija Radosavovic, Jitendra Malik

Comments Project page: https://austinapatel.github.io/imitate-video

Legged Locomotion in Challenging Terrains using Egocentric Vision

Ananye Agarwal, Ashish Kumar, Jitendra Malik, Deepak Pathak

Comments Oral presentation at CoRL 2022. Website at https://vision-locomotion.github.io

Learning Visual Locomotion with Cross-Modal Supervision

Antonio Loquercio, Ashish Kumar, Jitendra Malik

Comments Learning to walk from pixels in the real world by using proprioception as supervision. Project page for videos and code: https://antonilo.github.io/vision_locomotion/

Squeezeformer: An Efficient Transformer for Automatic Speech Recognition

Sehoon Kim, Amir Gholami, Albert Shaw, Nicholas Lee, Karttikeya Mangalam, Jitendra Malik, Michael W. Mahoney, Kurt Keutzer

Comments NeurIPS 2022

In-Hand Object Rotation via Rapid Motor Adaptation

Haozhi Qi, Ashish Kumar, Roberto Calandra, Yi Ma, Jitendra Malik

Comments CoRL 2022. Code and Website: https://haozhi.io/hora

Real-World Robot Learning with Masked Visual Pre-training

Ilija Radosavovic, Tete Xiao, Stephen James, Pieter Abbeel, Jitendra Malik, Trevor Darrell

Comments CoRL 2022; Project page: https://tetexiao.com/projects/real-mvp

Learning to Learn with Generative Models of Neural Network Checkpoints

William Peebles, Ilija Radosavovic, Tim Brooks, Alexei A. Efros, Jitendra Malik

Comments Code available at https://www.github.com/wpeebles/G.pt . Project page and videos available at https://www.wpeebles.com/Gpt

Differentiable Stereopsis: Meshes from multiple views using differentiable rendering

Shubham Goel, Georgia Gkioxari, Jitendra Malik

Comments In CVPR2022. Project webpage: https://shubham-goel.github.io/ds/

Journal ref In CVPR 2022 (pp. 8635-8644)