Center for Data Science, New York University(纽约大学数据科学中心);Genentech(基因泰克公司);Stanford University(斯坦福大学)
Taro Makino, Ji Won Park, Natasa Tagasovska, Takamasa Kudo, Paula Coelho, Jan-Christian Huetter, Heming Yao, Burkhard Hoeckendorf, Ana Carolina Leote, Stephen R… 展开作者
Taro Makino, Ji Won Park, Natasa Tagasovska, Takamasa Kudo, Paula Coelho, Jan-Christian Huetter, Heming Yao, Burkhard Hoeckendorf, Ana Carolina Leote, Stephen Ra, David Richmond, Kyunghyun Cho, Aviv Regev, Romain Lopez
Real-world datasets often combine data collected under different experimental conditions. This yields larger datasets, but also introduces spurious correlations that make it difficult to model the phenomena of interest. We address this by learning two embeddings to independently represent the phenomena of interest and the spurious correlations. The embedding representing the phenomena of interest is correlated with the target variable $y$, and is invariant to the environment variable $e$. In contrast, the embedding representing the spurious correlations is correlated with $e$. The invariance to $e$ is difficult to achieve on real-world datasets. Our primary contribution is an algorithm called Supervised Contrastive Block Disentanglement (SCBD) that effectively enforces this invariance. It is based purely on Supervised Contrastive Learning, and applies to real-world data better than existing approaches. We empirically validate SCBD on two challenging problems. The first problem is domain generalization, where we achieve strong performance on a synthetic dataset, as well as on Camelyon17-WILDS. We introduce a single hyperparameter $α$ to control the degree of invariance to $e$. When we increase $α$ to strengthen the degree of invariance, out-of-distribution performance improves at the expense of in-distribution performance. The second problem is batch correction, in which we apply SCBD to preserve biological signal and remove inter-well batch effects when modeling single-cell perturbations from 26 million Optical Pooled Screening images.
Block Disentanglement in CRL: Bridging Identifiability and Visual State Estimation
Rensselaer Polytechnic Institute(伦斯勒理工学院);The University of Texas at Austin(德克萨斯大学奥斯汀分校);Swiss AI, EPFL(瑞士人工智能,洛桑联邦理工学院);Google DeepMind(谷歌DeepMind);University of Delaware(特拉华大学)
Text Attribute Control via Closed-Loop Disentanglement
Institute of Artificial Intelligence, Beihang University(北京航空航天大学人工智能研究院);University of Oxford(牛津大学);Zhongguancun Laboratory(中关村实验室);Vienna University of Technology(维也纳工业大学)