Out-of-distribution generalization in neural networks is often hampered by spurious correlations. A common strategy is to mitigate this by removing spurious concepts from the neural network representation of the data. Existing concept-removal methods tend to be overzealous by inadvertently eliminating features associated with the main task of the model, thereby harming model performance. We propose an iterative algorithm that separates spurious from main-task concepts by jointly identifying two low-dimensional orthogonal subspaces in the neural network representation. We evaluate the algorithm on benchmark datasets for computer vision (Waterbirds, CelebA) and natural language processing (MultiNLI), and show that it outperforms existing concept removal methods
Projecting Out the Malice: A Global Subspace Approach to LLM Detoxification
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences(中国科学院人工智能安全国家重点实验室,计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Dalian University of Technology(大连理工大学)
Department of Machine Learning and Neural Computing(机器学习与神经计算系);Donders Institute for Brain, Cognition and Behaviour(大脑、认知与行为研究所);Radboud University(拉德堡德大学)
Safe and Reliable Diffusion Models via Subspace Projection
Faculty of Data Science, City University of Macau(澳门大学数据科学学院);School of Computer Science, University of Queensland(昆士兰大学计算机科学学院);Shandong Computer Science Center, Qilu University of Technology (Shandong Academy of Sciences)(齐鲁工业大学(山东省科学院)山东省计算中心)
nASR: An End-to-End Trainable Neural Layer for Channel-Level EEG Artifact Subspace Reconstruction in Real-Time BCI
Doctoral Candidate of Electrical & Computer Engineering, Univ. of Houston(电气与计算机工程博士候选人,休斯顿大学);Faculty of Electrical & Computer Engineering, Univ. of Houston(电气与计算机工程系,休斯顿大学)
Identifying Exoplanets with Deep Learning. IV. Removing Stellar Activity Signals from Radial Velocity Measurements Using Neural Networks
Massachusetts Institute of Technology(麻省理工学院);University of Texas at Austin(德克萨斯大学奥斯汀分校);University of Wisconsin-Madison(威斯康星大学麦迪逊分校);Université de Genève(日内瓦大学);Observatoire de Genève(日内瓦天文台);University of St Andrews(圣安德鲁斯大学);Facebook(脸书);Technical University of Denmark(丹麦技术大学);DTU Space, National Space Institute(丹麦技术大学国家空间研究所)
精确径向速度(RV)观测的系外行星探测目前受到恒星活动引入的虚假RV信号的限制。我们表明,线性回归和神经网络等机器学习技术可以有效地从RV观测中去除活动信号(由于星点/光斑)。以前的努力集中在使用高斯过程回归等建模技术及时仔细过滤活动信号(例如Haywood等人2014)。相反,我们系统地去除活动信号,只使用谱线的平均形状的变化,而没有关于何时收集观测结果的信息。我们在模拟数据(使用SOAP 2.0软件生成; Dumusque et al. 2014)和HARPS-N太阳望远镜对太阳的观测(Dumusque et al. 2015; Phillips et al. 2016; Collier Cameron et al. 2019)上训练了我们的机器学习模型。我们发现,这些技术可以预测和删除模拟数据(改善RV散射从82厘米/秒到3厘米/秒)和超过600个实际观测采取了近三年的HARPS-N太阳望远镜(改善RV散射从1.753米/秒到1.039米/秒,约1.7改善的一个因素)。在未来,这些或类似的技术可以从我们太阳系以外的恒星观测中去除活动信号,并最终帮助探测类太阳恒星周围的可居住区地球质量的系外行星。
Project and Generate: Divergence-Free Neural Operators for Incompressible Flows
Incubation Institute, Fudan University, Shanghai, China(复旦大学上海孵化院);Shanghai Academy of Artificial Intelligence for Science, Shanghai, China(上海人工智能科学研究院)