Test-Time Training with KV Binding Is Secretly Linear Attention
NVIDIA, Toronto, Ontario, Canada(NVIDIA,多伦多,安大略省,加拿大);University of Toronto, Toronto, Ontario, Canada(多伦多大学,多伦多,安大略省,加拿大);Vector Institute, Toronto, Ontario, Canada(向量研究所,多伦多,安大略省,加拿大);Technion -- Israel Institute of Technology, Haifa, Israel(技术ion -- 以色列理工学院,海法,以色列)
Junchen Liu, Sven Elflein, Or Litany, Zan Gojcic, Ruilong Li
Test-time training (TTT) with KV binding as sequence modeling layer is commonly interpreted as a form of online meta-learning that memorizes a key-value mapping at test time. However, our analysis reveals multiple phenomena that contradict this memorization-based interpretation. Motivated by these findings, we revisit the formulation of TTT and show that a broad class of TTT architectures can be expressed as a form of learned linear attention operator. Beyond explaining previously puzzling model behaviors, this perspective yields multiple practical benefits: it enables principled architectural simplifications, admits fully parallel formulations that preserve performance while improving efficiency, and provides a systematic reduction of diverse TTT variants to a standard linear attention form. Overall, our results reframe TTT not as test-time memorization, but as learned linear attention with enhanced representational capacity. Project page: https://research.nvidia.com/labs/sil/projects/tttla/.
Boosting Convolutional Neural Networks' Protein Binding Site Prediction Capacity Using SE(3)-invariant transformers, Transfer Learning and Homology-based Augmentation
GDEGAN: Gaussian Dynamic Equivariant Graph Attention Network for Ligand Binding Site Prediction
Department of Artificial Intelligence, Indian Institute of Technology, Kharagpur, India(人工智能系,印度理工学院,Khargpur,印度);Department of Computer Science, Indian Institute of Technology, Kharagpur, India(计算机科学系,印度理工学院,Khargpur,印度)
Divide & Bind Your Attention for Improved Generative Semantic Nursing
Bosch Center for Artificial Intelligence(博世人工智能中心);University of Mannheim(曼海姆大学);Max Planck Institute for Informatics(马克斯·普朗克信息学研究所);University of Tübingen(蒂宾根大学)
CAME-AB: Cross-Modality Attention with Mixture-of-Experts for Antibody Binding Site Prediction
Generative AI Research and Development Center The Hong Kong University of Science and Technology(生成式人工智能研究与发展中心 香港科学大学);MILES The University of Hong Kong(MILES 香港大学);College of Veterinary Medicine Jilin University(吉林大学兽医学院);School of Chemistry and Chemical Engineering South China University of Technology(华南理工大学化学与化工学院);School of Biomedical Sciences The University of Hong Kong(香港大学生物医学科学学院);Computational Immunology Centre BayVax Biotech Limited(计算免疫学中心 BayVax 生物技术有限公司);College of Electronic and Information Engineering Southwest University(西南大学电子与信息工程学院)