Humans are continuously exposed to a stream of visual data with a natural temporal structure. However, most successful computer vision algorithms work at image level, completely discarding the precious information carried by motion. In this paper, we claim that processing visual streams naturally leads to formulate the motion invariance principle, which enables the construction of a new theory of learning that originates from variational principles, just like in physics. Such principled approach is well suited for a discussion on a number of interesting questions that arise in vision, and it offers a well-posed computational scheme for the discovery of convolutional filters over the retina. Differently from traditional convolutional networks, which need massive supervision, the proposed theory offers a truly new scenario for the unsupervised processing of video signals, where features are extracted in a multi-layer architecture with motion invariance. While the theory enables the implementation of novel computer vision systems, it also sheds light on the role of information-based principles to drive possible biological solutions.
Diverse feature visualizations reveal invariances in early layers of deep neural networks
Centre for Integrative Neuroscience（综合神经科学中心）；Institute for Theoretical Physics（理论物理研究所）；Bernstein Center for Computational Neuroscience（伯恩斯坦计算神经科学中心）；University of Tübingen（图宾根大学）
Spatial Temporal Synergy: Balancing Change and Invariance in Text Driven 3D Human Motion Editing
East China Normal University（华东师范大学）；School of Computer Science and Technology, East China Normal University（华东师范大学计算机科学与技术学院）；School of Computer Science and Engineering, Shanghai Jiao Tong University（上海交通大学计算机科学与工程学院）；Shanghai Institute of Artificial Intelligence for Education, East China Normal University（华东师范大学上海教育人工智能研究院）；School of Statistics, East China Normal University（华东师范大学统计学院）；School of Artificial Intelligence, Beihang University（北京航空航天大学人工智能学院）
SIVA-RL: Sensitivity-Invariance Visual Alignment for Multimodal Reinforcement Learning
Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Shanghai Jiao Tong University（上海交通大学）；Sichuan University（四川大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；University of Macau（澳门大学）
Omni-Supervised Motion Editing: Balancing Change and Invariance through Positive-Negative Learning
Shanghai Institute of Artificial Intelligence for Education, East China Normal University, China（上海人工智能教育研究院，华东师范大学，中国）；School of Computer Science and Technology, East China Normal University, China（华东师范大学计算机科学与技术学院，中国）；School of Statistics, East China Normal University, China（华东师范大学统计学院，中国）；The 27th Research Institute of CETC, Zhengzhou, China（中国电子科技集团第27研究所，郑州，中国）；Key Laboratory of Advanced Theory and Application in Statistics and Data Science, MOE, China（教育部统计与数据科学先进理论与应用重点实验室，中国）；Shanghai Key Laboratory of Computer Software Evaluating and Testing, China（上海计算机软件评测测试重点实验室，中国）；School of Computer Science, Shanghai Jiao Tong University, China（上海交通大学计算机科学学院，中国）
Illumination-insensitive Binary Descriptor for Visual Measurement Based on Local Inter-patch Invariance
University of Electronic Science and Technology of China (UESTC)（电子科技大学）；Sichuan University（四川大学）；institut superieur d’electronique de Paris - ISEP（巴黎高等电子学院（ISEP））
Paying More Attention to Visual Tokens in Self-Evolving Large Multimodal Models
Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Aalto University（阿尔托大学）；Australian National University（澳大利亚国立大学）；Linköping University（林雪平大学）
Visual-Invariance-Augmented Feature Optimal Alignment for Transferable Adversarial Attacks against Closed-Source MLLMs
Nanyang Technological University（南洋理工大学）；Northeastern University（东北大学）；Chongqing University（重庆大学）；University of Chinese Academy of Sciences（中国科学院大学）；Sun Yat-sen University（中山大学）
Stretching Beyond the Obvious: A Gradient-Free Framework to Unveil the Hidden Landscape of Visual Invariance
Neuroscience Area, International School for Advanced Studies (SISSA), Trieste (Italy)（国际先进研究学院（SISSA）神经科学部门，特里埃斯蒂（意大利））；Boston Children’s Hospital, Harvard Medical School, Boston (USA)（哈佛医学院波士顿儿童医院，波士顿（美国））；Center for Brains, Minds, and Machines, MIT, Cambridge (USA)（麻省理工学院大脑、心智与机器中心，剑桥（美国））；Harvard-MIT Program in Health Sciences and Technology, MIT, Cambridge (USA)（哈佛-麻省理工学院健康科学与技术项目，麻省理工学院，剑桥（美国））
Exploiting Transformation Invariance and Equivariance for Self-supervised Sound Localisation
Shanghai Jiao Tong University（上海交通大学）；Shanghai AI Laboratory（上海人工智能实验室）
我们提出了一个简单而有效的自监督框架视听表示学习，本地化的声源在视频中。为了理解是什么使得能够学习有用的表示，我们系统地研究了数据增强的效果，并揭示了（1）数据增强的组合起着关键作用，即明确地鼓励视听表示对各种变换保持不变~（{ \em变换不变性}）;（2）强制几何一致性实质上提高了学习表示的质量，即，检测到的声源应当遵循应用于输入视频帧的相同变换~（{ \em变换等方差}）。大量的实验表明，我们的模型显着优于以前的方法在两个声音定位基准，即，Flickr-SoundNet和VGG-Sound。此外，我们还评估了音频检索和跨模态检索任务。在这两种情况下，我们的自监督模型表现出优越的检索性能，甚至与音频检索中的监督方法竞争。这表明，所提出的框架学习强大的多模态表示，有利于声音的本地化和推广到进一步的应用。\textit{All codes will be available}。
