Neural networks efficiently encode learned information within their parameters. Consequently, many tasks can be unified by treating neural networks themselves as input data. When doing so, recent studies demonstrated the importance of accounting for the symmetries and geometry of parameter spaces. However, those works developed architectures tailored to specific networks such as MLPs and CNNs without normalization layers, and generalizing such architectures to other types of networks can be challenging. In this work, we overcome these challenges by building new metanetworks - neural networks that take weights from other neural networks as input. Put simply, we carefully build graphs representing the input neural networks and process the graphs using graph neural networks. Our approach, Graph Metanetworks (GMNs), generalizes to neural architectures where competing methods struggle, such as multi-head attention layers, normalization layers, convolutional layers, ResNet blocks, and group-equivariant linear layers. We prove that GMNs are expressive and equivariant to parameter permutation symmetries that leave the input neural network functions unchanged. We validate the effectiveness of our method on several metanetwork tasks over diverse neural network architectures.
Metanetworks as Regulatory Operators: Learning to Edit for Requirement Compliance
National and Kapodistrian University of Athens（国家与卡波迪斯蒂亚那大学）；Archimedes/Athena RC（阿基米德/雅典RC）；Visible Machines, AI Research & Social Awareness Center（可见机器，人工智能研究与社会意识中心）
MPO: An Efficient Post-Processing Framework for Mixing Diverse Preference Alignment
Department of Statistics, Rutgers University, New Brunswick, United States；Department of Computer Science, Rutgers University, New Brunswick, United States；College of Management of Technology, EPFL, Switzerland；Department of Computer Science, ETH Zurich, Switzerland
OmniSapiens: A Foundation Model for Social Behavior Processing via Heterogeneity-Aware Relative Policy Optimization
Massachusetts Institute of Technology（麻省理工学院）；National University of Singapore（新加坡国立大学）；Nanyang Technological University（南洋理工大学）；Prince Sattam bin Abdulaziz University（普森·萨塔姆·本·阿卜杜勒阿齐兹大学）；University of Rochester（罗切斯特大学）
Uncovering the Limitations of Query Performance Prediction: Failures, Insights, and Implications for Selective Query Processing
Aix Marseille Université（艾克斯-马赛大学）；Université de Toulon（土伦大学）；CNRS（法国国家科学研究中心）；IMT（法国国立高等电信学院）；Univ. de Toulouse（图卢兹大学）；Université de Toulouse（图卢兹大学）；INSPE（国家教育与教学高级学院）；UT2J（图卢兹第二让·饶勒斯大学）；Université Toulouse Capitole（图卢兹第一大学）；UT1（图卢兹第一大学）；Edinburgh Napier University（爱丁堡龙比亚大学）
