Model Zoos: A Dataset of Diverse Populations of Neural Network Models
University of St.Gallen(圣加仑大学);Samsung - SAIT AI Lab(三星-三星先进技术研究院人工智能实验室);Institut de Robòtica i Informàtica Industrial(工业机器人与信息技术研究所);Universitat Politècnica de Catalunya(加泰罗尼亚理工大学)
Konstantin Schürholt, Diyar Taskiran, Boris Knyazev, Xavier Giró-i-Nieto, Damian Borth
In the last years, neural networks (NN) have evolved from laboratory environments to the state-of-the-art for many real-world problems. It was shown that NN models (i.e., their weights and biases) evolve on unique trajectories in weight space during training. Following, a population of such neural network models (referred to as model zoo) would form structures in weight space. We think that the geometry, curvature and smoothness of these structures contain information about the state of training and can reveal latent properties of individual models. With such model zoos, one could investigate novel approaches for (i) model analysis, (ii) discover unknown learning dynamics, (iii) learn rich representations of such populations, or (iv) exploit the model zoos for generative modelling of NN weights and biases. Unfortunately, the lack of standardized model zoos and available benchmarks significantly increases the friction for further research about populations of NNs. With this work, we publish a novel dataset of model zoos containing systematically generated and diverse populations of NN models for further research. In total the proposed model zoo dataset is based on eight image datasets, consists of 27 model zoos trained with varying hyperparameter combinations and includes 50'360 unique NN models as well as their sparsified twins, resulting in over 3'844'360 collected model states. Additionally, to the model zoo data we provide an in-depth analysis of the zoos and provide benchmarks for multiple downstream tasks. The dataset can be found at www.modelzoos.cc.
A Model Zoo on Phase Transitions in Neural Networks
Department of Computer Science University of St. Gallen, Switzerland(斯德哥尔摩大学计算机科学系);Department of Computer Science Dartmouth College, USA(达特茅斯学院计算机科学系);Department of Electrical and Computer Engineering National University of Singapore, Singapore(新加坡国立大学电子与计算机工程系)
Global Wheat Head Detection (GWHD) dataset: a large and diverse dataset of high resolution RGB labelled images to develop and benchmark wheat head detection methods
Arvalis, Institut du végétal(Arvalis植物研究所);INRAE(法国国家农业食品与环境研究院);Rothamsted Research(洛桑研究所);ETH Zurich(苏黎世联邦理工学院);CSIRO Agriculture and Food(联邦科学与工业研究组织农业与食品部);Nanjing Agricultural University(南京农业大学);National Agriculture and Food Research Organization(日本国家农业食品研究机构);University of Saskatchewan(萨斯喀彻温大学);The University of Queensland(昆士兰大学);The University of Tokyo(东京大学)
Hyper-Representations as Generative Models: Sampling Unseen Neural Network Weights
University of St.Gallen(圣加仑大学);Samsung - SAIT AI Lab(三星SAIT人工智能实验室);Institut de Robòtica i Informàtica Industrial(工业机器人与信息技术研究所);Universitat Politècnica de Catalunya(加泰罗尼亚理工大学)
Know2Vec: A Black-Box Proxy for Neural Network Retrieval
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院);College of Information and Intelligence Engineering, Zhejiang Wanli University(浙江万里学院信息与智能工程学院);Tsinghua University(清华大学)
EvoFlow: Evolving Diverse Agentic Workflows On The Fly
The Chinese University of Hong Kong(香港中文大学);Tongji University(同济大学);Wuhan University(武汉大学);Tsinghua University(清华大学);Nanyang Technological University(南洋理工大学);Shanghai AI Laboratory(上海人工智能实验室)