PA&DA: Jointly Sampling PAth and DAta for Consistent NAS
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；School of Computer Science and Technology, University of Chinese Academy of Sciences（中国科学院大学计算机科学与技术学院）；Kuaishou Technology（快手科技）
Based on the weight-sharing mechanism, one-shot NAS methods train a supernet and then inherit the pre-trained weights to evaluate sub-models, largely reducing the search cost. However, several works have pointed out that the shared weights suffer from different gradient descent directions during training. And we further find that large gradient variance occurs during supernet training, which degrades the supernet ranking consistency. To mitigate this issue, we propose to explicitly minimize the gradient variance of the supernet training by jointly optimizing the sampling distributions of PAth and DAta (PA&DA). We theoretically derive the relationship between the gradient variance and the sampling distributions, and reveal that the optimal sampling probability is proportional to the normalized gradient norm of path and training data. Hence, we use the normalized gradient norm as the importance indicator for path and training data, and adopt an importance sampling strategy for the supernet training. Our method only requires negligible computation cost for optimizing the sampling distributions of path and data, but achieves lower gradient variance during supernet training and better generalization performance for the supernet, resulting in a more consistent NAS. We conduct comprehensive comparisons with other improved approaches in various search spaces. Results show that our method surpasses others with more reliable ranking performance and higher accuracy of searched architectures, showing the effectiveness of our method. Code is available at https://github.com/ShunLu91/PA-DA.
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；University of Pittsburgh（匹兹堡大学）；Kuaishou Technology（快手科技）
采样、分组和聚合是点云多尺度分析中的三个重要组成部分。在本文中，我们提出了一种新的数据驱动的采样器学习策略逐点分析任务。与广泛使用的采样技术不同，Faraday Point Sampling（FPS），我们建议联合学习采样和下游应用程序。我们的关键见解是，像FPS这样的均匀采样方法并不总是最适合不同的任务：在边界区域周围采样更多的点可以使逐点分类更容易分割。为此，我们提出了一种新的采样器学习策略，该策略学习由任务相关的地面实况信息监督的采样点位移，并且可以与底层任务联合训练。我们进一步展示了我们的方法在各种逐点分析任务，包括语义部分分割，点云完成，和关键点检测。我们的实验表明，在各种基于点的网络中，采样器和任务的联合学习比使用FPS带来更好的性能。
BioAutoML-NAS: An End-to-End AutoML Framework for Multimodal Insect Classification via Neural Architecture Search on Large-Scale Biodiversity Data
Department of Computer Science and Engineering, United International University, Dhaka, Bangladesh（乌姆国际大学计算机科学与工程系，达卡，孟加拉国）；Applied Artificial Intelligence and Intelligent Systems (AAIINS) Laboratory, 1217, Dhaka, Bangladesh（应用人工智能与智能系统实验室（AAIINS），1217号，达卡，孟加拉国）；Department of Computer Science and Engineering, Penn State University, University Park, PA, USA（宾夕法尼亚州立大学计算机科学与工程系，University Park，PA，美国）；Faculty of Science and Information Technology, Charles Darwin University, Sydney, NSW, Australia（查尔斯达尔文大学科学与信息技术学院，悉尼，新南威尔士州，澳大利亚）；Faculty of Science and Technology, Charles Darwin University, Casuarina, 0909, NT, Australia（查尔斯达尔文大学科学与技术学院，Casuarina，0909，北领地，澳大利亚）
