Human motion prediction is a cornerstone of human-robot collaboration (HRC), as robots need to infer the future movements of human workers based on past motion cues to proactively plan their motion, ensuring safety in close collaboration scenarios. The diffusion model has demonstrated remarkable performance in predicting high-quality motion samples with reasonable diversity, but suffers from a slow generative process which necessitates multiple model evaluations, hindering real-world applications. To enable real-time prediction, in this work, we propose training a one-step multi-layer perceptron-based (MLP-based) diffusion model for motion prediction using knowledge distillation and Bayesian optimization. Our method contains two steps. First, we distill a pretrained diffusion-based motion predictor, TransFusion, directly into a one-step diffusion model with the same denoiser architecture. Then, to further reduce the inference time, we remove the computationally expensive components from the original denoiser and use knowledge distillation once again to distill the obtained one-step diffusion model into an even smaller model based solely on MLPs. Bayesian optimization is used to tune the hyperparameters for training the smaller diffusion model. Extensive experimental studies are conducted on benchmark datasets, and our model can significantly improve the inference speed, achieving real-time prediction without noticeable degradation in performance.
BACON: Bayesian Optimal Condensation Framework for Dataset Distillation
School of Electronic and Information Engineering, Beihang University(北京航空航天大学电子信息工程学院);University of Liverpool(利物浦大学);S-Lab, Nanyang Technological University(南洋理工大学S-Lab)
Synthetic Data Generation for Emotional Depth Faces: Optimizing Conditional DCGANs via Genetic Algorithms in the Latent Space and Stabilizing Training with Knowledge Distillation
Cyrus Intelligence Research Ltd(居鲁士智能研究有限公司);Pars AI
I$^2$MD: 3D Action Representation Learning with Inter- and Intra-modal Mutual Distillation
University of Science and Technology of China(中国科学技术大学);The University of Adelaide(阿德莱德大学);Hefei Comprehensive National Science Center(合肥综合性国家科学中心);Shanghai Artificial Intelligent Laboratory(上海人工智能实验室)
Iterative Distillation for Better Uncertainty Estimates in Multitask Emotion Recognition
Hong Kong University of Science and Technology(香港科技大学)
当识别情绪时,情绪表现中的细微差别会产生情绪感知的模糊性或不确定性。情感不确定性以前被解释为多个注释者之间的评分者之间的分歧。在本文中,我们考虑了一个更常见和更具挑战性的场景:当只有单个情感标签可用时,对情感不确定性进行建模。从贝叶斯的角度来看,我们建议使用深度集成来捕获多个情感描述符的不确定性,即,动作单元、离散表达式标签和连续描述符。我们进一步应用迭代自蒸馏。多代迭代蒸馏显著提高了情感识别和不确定性估计的性能。我们的方法生成单个学生模型,该模型为域内样本提供准确的不确定性估计,并且生成可以检测域外样本的学生集合。我们使用Aff-wild 2数据集进行的情感识别和不确定性估计的实验表明,我们的算法比Temperature Scaling和Monte Carol Dropout给出了更可靠的不确定性估计。
The University of Tokyo Hospital(东京大学医院);Graduate School of Medicine, The University of Tokyo(东京大学医学院);Junpukai Health Maintenance Center(顺会健康体检中心);Hokkaido University(北海道大学)
Biomechanical 3D Body: Self-Supervised Distillation of Biomechanical Pose from a 3D Body Foundation Model
Shirley Ryan AbilityLab(雪莉·瑞安能力实验室);Northwestern University(西北大学);University of Illinois Chicago(伊利诺伊大学芝加哥分校);University of Texas at Austin(德克萨斯大学奥斯汀分校)
Enhanced Self-Distillation Framework for Efficient Spiking Neural Network Training
ZJU-UIUC Institute, Zhejiang University(浙江大学之江-伊利诺伊大学联合学院);College of Information Science and Electronic Engineering, Zhejiang University(浙江大学信息科学与电子工程学院)
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training
Technical University of Munich(慕尼黑工业大学);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心);Helmholtz Munich(亥姆霍兹慕尼黑中心);Munich Data Science Institute (MDSI)(慕尼黑数据科学研究所)
Breaking Class Barriers: Efficient Dataset Distillation via Inter-Class Feature Compensator
Centre for Frontier AI Research, Agency for Science, Technology and Research(新加坡科技研究局前沿人工智能研究中心);Institute of High Performance Computing, Agency for Science, Technology and Research(新加坡科技研究局高性能计算研究所);University of Nevada, Reno(内华达大学雷诺分校)
BayTTA: Uncertainty-aware medical image classification with optimized test-time augmentation using Bayesian model averaging
Université du Québec à Montréal(魁北克大学蒙特利尔分校);Institute for Intelligent Systems Research and Innovation (IISRI), Deakin University(迪肯大学智能系统研究与创新研究所);Cracow University of Technology(克拉科夫工业大学);Mila - Quebec AI Institute(米拉-魁北克人工智能研究所)