BTTackler: A Diagnosis-based Framework for Efficient Deep Learning Hyperparameter Optimization
School of Software, BNRist, \ University Beijing China；School of Software, EIRI, \ University Beijing China；School of Software, \ University Beijing China；School of Software, BNRist, \ University；School of Software, EIRI, \ University；School of Software, \ University
Zhongyi Pei, Zhiyao Cen, Yipeng Huang, Chen Wang, Lin Liu, Philip Yu, Mingsheng Long
Hyperparameter optimization (HPO) is known to be costly in deep learning, especially when leveraging automated approaches. Most of the existing automated HPO methods are accuracy-based, i.e., accuracy metrics are used to guide the trials of different hyperparameter configurations amongst a specific search space. However, many trials may encounter severe training problems, such as vanishing gradients and insufficient convergence, which can hardly be reflected by accuracy metrics in the early stages of the training and often result in poor performance. This leads to an inefficient optimization trajectory because the bad trials occupy considerable computation resources and reduce the probability of finding excellent hyperparameter configurations within a time limitation. In this paper, we propose \textbf{Bad Trial Tackler (BTTackler)}, a novel HPO framework that introduces training diagnosis to identify training problems automatically and hence tackles bad trials. BTTackler diagnoses each trial by calculating a set of carefully designed quantified indicators and triggers early termination if any training problems are detected. Evaluations are performed on representative HPO tasks consisting of three classical deep neural networks (DNN) and four widely used HPO methods. To better quantify the effectiveness of an automated HPO method, we propose two new measurements based on accuracy and time consumption. Results show the advantage of BTTackler on two-fold: (1) it reduces 40.33\% of time consumption to achieve the same accuracy comparable to baseline methods on average and (2) it conducts 44.5\% more top-10 trials than baseline methods on average within a given time budget. We also released an open-source Python library that allows users to easily apply BTTackler to automated HPO processes with minimal code changes.
Alleviating Hyperparameter-Tuning Burden in SVM Classifiers for Pulmonary Nodules Diagnosis with Multi-Task Bayesian Optimization
Yau Mathematical Sciences Center Tsinghua University（姚数学科学中心清华大学）；The First Affiliated Hospital of Guangzhou Medical University（广州医科大学第一附属医院）；China State Key Laboratory of Respiratory Disease（中国呼吸疾病国家重点实验室）；Academy of Mathematics and Systems Science Chinese Academy of Sciences（数学科学学院中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）；Department of Thoracic Surgery and Oncology The First Affiliated Hospital of Guangzhou Medical University（胸外科与肿瘤科广州医科大学第一附属医院）
Time-Efficient Hybrid Hyperparameter Tuning Approach for Cardiovascular Disease Classification
Department of Computer Science, Institute of Science, Banaras Hindu University（计算机科学系，科学学院，班纳拉森胡大学）；School of Computer Science, University of Petroleum and Energy Studies（计算机科学学院，石油与能源研究大学）
National Space Science Center.CAS（中国科学院国家空间科学中心）；University of Chinese Academy of Sciences（中国科学院大学）；Science and Technology on Complex Aviation System Simulation Laboratory（复杂航空系统仿真科学与技术实验室）
数据驱动方法在故障诊断中取得了很大的进展，尤其是深度学习方法。深度学习适用于处理大数据，具有很强的特征提取能力，可以实现端到端的故障诊断系统。但设计神经网络架构需要丰富的专业知识和调试经验，还需要大量的实验来筛选模型和超参数，增加了深度学习模型的开发难度。近年来，神经结构搜索（Neural Architecture Search，NAS）发展迅速，成为深度学习的下一个发展方向。提出了一种基于强化学习的NAS故障诊断方法。一个递归神经网络被用来作为代理生成网络结构。生成的网络在验证数据集上的准确性作为奖励反馈给代理，并通过策略梯度算法更新代理的参数。利用PHM 2009 Data Challenge齿轮箱数据集验证了该方法的有效性，并与其他人工设计的网络结构进行了比较，取得了较好的效果。据作者所知，这是第一次将NAS应用于故障诊断。
Enhancing Diagnostic Accuracy for Urinary Tract Disease through Explainable SHAP-Guided Feature Selection and Classification
Labcin - Nature Inspired Computing Laboratory（Labcin - 自然启发计算实验室）；Federal University of Espírito Santo（巴西圣埃斯皮里图联邦大学）；Graduate Program in Computer Science（计算机科学研究生项目）
An Evaluation of Machine Learning Approaches for Early Diagnosis of Autism Spectrum Disorder
Islamic University（伊斯兰大学）；Khulna Medical College & Hospital（库尔纳医学院及医院）；City University（城市大学）；University of Missouri（密苏里大学）；Rensselaer Polytechnic Institute（伦斯勒理工学院）
Cross-dataset Multivariate Time-series Model for Parkinson's Diagnosis via Keyboard Dynamics
Unit of Artificial Intelligence and Computer Systems, Department of Engineering, Università Campus Bio-Medico di Roma, Rome, Italy（人工智能与计算机系统单位，工程系，罗马生物医学大学）；Eustema S.p.A., Research and Development Centre, Naples, Italy（Eustema公司，研发中心，意大利那不勒斯）；Department of Radiation Sciences, Biomedical Engineering, Umeå University, Umeå, Sweden（辐射科学系，生物医学工程，乌梅大学）
Detach-ROCKET: Sequential feature selection for time series classification with random convolutional kernels
KTH Royal Institute of Technology（瑞典皇家理工学院）；Science for Life Laboratory（生命科学实验室）；Digital Futures（数字未来机构）；University of Trieste（的里雅斯特大学）；AREA Science Park（AREA科学园）；Research and Technology Institute（研究与技术研究所）
Learning ECG Representations via Poly-Window Contrastive Learning
1 Department of Quantitative Theory \& Methods；2 Department of Mathematics；3 Center for Data Science, Nell Hodgson Woodruff School of Nursing Emory University, Atlanta, GA, USA；4 Department of Computer Science, Georgia State University, Atlanta, GA, USA；5 Microsoft
