Rethinking Default Values: a Low Cost and Efficient Strategy to Define Hyperparameters
Federal Technology University - Paraná(巴拉那联邦技术大学);São Paulo State University (UNESP)(圣保罗州立大学);University of São Paulo (USP)(圣保罗大学);Federal University of Ouro Preto (UFOP)(欧鲁普雷图联邦大学);State University of Londrina (UEL)(隆德里纳州立大学)
Rafael Gomes Mantovani, André Luis Debiaso Rossi, Edesio Alcobaça, Jadson Castro Gertrudes, Sylvio Barbon Junior, André Carlos Ponce de Leon Ferreira de Carvalho
Machine Learning (ML) algorithms have been increasingly applied to problems from several different areas. Despite their growing popularity, their predictive performance is usually affected by the values assigned to their hyperparameters (HPs). As consequence, researchers and practitioners face the challenge of how to set these values. Many users have limited knowledge about ML algorithms and the effect of their HP values and, therefore, do not take advantage of suitable settings. They usually define the HP values by trial and error, which is very subjective, not guaranteed to find good values and dependent on the user experience. Tuning techniques search for HP values able to maximize the predictive performance of induced models for a given dataset, but have the drawback of a high computational cost. Thus, practitioners use default values suggested by the algorithm developer or by tools implementing the algorithm. Although default values usually result in models with acceptable predictive performance, different implementations of the same algorithm can suggest distinct default values. To maintain a balance between tuning and using default values, we propose a strategy to generate new optimized default values. Our approach is grounded on a small set of optimized values able to obtain predictive performance values better than default settings provided by popular tools. After performing a large experiment and a careful analysis of the results, we concluded that our approach delivers better default values. Besides, it leads to competitive solutions when compared to tuned values, making it easier to use and having a lower cost. We also extracted simple rules to guide practitioners in deciding whether to use our new methodology or a HP tuning approach.
Default Machine Learning Hyperparameters Do Not Provide Informative Initialization for Bayesian Optimization
Department of Telematics and Computing, Universidad Pontificia Comillas(电信与计算系,Pontificia Comillas大学);Institute for Research in Technology (IIT), Universidad Pontificia Comillas(技术研究 institute,Pontificia Comillas大学)
A meta-learning recommender system for hyperparameter tuning: predicting when tuning improves SVM classifiers
Federal Technology University(联邦技术大学);University of São Paulo(圣保罗大学);Universidade Estadual Paulista(保利斯塔州立大学);Eindhoven University of Technology(埃因霍温理工大学)
When Is an LLM Worth It for Hyperparameter Optimization? A Budget-Matched Study on Tabular Data Finds the Warm-Start Is a Default Configuration, Not the Model
Tuning Word2vec for Large Scale Recommendation Systems
Twitter(推特);Imperial College London(伦敦帝国理工学院)
Word2vec是一种功能强大的机器学习工具,它起源于自然语言处理(NLP),目前已应用于推荐系统、预测和网络分析等多个领域。由于Word2vec经常使用现成的,我们解决的问题,是否默认的超参数是适合的推荐系统。在本文中,我们首先阐明了超参数优化的重要性,并表明,无约束优化产生了平均221%的提高命中率比thefdefault参数。然而,无约束优化会导致超参数设置非常昂贵,并且对于大规模优化任务不可行。为此,我们证明了138%的平均改善命中率与运行时间约束的超参数优化。此外,为了使超参数优化适用于大规模推荐问题,其中目标数据集太大,无法搜索,我们研究了从样本中泛化超参数设置。我们表明,应用约束超参数优化,仅使用10%的数据样本仍然产生了91%的平均改善命中率超过默认参数时,应用到thefuldatasets。最后,我们将使用我们的样本约束优化方法学习的超参数应用于Twitter上的Who To Follow推荐服务,并能够将关注率提高15%。
Application of an automated machine learning-genetic algorithm (AutoML-GA) coupled with computational fluid dynamics simulations for rapid engine design optimization
Argonne National Laboratory(阿贡国家实验室);Parallel Works Inc.(Parallel Works 公司);Convergent Science, Inc.(Convergent Science 公司)
Unmute the Patch Tokens: Rethinking Probing in Multi-Label Audio Classification
University of Kassel(卡塞尔大学);Fraunhofer IEE(弗劳恩霍夫研究所);Ghent University(根特大学);ML and Systems Biology, MPI of Biochemistry(生物化学Max Planck研究所);INRIA Montpellier(蒙彼利埃INRIA)
Robustness of Algorithms for Causal Structure Learning to Hyperparameter Choice
University of Essex(埃塞克斯大学);Institute for Analytics and Data Science, University of Essex(埃塞克斯大学分析与数据科学研究所);Institute for Social and Economic Research, University of Essex(埃塞克斯大学社会与经济研究所)
Rethinking the Teacher-Student Framework for Test-Time Adaptation
Poznan University of Technology(波兹南理工大学);Graz University of Technology(格拉茨理工大学);IDEAS Research Institute(IDEAS研究院);Computer Vision Center, Universitat Autonoma de Barcelona(巴塞罗那自治大学计算机视觉中心);NASK - National Research Institute(NASK国家研究院);Gdańsk University of Technology(格但斯克理工大学)
Better Trees: An empirical study on hyperparameter tuning of classification decision tree induction algorithms
Federal Technology University, Paraná(巴拉那联邦技术大学);Pavol Jozef Šafárik University(帕沃尔·约瑟夫·沙法里克大学);ELTE Eötvös Loránd University(厄特沃什·罗兰大学);São Paulo State University (Unesp)(圣保罗州立大学);Federal University of São Carlos(圣卡洛斯联邦大学);University of Trieste (UniTS)(的里雅斯特大学);Eindhoven University of Technology (TU/e)(埃因霍温理工大学);Institute of Mathematics and Computer Sciences (ICMC), University of São Paulo (USP)(圣保罗大学数学与计算机科学研究所)
Rethinking Distribution Shifts: Empirical Analysis and Modeling for Tabular Data
Department of Industrial Engineering and Operations Research(工业工程与运筹学系);Department of Computer Science and Technology(计算机科学与技术系);Decision, Risk, and Operations Division(决策、风险与运营部);Columbia University(哥伦比亚大学);Tsinghua University(清华大学)