Position: A Call to Action for a Human-Centered AutoML Paradigm
L3S Research Center（L3S研究中心）；Ludwig-Maximilians-Universität München（慕尼黑大学）；Fraunhofer Institute for Integrated Circuits IIS, Fraunhofer IIS（弗劳恩霍夫集成电路研究所）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Microsoft（微软公司）；Albert-Ludwigs-Universität Freiburg（阿尔伯特-路德维希-弗莱堡大学）
Marius Lindauer, Florian Karl, Anne Klier, Julia Moosbauer, Alexander Tornede, Andreas Mueller, Frank Hutter, Matthias Feurer, Bernd Bischl
Automated machine learning (AutoML) was formed around the fundamental objectives of automatically and efficiently configuring machine learning (ML) workflows, aiding the research of new ML algorithms, and contributing to the democratization of ML by making it accessible to a broader audience. Over the past decade, commendable achievements in AutoML have primarily focused on optimizing predictive performance. This focused progress, while substantial, raises questions about how well AutoML has met its broader, original goals. In this position paper, we argue that a key to unlocking AutoML's full potential lies in addressing the currently underexplored aspect of user interaction with AutoML systems, including their diverse roles, expectations, and expertise. We envision a more human-centered approach in future AutoML research, promoting the collaborative design of ML systems that tightly integrates the complementary strengths of human expertise and AutoML methodologies.
UniAutoML: A Human-Centered Framework for Unified Discriminative and Generative AutoML with Large Language Models
Glasgow College, University of Electronic Science and Technology of China（电子科技大学格拉斯哥学院）；School of Engineering, University of Glasgow（格拉斯哥大学工程学院）；Toursun Synbio（同舜合成生物）；Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences（中国科学院大学电子电气与通信工程学院）；School of Information and Software Engineering, University of Electronic Science and Technology Of China（电子科技大学信息与软件工程学院）；Research Institute, Ubtech Robotics Corp（优必选机器人公司研究院）；Qiyuan Lab（启源实验室）；Department of Computer Science, Johns Hopkins University（约翰斯·霍普金斯大学计算机科学系）
University of Toronto（多伦多大学）；University of Wisconsin（威斯康星大学）；IIT Bombay（印度理工学院孟买分校）；Carnegie Mellon University（卡内基梅隆大学）；Stanford University（斯坦福大学）
The challenge that climate change poses to humanity has spurred a rapidly developing field of artificial intelligence research focused on climate change applications. The climate change AI (CCAI) community works on a diverse, challenging set of problems which often involve physics-constrained ML or heterogeneous spatiotemporal data. It would be desirable to use automated machine learning (AutoML) techniques to automatically find high-performing architectures and hyperparameters for a given dataset. In this work, we benchmark popular AutoML libraries on three high-leverage CCAI applications: cl…
College of Transportation Engineering, Tongji University（同济大学交通运输工程学院）；University of California, Berkeley（加利福尼亚大学伯克利分校）；Institute for Infocomm Research, Agency for Science Technology and Research (A*STAR)（新加坡科技研究局信息通信研究所）
With the enrichment of smartphones, driving distractions caused by phone usages have become a threat to driving safety. A promising way to mitigate driving distractions is to detect them and give real-time safety warnings. However, existing detection algorithms face two major challenges, low user acceptance caused by in-vehicle camera sensors, and uncertain accuracy of pre-trained models due to drivers individual differences. Therefore, this study proposes a domain-specific automated machine learning (AutoML) to self-learn the optimal models to detect distraction based on lane-keeping performa…
This work improves the quality of automated machine learning (AutoML) systems by using dataset and function descriptions while significantly decreasing computation time from minutes to milliseconds by using a zero-shot approach. Given a new dataset and a well-defined machine learning task, humans begin by reading a description of the dataset and documentation for the algorithms to be used. This work is the first to use these textual descriptions, which we call privileged information, for AutoML. We use a pre-trained Transformer model to process the privileged text and demonstrate that using th…
As big data becomes ubiquitous across domains, and more and more stakeholders aspire to make the most of their data, demand for machine learning tools has spurred researchers to explore the possibilities of automated machine learning (AutoML). AutoML tools aim to make machine learning accessible for non-machine learning experts (domain experts), to improve the efficiency of machine learning, and to accelerate machine learning research. But although automation and efficiency are among AutoML's main selling points, the process still requires human involvement at a number of vital steps, includin…
IBM Research（IBM研究院）；University of Michigan（密歇根大学）
Human-Centered AI (HCAI) refers to the research effort that aims to design and implement AI techniques to support various human tasks, while taking human needs into consideration and preserving human control. In this short position paper, we illustrate how we approach HCAI using a series of research projects around Data Science (DS) works as a case study. The AI techniques built for supporting DS works are collectively referred to as AutoML systems, and their goals are to automate some parts of the DS workflow. We illustrate a three-step systematical research approach(i.e., explore, build, and…
Data science (DS) projects often follow a lifecycle that consists of laborious tasks for data scientists and domain experts (e.g., data exploration, model training, etc.). Only till recently, machine learning(ML) researchers have developed promising automation techniques to aid data workers in these tasks. This paper introduces AutoDS, an automated machine learning (AutoML) system that aims to leverage the latest ML automation techniques to support data science projects. Data workers only need to upload their dataset, then the system can automatically suggest ML configurations, preprocess data…
Institute for Biomedical Informatics, University of Pennsylvania（宾夕法尼亚大学生物医学信息学研究所）
As data science continues to grow in popularity, there will be an increasing need to make data science tools more scalable, flexible, and accessible. In particular, automated machine learning (AutoML) systems seek to automate the process of designing and optimizing machine learning pipelines. In this chapter, we present a genetic programming-based AutoML system called TPOT that optimizes a series of feature preprocessors and machine learning models with the goal of maximizing classification accuracy on a supervised classification problem. Further, we analyze a large database of pipelines that …
Columbia University（哥伦比亚大学）；University of Chicago（芝加哥大学）
We introduce an automatic machine learning (AutoML) modeling architecture called Autostacker, which combines an innovative hierarchical stacking architecture and an Evolutionary Algorithm (EA) to perform efficient parameter search. Neither prior domain knowledge about the data nor feature preprocessing is needed. Using EA, Autostacker quickly evolves candidate pipelines with high predictive accuracy. These pipelines can be used as is or as a starting point for human experts to build on. Autostacker finds innovative combinations and structures of machine learning models, rather than selecting a…
University of Cambridge（剑桥大学）；UCLA（加州大学洛杉矶分校）；The Alan Turing Institute（艾伦·图灵研究所）
Successful application of machine learning models to real-world prediction problems, e.g. financial forecasting and personalized medicine, has proved to be challenging, because such settings require limiting and quantifying the uncertainty in the model predictions, i.e. providing valid and accurate prediction intervals. Conformal Prediction is a distribution-free approach to construct valid prediction intervals in finite samples. However, the prediction intervals constructed by Conformal Prediction are often (because of over-fitting, inappropriate measures of nonconformity, or other issues) ov…
IBM Research（IBM研究院）；University of Massachusetts Boston（马萨诸塞大学波士顿分校）
We present AutoDOViz, an interactive user interface for automated decision optimization (AutoDO) using reinforcement learning (RL). Decision optimization (DO) has classically being practiced by dedicated DO researchers where experts need to spend long periods of time fine tuning a solution through trial-and-error. AutoML pipeline search has sought to make it easier for a data scientist to find the best machine learning pipeline by leveraging automation to search and tune the solution. More recently, these advances have been applied to the domain of AutoDO, with a similar goal to find the best …
Counterfactual Explanations for Time Series Should be Human-Centered and Temporally Coherent in Interventions
Data Mining Group, Eindhoven University of Technology, the Netherlands（埃因霍温技术大学数据挖掘组，荷兰）；Biomedical Signals and Systems Group, University of Twente, the Netherlands（代尔夫特理工大学生物医学信号与系统组，荷兰）
HCRide: Harmonizing Passenger Fairness and Driver Preference for Human-Centered Ride-Hailing
Department of Computer Science, Florida State University（佛罗里达州立大学计算机科学系）；Department of Computer Science and Engineering, Lehigh University（莱斯利大学计算机科学与工程系）
Universidade Federal de Minas Gerais（米纳斯吉拉斯联邦大学）；Stanford University（斯坦福大学）；SLAC National Accelerator Laboratory（SLAC国家加速器实验室）
Serial crystallography is the field of science that studies the structure and properties of crystals via diffraction patterns. In this paper, we introduce a new serial crystallography dataset comprised of real and synthetic images; the synthetic images are generated through the use of a simulator that is both scalable and accurate. The resulting dataset is called DiffraNet, and it is composed of 25,457 512x512 grayscale labeled images. We explore several computer vision approaches for classification on DiffraNet such as standard feature extraction algorithms associated with Random Forests and …
Institute for Advanced Studies in Basic Sciences (IASBS)（基础科学高级研究学院）；Charles University（查理大学）；Institute for Research in Fundamental Sciences (IPM)（基础科学研究所）
Nowadays, automobile manufacturers make efforts to develop ways to make cars fully safe. Monitoring driver's actions by computer vision techniques to detect driving mistakes in real-time and then planning for autonomous driving to avoid vehicle collisions is one of the most important issues that has been investigated in the machine vision and Intelligent Transportation Systems (ITS). The main goal of this study is to prevent accidents caused by fatigue, drowsiness, and driver distraction. To avoid these incidents, this paper proposes an integrated safety system that continuously monitors the d…
