arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.22537cs.LG
收藏

LUMOS: 通过L0正则化学习统一特征和参数适应以普及SciML工作流

LUMOS: Democratizing SciML Workflows with L0-Regularized Learning for Unified Feature and Parameter Adaptation

Oregon State University(俄勒冈州立大学);Florida International University(佛罗里达国际大学);Argonne National Laboratory(阿贡国家实验室)

Shouwei Gao, Xu Zheng, Dongsheng Luo, Sheng Di, Wenqian Dong

AI总结:

LUMOS通过L0正则化学习统一特征和参数适应,实现SciML模型设计的普及化。

摘要:

科学机器学习(SciML)的快速发展加速了跨不同领域的发现,但设计有效的SciML模型仍然是一项具有挑战性的任务。在实践中,构建此类模型通常需要大量的先验知识和手动专业知识,特别是在确定应使用哪些输入特征以及模型应有多大时。我们介绍了LUMOS,一个基于L0正则化学习的端到端框架,该框架将特征选择和模型剪枝统一起来,以普及SciML模型设计。通过使用半随机门控和重参数化技术,LUMOS在训练过程中动态选择信息性特征并修剪冗余参数,从而减少对手动调优的依赖,同时保持预测准确性。我们在13种多样的SciML工作负载上评估了LUMOS,包括天文学和分子科学,并展示了其有效性和通用性。在13个SciML模型上的实验表明,LUMOS平均实现了71.45%的参数减少和6.4倍的推理加速。此外,使用最多八个GPU的分布式数据并行(DDP)训练证实了其可扩展性。

英文摘要:

The rapid growth of scientific machine learning (SciML) has accelerated discovery across diverse domains, yet designing effective SciML models remains a challenging task. In practice, building such models often requires substantial prior knowledge and manual expertise, particularly in determining which input features to use and how large the model should be. We introduce LUMOS, an end-to-end framework based on L0-regularized learning that unifies feature selection and model pruning to democratize SciML model design. By employing semi-stochastic gating and reparameterization techniques, LUMOS dynamically selects informative features and prunes redundant parameters during training, reducing the reliance on manual tuning while maintaining predictive accuracy. We evaluate LUMOS across 13 diverse SciML workloads, including cosmology and molecular sciences, and demonstrate its effectiveness and generalizability. Experiments on 13 SciML models show that LUMOS achieves 71.45% parameter reduction and a 6.4x inference speedup on average. Furthermore, Distributed Data Parallel (DDP) training on up to eight GPUs confirms the scalability of

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑