本文提出开源无代码工具AutoTrain,支持在自定义数据集上跨多种模态和任务训练或微调模型,兼容本地与云端环境及Hugging Face Hub上数万个模型。
摘要:
随着开源模型的进步,在自定义数据集上训练(或微调)模型已成为开发针对特定工业或开源应用的定制化解决方案的关键部分。然而,目前尚无单一工具能够简化跨不同模态或任务的训练过程。我们引入了AutoTrain(又称AutoTrain Advanced)——一个开源的无代码工具/库,可用于训练(或微调)不同类型任务的模型,例如:大语言模型(LLM)微调、文本分类/回归、token分类、序列到序列任务、句子变换器微调、视觉语言模型(VLM)微调、图像分类/回归,甚至表格数据上的分类和回归任务。AutoTrain Advanced是一个开源库,提供了在自定义数据集上训练模型的最佳实践。该库可在 https://github.com/huggingface/autotrain-advanced 获取。AutoTrain可在完全本地模式或云机器上使用,并支持Hugging Face Hub上共享的数万个模型及其变体。
英文摘要:
With the advancements in open-source models, training (or finetuning) models on custom datasets has become a crucial part of developing solutions which are tailored to specific industrial or open-source applications. Yet, there is no single tool which simplifies the process of training across different types of modalities or tasks. We introduce AutoTrain (aka AutoTrain Advanced) -- an open-source, no code tool/library which can be used to train (or finetune) models for different kinds of tasks such as: large language model (LLM) finetuning, text classification/regression, token classification, sequence-to-sequence task, finetuning of sentence transformers, visual language model (VLM) finetuning, image classification/regression and even classification and regression tasks on tabular data. AutoTrain Advanced is an open-source library providing best practices for training models on custom datasets. The library is available at https://github.com/huggingface/autotrain-advanced. AutoTrain can be used in fully local mode or on cloud machines and works with tens of thousands of models shared on Hugging Face Hub and their variations.
How code helps different tasks? A decompositional lens on LLM post-training
Sun Yat-sen University(中山大学);Shenzhen Loop Area Institute(深圳河套学院);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Shenzhen Research Institute of Big Data(深圳市大数据研究院)
SynCoBERT: Syntax-Guided Multi-Modal Contrastive Pre-Training for Code Representation
Wuhan University(武汉大学);Noah’s Ark Lab, Huawei(华为诺亚方舟实验室);Huazhong University of Science and Technology(华中科技大学);Deakin University(迪肯大学);Monash University(莫纳什大学);Yunnan University(云南大学)
CODE-MVP: Learning to Represent Source Code from Multiple Views with Contrastive Pre-Training
Wuhan University(武汉大学);Huawei Noah’s Ark Lab(华为诺亚方舟实验室);Huazhong University of Science and Technology(华中科技大学);Monash University(莫纳什大学);Yunnan University(云南大学)
Promise and Peril of Collaborative Code Generation Models: Balancing Effectiveness and Memorization
Centre for Research on Intelligent Software Engineering(智能软件工程研究中心);School of Computing and Information Systems(计算与信息系统学院);Singapore Management University(新加坡管理大学)
Learning Binary Autoencoder-Based Codes with Progressive Training
The Institute for Artificial Intelligence Research and Development of Serbia(塞尔维亚人工智能研究与发展研究所);Faculty of Technical Sciences, University of Novi Sad(诺维萨德大学技术科学学院)
Aletheia: What Makes RLVR For Code Verifiers Tick?
INSAIT, Sofia University "St. Kliment Ohridski", Bulgaria(保加利亚索菲亚大学INSAIT实验室);Ubiquitous Knowledge Processing Lab (UKP Lab), Department of Computer Science, Technical University of Darmstadt and National Research Center for Applied Cybersecurity(德累斯顿技术大学计算机科学系及应用网络安全国家研究中心通用知识处理实验室);ATHENE, Germany(德国ATHENE研究院)
Pull Requests as a Training Signal for Repo-Level Code Editing
King's College London, UK(伦敦国王学院);Chinese University of Hong Kong, HK(香港中文大学);National University of Singapore, SG(新加坡国立大学);Microsoft Research Asia, CN(微软亚洲研究院);The Alan Turing Institute, UK(艾伦·图灵研究所)