arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.08181cs.LG
收藏

AutoAdapt:一种用于大语言模型的自动化领域适应框架

AutoAdapt: An Automated Domain Adaptation Framework for LLMs

Sidharth Sinha, Anson Bastos, Xuchao Zhang, Akshay Nambi, Chetan Bansal, Saravan Rajmohan

AI总结:

AutoAdapt通过多代理辩论系统和AutoRefine方法,实现高效可靠的LLM领域适应,提升领域适应性能25%。

摘要:

大型语言模型(LLMs)在开放领域表现出色,但在数据有限且知识不断变化的专用领域中表现不佳。现有的领域适应实践严重依赖手动尝试和错误的过程,导致显著的超参数复杂性,并且对数据和用户偏好高度敏感,所有这些都在LLM训练的高成本下进行。此外,超参数选择在不同模型/领域之间的交互和可转移性仍然理解有限,即使付出大量努力,适应收益也存在不确定性。为了解决这些挑战,我们提出了AutoAdapt,一种新颖的端到端自动化框架,用于高效且可靠的LLM领域适应。AutoAdapt利用文献和开源资源中的精选知识库来减少专家干预。为了缩小搜索空间,我们设计了一种新型的多代理辩论系统,其中提案代理和批评代理迭代交互,以对齐用户意图,并将数据信号和最佳实践纳入规划过程。为了在紧预算下优化超参数,我们提出了AutoRefine,一种新颖的基于LLM的替代方案,用以替代成本高昂的黑箱搜索。在10个任务中,AutoAdapt在状态-of-the-art自动化机器学习基线之上实现了25%的平均相对准确率提升,且开销极小。

英文摘要:

Large language models (LLMs) excel in open domains but struggle in specialized settings with limited data and evolving knowledge. Existing domain adaptation practices rely heavily on manual trial-and-error processes, incur significant hyperparameter complexity, and are highly sensitive to data and user preferences, all under the high cost of LLM training. Moreover, the interactions and transferability of hyperparameter choices across models/domains remain poorly understood, making adaptation gains uncertain even with substantial effort. To solve these challenges, we present AutoAdapt, a novel end-to-end automated framework for efficient and reliable LLM domain adaptation. AutoAdapt leverages curated knowledge bases from literature and open-source resources to reduce expert intervention. To narrow the search space, we design a novel multi-agent debating system in which proposal and critic agents iteratively interact to align user intent and incorporate data signals and best practices into the planning process. To optimize hyperparameters under tight budgets, we propose AutoRefine, a novel LLM-based surrogate that replaces costly black-box search. Across 10 tasks, AutoAdapt achieves a 25% average relative accuracy improvement over state-of-the-art Automated Machine Learning baselines with minimal overhead.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑