发表机构
The Hong Kong University of Science and Technology; University of California, Berkeley（香港科技大学; 加州大学伯克利分校）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出基于结构的迁移学习框架，利用LLM等智能体生成的支撑集改进目标估计，通过映射、加权聚合和稀疏修正，理论保证不劣于Lasso，并经模拟和基因表达数据验证。
AI 中文摘要
迁移学习利用相关来源的信息改进目标研究中的估计。经典迁移学习通常是基于数据的，需要访问源数据或基于这些数据拟合的模型。在许多现代研究中，这两者都不可用，因为它们通常是专有的或未报告的。可以迁移的替代是结构：从源导出的概括性信息，例如预测变量的支撑集，这些信息在无需访问源数据或模型的情况下即可获得且可解释。大型语言模型（LLM）的普及提供了此类结构信息的丰富来源。然而，基于数据的迁移学习无法很好地利用它，因为访问LLM的数据和模型通常是不可行的。在本文中，我们提出了一种新颖的基于结构的迁移学习框架。具体而言，我们关注线性模型，旨在利用由多个智能体（如LLM）生成的支撑集来改进目标研究中的估计。我们的程序将每个支撑集映射到目标参数空间，使用数据驱动的权重聚合结果，并针对不准确的支撑集应用稀疏修正。我们建立了非渐近保证：当支撑集具有信息性时，估计器优于仅使用目标数据的Lasso，否则不会更差。模拟实验和基因表达应用证实了这两个性质。
英文摘要
Transfer learning improves estimation in a target study using information from related sources. Classical transfer learning is typically data-based, requiring access to the source data or to a model fitted on them. Neither is available in many modern studies, as both are often proprietary or unreported. What can be transferred instead is structure: summarized information derived from a source, such as the supports of predictors, which is available and interpretable without access to the source data or model. The prevalence of the Large Language Model (LLM) provides a rich source of such structure information. However, data-based transfer learning can not well utilize it as accessing the data and model of the LLM is often infeasible. In this paper, we propose a novel transfer learning framework based on structure. In particular, we focus on linear model and aim to use supports generated by multiple agents, such as LLM, to improve estimation in the target study. Our procedure maps each support into the target parameter space, aggregates the results with data-driven weights, and applies a sparse correction against inaccurate supports. We establish nonasymptotic guarantees: the estimator improves on the target-only Lasso when the supports are informative and never does worse otherwise. Simulations and a gene-expression application confirm both properties.
