面向低资源LLM任务的可迁移建模策略：一种基于提示与对齐的方法
Transferable Modeling Strategies for Low-Resource LLM Tasks: A Prompt and Alignment-Based Approach
AI总结：
本文提出一种结合知识迁移、对齐损失、软提示微调和冻结策略的统一框架，在极少标注下提升大语言模型在低资源跨语言任务上的性能、稳定性与可扩展性。
摘要：
本文针对大语言模型在低资源语言场景下迁移和适应能力有限的问题，提出一个统一框架，将知识迁移模块与参数高效微调策略相结合。该方法引入知识对齐损失和软提示微调，引导模型在极少标注条件下有效吸收目标语言或任务的结构特征，从而提升泛化性能和训练稳定性。该框架包含轻量级适应模块以降低计算成本。训练过程中，它整合冻结策略和提示注入，在保留模型原有知识的同时实现对新任务的快速适应。研究还开展了稳定性分析实验和合成伪数据迁移实验，以系统评估该方法在不同低资源任务上的适用性和鲁棒性。实验结果表明，与现有多语言预训练模型和主流迁移方法相比，所提方法在MLQA、XQuAD和PAWS-X等跨语言任务上取得了更高的性能和稳定性，并在数据极度稀缺条件下表现出尤为明显的优势。所提方法具有较强的通用性和可扩展性，在保持大语言模型通用能力的同时增强了任务特定适应性，使其非常适合复杂语义建模和多语言处理任务。
英文摘要：
This paper addresses the limited transfer and adaptation capabilities of large language models in low-resource language scenarios. It proposes a unified framework that combines a knowledge transfer module with parameter-efficient fine-tuning strategies. The method introduces knowledge alignment loss and soft prompt tuning to guide the model in effectively absorbing the structural features of target languages or tasks under minimal annotation. This enhances both generalization performance and training stability. The framework includes lightweight adaptation modules to reduce computational costs. During training, it integrates freezing strategies and prompt injection to preserve the model's original knowledge while enabling quick adaptation to new tasks. The study also conducts stability analysis experiments and synthetic pseudo-data transfer experiments to systematically evaluate the method's applicability and robustness across different low-resource tasks. Experimental results show that compared with existing multilingual pre-trained models and mainstream transfer methods, the proposed approach achieves higher performance and stability on cross-lingual tasks such as MLQA, XQuAD, and PAWS-X. It demonstrates particularly strong advantages under extremely data-scarce conditions. The proposed method offers strong generality and scalability. It enhances task-specific adaptability while preserving the general capabilities of large language models. This makes it well-suited for complex semantic modeling and multilingual processing tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
