arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2512.00630cs.LGq-fin.CP
收藏

基于Qwen3-8B模型的金融文本分类研究:rLoRA微调

Financial Text Classification Based On rLoRA Finetuning On Qwen3-8B model

LL Funds LLC(LL Funds有限责任公司)

Zhiming Lian

AI总结:

本文提出基于rLoRA微调的Qwen3-8B模型,用于金融文本分类,展示了其在准确率和训练效率上的优势。

摘要:

金融文本分类已成为量化交易系统及相关任务中的重要组成部分,例如金融情感分析和金融新闻分类。本文评估了大型语言模型Qwen3-8B在这两项任务上的性能。Qwen3-8B是一种最先进的模型,具有强大的指令遵循和多语言能力,与标准模型不同,主要是因为它专门优化了高效的微调和在基于推理的基准测试中的高性能,使其适用于金融应用。为了适应此模型,我们应用了Noisy Embedding Instruction微调方法,并基于我们之前的工作,该方法通过在监督适应过程中向嵌入层注入受控噪声来提高鲁棒性。我们进一步通过Rank-stabilized Low-Rank Adaptation低秩优化方法和FlashAttention,实现了更高的效率,允许更快的训练和更低的GPU内存需求。对于这两种任务,我们将Qwen3-8B与标准经典Transformer模型(如T5、BERT和RoBERTa)以及大规模模型(如LLaMA1-7B、LLaMA2-7B和Baichuan2-7B)进行基准测试。研究结果表明,Qwen3-8B在分类准确率方面优于这些基线,并且需要更少的训练周期。指令微调与内存高效优化方法的结合表明,Qwen3-8B可能成为实时金融NLP应用的可扩展且经济的选项。Qwen3-8B为未来动态量化交易系统的进步提供了非常有前途的基础。

英文摘要:

Financial text classification has increasingly become an important aspect in quantitative trading systems and related tasks, such as financial sentiment analysis and the classification of financial news. In this paper, we assess the performance of the large language model Qwen3-8B on both tasks. Qwen3-8B is a state-of-the-art model that exhibits strong instruction-following and multilingual capabilities, and is distinct from standard models, primarily because it is specifically optimized for efficient fine tuning and high performance on reasoning-based benchmarks, making it suitable for financial applications. To adapt this model, we apply Noisy Embedding Instruction Finetuning and based on our previous work, this method increases robustness by injecting controlled noise into the embedding layers during supervised adaptation. We improve efficiency further with Rank-stabilized Low-Rank Adaptation low-rank optimization approach, and FlashAttention, which allow for faster training with lower GPU memory. For both tasks, we benchmark Qwen3-8B against standard classical transformer models, such as T5, BERT, and RoBERTa, and large models at scale, such as LLaMA1-7B, LLaMA2-7B, and Baichuan2-7B. The findings reveal that Qwen3-8B consistently surpasses these baselines by obtaining better classification accuracy and needing fewer training epochs. The synergy of instruction-based fine-tuning and memory-efficient optimization methods suggests Qwen3-8B can potentially serve as a scalable, economical option for real-time financial NLP applications. Qwen3-8B provides a very promising base for advancing dynamic quantitative trading systems in the future.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑