评估大型语言模型在SEC申报文件和财报电话会议记录中金融目标立场检测的表现
Evaluating Large Language Models for Stance Detection on Financial Targets from SEC Filing Reports and Earnings Call Transcripts
AI总结：
本研究构建了针对债务、每股收益和销售额的句子级金融立场检测语料库，并系统评估了LLMs在零样本、少样本和思维链策略下的表现，发现少样本结合思维链方法效果最佳，验证了无需大量标注数据即可实现金融目标立场检测的可行性。
摘要：
来自美国证券交易委员会（SEC）申报文件和季度财报电话会议记录（ECTs）的金融叙述对投资者、审计师和监管机构非常重要。然而，这些内容的长度、金融术语和微妙语言使得细粒度分析变得困难。此前金融领域的情感分析需要大量且昂贵的标注数据集，使得针对特定金融目标的句子级立场检测具有挑战性。在这项工作中，我们引入了一个针对三个核心财务指标——债务、每股收益（EPS）和销售额——的句子级立场检测语料库。这些句子从10-K年度报告和ECTs中提取，并使用先进的ChatGPT-o3-pro模型在严格的人工验证下进行立场（正面、负面、中性）标注。利用该语料库，我们使用零样本、少样本和思维链（CoT）提示策略对现代大型语言模型（LLMs）进行了系统评估。我们的结果表明，与监督基线相比，结合CoT提示的少样本方法表现最佳，且LLMs在SEC和ECT数据集上的性能存在差异。我们的研究结果突显了在无需大量标注数据的情况下，利用LLMs进行金融领域目标特定立场检测的实际可行性。
英文摘要：
Financial narratives from U.S. Securities and Exchange Commission (SEC) filing reports and quarterly earnings call transcripts (ECTs) are very important for investors, auditors, and regulators. However, their length, financial jargon, and nuanced language make fine-grained analysis difficult. Prior sentiment analysis in the financial domain required a large, expensive labeled dataset, making the sentence-level stance towards specific financial targets challenging. In this work, we introduce a sentence-level corpus for stance detection focused on three core financial metrics: debt, earnings per share (EPS), and sales. The sentences were extracted from Form 10-K annual reports and ECTs, and labeled for stance (positive, negative, neutral) using the advanced ChatGPT-o3-pro model under rigorous human validation. Using this corpus, we conduct a systematic evaluation of modern large language models (LLMs) using zero-shot, few-shot, and Chain-of-Thought (CoT) prompting strategies. Our results show that few-shot with CoT prompting performs best compared to supervised baselines, and LLMs' performance varies across the SEC and ECT datasets. Our findings highlight the practical viability of leveraging LLMs for target-specific stance in the financial domain without requiring extensive labeled data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
