发表机构
Asian Institute of Digital Finance, National University of Singapore(新加坡国立大学亚洲数字金融研究院)
机构由 AI 辅助整理,请以论文原文为准。AI 中文总结
该研究提出Search-to-Record任务与DelistBench基准,评估支持搜索的LLM在公司事件数据库补全中的表现,发现网络检索可显著提升准确率,低成本系统也能接近最优效果,并给出部署指导。
AI 中文摘要
金融机构需要一种独立的方式来检测供应商数据库中缺失、过时和分类错误的公司事件记录。我们引入了Search-to-Record,这是一种数据库保障任务,其中支持搜索的大型语言模型会针对已知的证券范围和历史截止日期,从公开来源重建机构定义的事件记录;同时推出DelistBench,这是一个包含1200条记录的基准,用于证券层面的退市公告。我们在闭卷和启用网络的配对条件下评估了五个模型。网络访问将七天内的公告日期准确率提高了34.0至48.0个百分点,将事件状态准确率提高了约2.8至21.7个百分点;最佳系统在七天内实现了81.5%的整体联合准确率。经济型网络系统在七天内实现了75.9-78.3%的整体联合准确率,但其API成本仅为最昂贵网络系统的4.5-6.6%。基于风险的分类识别出低错误子集,尽管最高覆盖的操作点仍将27.3%的平衡测试集发送至审查。评估确定网络检索是时间提升的主要来源,并表明低成本系统可以接近最佳系统的准确率。总体而言,Search-to-Record、DelistBench及评估提供了具体的部署指导:根据本地事件发生率和市场组合校准分类,保留正事件召回率,并将正例和模糊案例路由至针对性审查。
英文摘要
Financial institutions need an independent way to detect missing, stale, and misclassified corporate-event records in vendor databases. We introduce Search-to-Record, a database-assurance task in which search-enabled large language models reconstruct institution-defined event records from public sources for a known security universe and historical cutoff, and DelistBench, a 1,200-record benchmark for security-level delisting announcements. We evaluate five models in paired closed-book and web-enabled conditions. Web access raises announcement-date accuracy within seven days by 34.0 to 48.0 percentage points and event-status accuracy by approximately 2.8 to 21.7 points; the best system achieves 81.5% overall joint accuracy within seven days. Economy web systems achieve 75.9-78.3% overall joint accuracy within seven days at 4.5-6.6% of the API cost of the most expensive web system. Risk-based triage identifies low-error subsets, although the highest-coverage operating point still sends 27.3% of the balanced test set to review. The evaluation identifies web retrieval as the main source of timing gains and shows that low-cost systems can approach the best system's accuracy. Together, Search-to-Record, DelistBench, and the evaluation provide concrete deployment guidance: calibrate triage to local event prevalence and market mix, preserve positive-event recall, and route positive and ambiguous cases to targeted review.