arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.08704cs.AI
收藏

评估大型语言模型的金融智能:用LLM引擎基准测试SuperInvesting AI

Evaluating Financial Intelligence in Large Language Models: Benchmarking SuperInvesting AI with LLM Engines

Akshay Gulati, Kanha Singhania, Tushar Banga, Parth Arora, Anshul Verma, Vaibhav Kumar Singh, Agyapal Digra, Jayant Singh Bisht, Danish Sharma, Varun Singla, Shubh Garg

AI总结:

本文提出AFIB基准测试,评估SuperInvesting等AI系统在金融分析中的性能,发现SuperInvesting在准确性、完整性和一致性方面表现最优。

摘要:

大型语言模型越来越多地用于金融分析和投资研究,但对其金融推理能力的系统评估仍然有限。在本工作中,我们介绍了人工智能金融智能基准(AFIB),一个多维评估框架,旨在评估五个维度上的金融分析能力:事实准确性、分析完整性、数据时效性、模型一致性以及失败模式。我们评估了五个AI系统:GPT、Gemini、Perplexity、Claude和SuperInvesting,使用了95多个结构化的金融分析问题数据集,这些问题是基于真实的股票研究任务衍生而来的。结果揭示了模型之间性能存在显著差异。在本基准设置中,SuperInvesting实现了最高的综合性能,其平均事实准确性得分为8.96/10,完整性得分为56.65/70,同时在所有评估系统中表现出最低的幻觉率。以检索为导向的系统如Perplexity在数据时效性任务中表现强劲,由于拥有实时信息访问能力,但其分析综合和一致性表现较弱。总体而言,结果表明大型语言模型的金融智能本质上是多维的,结合结构化金融数据访问与分析推理能力的系统,为复杂的投资研究流程提供了最可靠的性能。

英文摘要:

Large language models are increasingly used for financial analysis and investment research, yet systematic evaluation of their financial reasoning capabilities remains limited. In this work, we introduce the AI Financial Intelligence Benchmark (AFIB), a multi-dimensional evaluation framework designed to assess financial analysis capabilities across five dimensions: factual accuracy, analytical completeness, data recency, model consistency, and failure patterns. We evaluate five AI systems: GPT, Gemini, Perplexity, Claude, and SuperInvesting, using a dataset of 95+ structured financial analysis questions derived from real-world equity research tasks. The results reveal substantial differences in performance across models. Within this benchmark setting, SuperInvesting achieves the highest aggregate performance, with an average factual accuracy score of 8.96/10 and the highest completeness score of 56.65/70, while also demonstrating the lowest hallucination rate among evaluated systems. Retrieval-oriented systems such as Perplexity perform strongly on data recency tasks due to live information access but exhibit weaker analytical synthesis and consistency. Overall, the results highlight that financial intelligence in large language models is inherently multi-dimensional, and systems that combine structured financial data access with analytical reasoning capabilities provide the most reliable performance for complex investment research workflows.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑