LLMs have revolutionized NLP and demonstrated potential across diverse domains. More and more financial LLMs have been introduced for finance-specific tasks, yet comprehensively assessing their value is still challenging. In this paper, we introduce FLAME, a comprehensive financial LLMs evaluation system in Chinese, which includes two core evaluation benchmarks: FLAME-Cer and FLAME-Sce. FLAME-Cer covers 14 types of authoritative financial certifications, including CPA, CFA, and FRM, with a total of approximately 16,000 carefully selected questions. All questions have been manually reviewed to ensure accuracy and representativeness. FLAME-Sce consists of 10 primary core financial business scenarios, 21 secondary financial business scenarios, and a comprehensive evaluation set of nearly 100 tertiary financial application tasks. We evaluate 6 representative LLMs, including GPT-4o, GLM-4, ERNIE-4.0, Qwen2.5, XuanYuan3, and the latest Baichuan4-Finance, revealing Baichuan4-Finance excels other LLMs in most tasks. By establishing a comprehensive and professional evaluation system, FLAME facilitates the advancement of financial LLMs in Chinese contexts. Instructions for participating in the evaluation are available on GitHub: https://github.com/FLAME-ruc/FLAME.
FinDeepIndicator: Benchmarking Deep Research Agents in End-to-End Financial Indicator Construction
School of Computing, National University of Singapore（新加坡国立大学计算机学院）；China Economics and Management Academy, Central University of Finance and Economics（中央财经大学中国经济与管理研究院）；Asian Institute of Digital Finance, National University of Singapore（新加坡国立大学亚洲数字金融研究所）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
FinTagging: Benchmarking LLMs for Extracting and Structuring Financial Information
Georgia Institute of Technology（佐治亚理工学院）；Columbia University（哥伦比亚大学）；California State University（加州州立大学）；University of Montreal（蒙特利尔大学）；Carnegie Mellon University（卡内基梅隆大学）；Rensselaer Polytechnic Institute（莱斯利理工学院）；The University of Manchester（曼彻斯特大学）；Harvard University（哈佛大学）
FinDABench: Benchmarking Financial Data Analysis Ability of Large Language Models
School of Computer Science and Technology, East China Normal University（华东师范大学计算机科学与技术学院）；Shanghai Institute of AI for Education, East China Normal University（华东师范大学上海人工智能教育研究院）；PICC AMC（中国人民保险资产管理公司）；Bytedance（字节跳动）
Are LLMs Rational Investors? A Study on Detecting and Reducing the Financial Bias in LLMs
School of Computer Science, Fudan University（复旦大学计算机科学技术学院）；Institute of Fintech, Fudan University（复旦大学金融科技研究院）；School of Electronics and Information Engineering, Tongji University（同济大学电子与信息工程学院）；Tandon School of Engineering, New York University（纽约大学坦登工程学院）；DataGrand Inc.（数据堂公司）
