Unstructured data, especially text, continues to grow rapidly in various domains. In particular, in the financial sphere, there is a wealth of accumulated unstructured financial data, such as the textual disclosure documents that companies submit on a regular basis to regulatory agencies, such as the Securities and Exchange Commission (SEC). These documents are typically very long and tend to contain valuable soft information about a company's performance. It is therefore of great interest to learn predictive models from these long textual documents, especially for forecasting numerical key performance indicators (KPIs). Whereas there has been a great progress in pre-trained language models (LMs) that learn from tremendously large corpora of textual data, they still struggle in terms of effective representations for long documents. Our work fills this critical need, namely how to develop better models to extract useful information from long textual documents and learn effective features that can leverage the soft financial and risk information for text regression (prediction) tasks. In this paper, we propose and implement a deep learning framework that splits long documents into chunks and utilizes pre-trained LMs to process and aggregate the chunks into vector representations, followed by self-attention to extract valuable document-level features. We evaluate our model on a collection of 10-K public disclosure reports from US banks, and another dataset of reports submitted by US companies. Overall, our framework outperforms strong baseline methods for textual modeling as well as a baseline regression model using only numerical data. Our work provides better insights into how utilizing pre-trained domain-specific and fine-tuned long-input LMs in representing long documents can improve the quality of representation of textual data, and therefore, help in improving predictive analyses.
FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial Reasoning
MBZUAI;Syllogia;The University of Tokyo(东京大学);The Fin AI(Fin AI);Cornell University(康奈尔大学);The University of Melbourne(墨尔本大学);IIT Delhi(德里理工学院);The University of Manchester(曼彻斯特大学)
Harnessing Earnings Reports for Stock Predictions: A QLoRA-Enhanced LLM Approach
Columbia University(哥伦比亚大学);Central University of Finance and Economics(中央财经大学);New York University(纽约大学);Cornell University(康奈尔大学);University of Michigan(密歇根大学);Rutgers University(罗格斯大学)
Retrieval-augmented Large Language Models for Financial Time Series Forecasting
School of Artificial Intelligence, Wuhan University(武汉大学人工智能学院);School of Computer Science, Wuhan University(武汉大学计算机学院);The Fin AI(Fin AI);Columbia University(哥伦比亚大学);Stevens Institute of Technology(史蒂文斯理工学院);University of Manchester(曼彻斯特大学);University of Montreal(蒙特利尔大学)
FinSphere, a Real-Time Stock Analysis Agent Powered by Instruction-Tuned LLMs and Domain Tools
JF SmartInvest Holdings Ltd.(JF SmartInvest控股有限公司);Columbia University(哥伦比亚大学);Shanghai University of Finance and Economics(上海财经大学);Johns Hopkins University(约翰霍普金斯大学)
Open-FinLLMs: Open Multimodal Large Language Models for Financial Applications
The Fin AI(Fin AI);Wuhan University(武汉大学);Columbia University(哥伦比亚大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Nanjing University(南京大学);Rensselaer Polytechnic Institute(罗格斯理工学院);The University of Manchester(曼彻斯特大学);Stevens Institute of Technology(史泰斯理工学院);National University of Singapore(新加坡国立大学);University of Florida(佛罗里达大学);University of Montreal(蒙特利尔大学);Yale University(耶鲁大学);New York University(纽约大学);Harvard University(哈佛大学);NVIDIA(英伟达);Artificial Intelligence Research Centre(人工智能研究中心);Archimedes/Athena Research Centre(Archimedes/Athena研究中心)
FinBERT: A Pretrained Language Model for Financial Communications
Hong Kong University of Science and Technology(香港科技大学)
上下文预训练语言模型,例如BERT(Devlin et al.,2019年),通过对大规模未标记文本资源的训练,在各种NLP任务中取得了重大突破。金融领域也积累了大量的金融沟通文本。但是,没有预训练的金融特定语言模型可用。在这项工作中,我们解决了需要通过预训练金融领域特定的BERT模型,FinBERT,使用大规模的金融通信语料库。在三个金融情感分类任务上的实验证实了FinBERT模型相对于一般领域BERT模型的优势。代码和预训练模型可在https://github.com/yya518/FinBERT上获得。我们希望这对从事金融NLP任务的从业者和研究人员有用。
FinEval: A Chinese Financial Domain Knowledge Evaluation Benchmark for Large Language Models
Shanghai University of Finance and Economics(上海财经大学);University of California, Irvine(加州大学欧文分校);Fudan University(复旦大学);Ping An Technology (Shenzhen) Co., Ltd.(平安科技(深圳)有限公司);Tencent YouTu Lab(腾讯优图实验室);HSBC Lab(汇丰实验室)