FinBERT: Financial Sentiment Analysis with Pre-trained Language Models
University of Amsterdam（阿姆斯特丹大学）；Naspers Group（纳斯帕斯集团）
Dogu Araci
备注：
This thesis is submitted in partial fulfillment for the degree of Master of Science in Information Studies: Data Science, University of Amsterdam. June 25, 2019
Financial sentiment analysis is a challenging task due to the specialized language and lack of labeled data in that domain. General-purpose models are not effective enough because of the specialized language used in a financial context. We hypothesize that pre-trained language models can help with this problem because they require fewer labeled examples and they can be further trained on domain-specific corpora. We introduce FinBERT, a language model based on BERT, to tackle NLP tasks in the financial domain. Our results show improvement in every measured metric on current state-of-the-art results for two financial sentiment analysis datasets. We find that even with a smaller training set and fine-tuning only a part of the model, FinBERT outperforms state-of-the-art machine learning methods.
Hong Kong University of Science and Technology（香港科技大学）
Contextual pretrained language models, such as BERT (Devlin et al., 2019), have made significant breakthrough in various NLP tasks by training on large scale of unlabeled text re-sources.Financial sector also accumulates large amount of financial communication text.However, there is no pretrained finance specific language models available. In this work,we address the need by pretraining a financial domain specific BERT models, FinBERT, using a large scale of financial communication corpora. Experiments on three financial sentiment classification tasks confirm the advantage of FinBERT over gene…
University of California San Diego（加州大学圣迭戈分校）；Georgia Institute of Technology（佐治亚理工学院）；New York University（纽约大学）；Tulane University（路易斯安那州立大学）；Southern Methodist University（南方 Methodist 大学）
Event-Aware Sentiment Factors from LLM-Augmented Financial Tweets: A Transparent Framework for Interpretable Quant Trading
Department of Physics, University of Cambridge, Cambridge, UK（剑桥大学物理系）；Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, UK（剑桥大学应用数学与理论物理系）
The Ohio State University（俄亥俄州立大学）；Google Research（谷歌研究院）
Market sentiment analysis on social media content requires knowledge of both financial markets and social media jargon, which makes it a challenging task for human raters. The resulting lack of high-quality labeled data stands in the way of conventional supervised learning methods. Instead, we approach this problem using semi-supervised learning with a large language model (LLM). Our pipeline generates weak financial sentiment labels for Reddit posts with an LLM and then uses that data to train a small model that can be served in production. We find that prompting the LLM to produce Chain-of-T…
School of Management, University of Science and Technology of China（中国科学技术大学管理学院）
Financial named entity recognition (FinNER) from literature is a challenging task in the field of financial text information extraction, which aims to extract a large amount of financial knowledge from unstructured texts. It is widely accepted to use sequence tagging frameworks to implement FinNER tasks. However, such sequence tagging models cannot fully take advantage of the semantic information in the texts. Instead, we formulate the FinNER task as a machine reading comprehension (MRC) problem and propose a new model termed FinBERT-MRC. This formulation introduces significant prior informati…
Fine-tuning of lightweight large language models for sentiment classification on heterogeneous financial textual data
International Business School, Zhejiang University（浙江大学国际商务学院）；Center for Data Science, Zhejiang University（浙江大学数据科学中心）；Centre for Statistics, Georg-August-Universität Göttingen（哥廷根大学统计中心）
Hypernym and synonym matching are one of the mainstream Natural Language Processing (NLP) tasks. In this paper, we present systems that attempt to solve this problem. We designed these systems to participate in the FinSim-3, a shared task of FinNLP workshop at IJCAI-2021. The shared task is focused on solving this problem for the financial domain. We experimented with various transformer based pre-trained embeddings by fine-tuning these for either classification or phrase similarity tasks. We also augmented the provided dataset with abbreviations derived from prospectus provided by the organiz…
Newsletters and social networks can reflect the opinion about the market and specific stocks from the perspective of analysts and the general public on products and/or services provided by a company. Therefore, sentiment analysis of these texts can provide useful information to help investors trade in the market. In this paper, a hierarchical stack of Transformers model is proposed to identify the sentiment associated with companies and stocks, by predicting a score (of data type real) in a range between -1 and +1. Specifically, we fine-tuned a RoBERTa model to process headlines and microblogs…
Beyond Sentiment: Structured Information Extraction from Financial News
School of ASEE, Beihang University（北京航空航天大学ASEE学院）；School of BME, Beihang University（北京航空航天大学生物医学工程学院）；CAIR and CECS, VinUniversity（VinUniversity CAIR与CECS机构）
Sentiment analysis as a sub-field of natural language processing has received increased attention in the past decade enabling organisations to more effectively manage their reputation through online media monitoring. Many drivers impact reputation, however, this thesis focuses only the aspect of financial performance and explores the gap with regards to financial sentiment analysis in a South African context. Results showed that pre-trained sentiment analysers are least effective for this task and that traditional lexicon-based and machine learning approaches are best suited to predict financi…
Indian Institute of Management Ahmedabad（印度管理学院艾哈迈达巴德分校）
Mining financial text documents and understanding the sentiments of individual investors, institutions and markets is an important and challenging problem in the literature. Current approaches to mine sentiments from financial texts largely rely on domain specific dictionaries. However, dictionary based methods often fail to accurately predict the polarity of financial texts. This paper aims to improve the state-of-the-art and introduces a novel sentiment analysis approach that employs the concept of financial and non-financial performance indicators. It presents an association rule mining bas…
Stock market forecasting is very important in the planning of business activities. Stock price prediction has attracted many researchers in multiple disciplines including computer science, statistics, economics, finance, and operations research. Recent studies have shown that the vast amount of online information in the public domain such as Wikipedia usage pattern, news stories from the mainstream media, and social media discussions can have an observable effect on investors opinions towards financial markets. The reliability of the computational models on stock market prediction is important…
City University of Hong Kong（香港城市大学）；USTC-CityU Joint Advanced Research Centre（中国科学技术大学-香港城市大学联合高等研究中心）
With the proliferation of its applications in various industries, sentiment analysis by using publicly available web data has become an active research area in text classification during these years. It is argued by researchers that semi-supervised learning is an effective approach to this problem since it is capable to mitigate the manual labeling effort which is usually expensive and time-consuming. However, there was a long-term debate on the effectiveness of unlabeled data in text classification. This was partially caused by the fact that many assumptions in theoretic analysis often do not…
Insight Centre for Data Analytics（洞察数据分析中心）；National University of Ireland, Galway（爱尔兰国立大学戈尔韦分校）
We introduce FinLin, a novel corpus containing investor reports, company reports, news articles, and microblogs from StockTwits, targeting multiple entities stemming from the automobile industry and covering a 3-month period. FinLin was annotated with a sentiment score and a relevance score in the range [-1.0, 1.0] and [0.0, 1.0], respectively. The annotations also include the text spans selected for the sentiment, thus, providing additional insight into the annotators' reasoning. Overall, FinLin aims to complement the current knowledge by providing a novel and publicly available financial sen…
Aspect-Level Obfuscated Sentiment in Thai Financial Disclosures and Its Impact on Abnormal Returns
Department of Linguistics, Faculty of Arts, Chulalongkorn University（语言学系，艺术学院，朱拉隆梭大学）；Sasin Graduate Institute of Business Administration, Chulalongkorn University（工商管理研究生院，朱拉隆梭大学）
Increasing attention has been drawn to the sentiment analysis of financial documents. The most popular examples of such documents include analyst reports and economic news, the analysis of which is frequently used to capture the trends in market sentiments. On the other hand, the significance of the role sentiment analysis plays in the financial domain has given rise to the efforts to construct a financial domain-specific sentiment lexicon. Sentiment lexicons lend a hand for solving various text mining tasks, such as unsupervised classification of text data, while alleviating the arduous human…
The emergence and rapid progress of the Internet have brought ever-increasing impact on financial domain. How to rapidly and accurately mine the key information from the massive negative financial texts has become one of the key issues for investors and decision makers. Aiming at the issue, we propose a sentiment analysis and key entity detection approach based on BERT, which is applied in online financial text mining and public opinion analysis in social media. By using pre-train model, we first study sentiment analysis, and then we consider key entity detection as a sentence matching or Mach…
Barcelona Supercomputing Center（巴塞罗那超级计算中心）；LHF Labs（LHF实验室）；Artificial Intelligence Finance Institute（人工智能金融研究所）；NYU Courant（纽约大学库朗数学科学研究所）
We introduce a new language representation model in finance called Financial Embedding Analysis of Sentiment (FinEAS). In financial markets, news and investor sentiment are significant drivers of security prices. Thus, leveraging the capabilities of modern NLP approaches for financial sentiment analysis is a crucial component in identifying patterns and trends that are useful for market participants and regulators. In recent years, methods that use transfer learning from large Transformer-based language models like BERT, have achieved state-of-the-art results in text classification tasks, incl…
In this paper, we design an integrated algorithm to evaluate the sentiment of Chinese market. Firstly, with the help of the web browser automation, we crawl a lot of news and comments from several influential financial websites automatically. Secondly, we use techniques of Natural Language Processing(NLP) under Chinese context, including tokenization, Word2vec word embedding and semantic database WordNet, to compute Senti-scores of these news and comments, and then construct the sentimental factor. Here, we build a finance-specific sentimental lexicon so that the sentimental factor can reflect…
This paper describes our participation in Task 5 track 2 of SemEval 2017 to predict the sentiment of financial news headlines for a specific company on a continuous scale between -1 and 1. We tackled the problem using a number of approaches, utilising a Support Vector Regression (SVR) and a Bidirectional Long Short-Term Memory (BLSTM). We found an improvement of 4-6% using the LSTM model over the SVR and came fourth in the track. We report a number of different evaluations using a finance specific word embedding model and reflect on the effects of using different evaluation metrics.
This paper discusses the approach taken by the UWaterloo team to arrive at a solution for the Fine-Grained Sentiment Analysis problem posed by Task 5 of SemEval 2017. The paper describes the document vectorization and sentiment score prediction techniques used, as well as the design and implementation decisions taken while building the system for this task. The system uses text vectorization models, such as N-gram, TF-IDF and paragraph embeddings, coupled with regression model variants to predict the sentiment scores. Amongst the methods examined, unigrams and bigrams coupled with simple linea…
FinXABSA: Explainable Finance through Aspect-Based Sentiment Analysis
Asian Institute of Digital Finance（亚洲数字金融学院）；National University of Singapore（新加坡国立大学）；School of Computer Science and Engineering（计算机科学与工程学院）；Nanyang Technological University（南洋理工大学）；Institute of High Performance Computing (IHPC)（高性能计算研究所）；Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局）；College of Design & Engineering（设计与工程学院）
