Removing Non-Stationary Knowledge From Pre-Trained Language Models for Entity-Level Sentiment Classification in Finance
Underwood International College, Yonsei University（延世大学安德伍德国际学院）；NCSOFT；KB Kookmin Bank（KB国民银行）；College of Business & Economics, Chung-Ang University（中央大学工商经济学院）
Extraction of sentiment signals from news text, stock message boards, and business reports, for stock movement prediction, has been a rising field of interest in finance. Building upon past literature, the most recent works attempt to better capture sentiment from sentences with complex syntactic structures by introducing aspect-level sentiment classification (ASC). Despite the growing interest, however, fine-grained sentiment analysis has not been fully explored in non-English literature due to the shortage of annotated finance-specific data. Accordingly, it is necessary for non-English languages to leverage datasets and pre-trained language models (PLM) of different domains, languages, and tasks to best their performance. To facilitate finance-specific ASC research in the Korean language, we build KorFinASC, a Korean aspect-level sentiment classification dataset for finance consisting of 12,613 human-annotated samples, and explore methods of intermediate transfer learning. Our experiments indicate that past research has been ignorant towards the potentially wrong knowledge of financial entities encoded during the training phase, which has overestimated the predictive power of PLMs. In our work, we use the term "non-stationary knowledge'' to refer to information that was previously correct but is likely to change, and present "TGT-Masking'', a novel masking pattern to restrict PLMs from speculating knowledge of the kind. Finally, through a series of transfer learning with TGT-Masking applied we improve 22.63% of classification accuracy compared to standalone models on KorFinASC.
Fine-tuning of lightweight large language models for sentiment classification on heterogeneous financial textual data
International Business School, Zhejiang University（浙江大学国际商务学院）；Center for Data Science, Zhejiang University（浙江大学数据科学中心）；Centre for Statistics, Georg-August-Universität Göttingen（哥廷根大学统计中心）
NCL-BU at SemEval-2026 Task 3: Fine-tuning XLM-RoBERTa for Multilingual Dimensional Sentiment Regression
Independent Researcher（独立研究者）；Centre for Applied Creative Technologies (CFACT+), Bournemouth University, UK（应用创意技术中心（CFACT+），伯恩茅斯大学，英国）；School of Computing, Newcastle University, Newcastle upon Tyne, UK（计算学院，新castle大学，新castle upon Tyne，英国）
Can Large Language Models (or Humans) Disentangle Text?
Chalmers University of Technology（查尔姆斯理工大学）；Linköping University（林雪平大学）；The Center for Advanced Studies in Behavioral Sciences, Stanford University（斯坦福大学行为科学高级研究中心）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；University of Gothenburg（哥德堡大学）
Revolutionizing Finance with LLMs: An Overview of Applications and Insights
School of Computing, The University of Georgia（佐治亚大学计算机学院）；Computer Science and Engineering, The University of Texas at Arlington（德克萨斯大学阿灵顿分校计算机科学与工程系）；Department of Geography, University of Georgia（佐治亚大学地理系）
FinXABSA: Explainable Finance through Aspect-Based Sentiment Analysis
Asian Institute of Digital Finance（亚洲数字金融学院）；National University of Singapore（新加坡国立大学）；School of Computer Science and Engineering（计算机科学与工程学院）；Nanyang Technological University（南洋理工大学）；Institute of High Performance Computing (IHPC)（高性能计算研究所）；Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局）；College of Design & Engineering（设计与工程学院）
The topic of aspect-based sentiment analysis (ABSA) has been explored for a variety of industries, but it still remains much unexplored in finance. The recent release of data for an open challenge (FiQA) from the companion proceedings of WWW '18 has provided valuable finance-specific annotations. FiQA contains high quality labels, but it still lacks data quantity to apply traditional ABSA deep learning architecture. In this paper, we employ high-level semantic representations and methods of inductive transfer learning for NLP. We experiment with extensions of recently developed domain adaptati…
Hong Kong University of Science and Technology（香港科技大学）
Contextual pretrained language models, such as BERT (Devlin et al., 2019), have made significant breakthrough in various NLP tasks by training on large scale of unlabeled text re-sources.Financial sector also accumulates large amount of financial communication text.However, there is no pretrained finance specific language models available. In this work,we address the need by pretraining a financial domain specific BERT models, FinBERT, using a large scale of financial communication corpora. Experiments on three financial sentiment classification tasks confirm the advantage of FinBERT over gene…
Exploring Large Language Models for Financial Applications: Techniques, Performance, and Challenges with FinMA
Lyman Briggs College, Michigan State University（密歇根州立大学Lyman Briggs学院）；Department of Finance, Michigan State University（密歇根州立大学金融系）；African Institute for Mathematical Sciences, Rwanda（刚果(金)数学科学研究所）
Sentiment Analysis in Finance: From Transformers Back to eXplainable Lexicons (XLex)
Department of Computer Science, Metropolitan College, Boston University, Boston, MA 02215, USA（波士顿大学计算机科学系）；Faculty of Computer Science（计算机科学系）；Methodius University, 1000 Skopje, North Macedonia（Methodius大学）
Multi-domain sentiment classification deals with the scenario where labeled data exists for multiple domains but insufficient for training effective sentiment classifiers that work across domains. Thus, fully exploiting sentiment knowledge shared across domains is crucial for real world applications. While many existing works try to extract domain-invariant features in high-dimensional space, such models fail to explicitly distinguish between shared and private features at text-level, which to some extent lacks interpretablity. Based on the assumption that removing domain-related tokens from t…
In this paper, we design an integrated algorithm to evaluate the sentiment of Chinese market. Firstly, with the help of the web browser automation, we crawl a lot of news and comments from several influential financial websites automatically. Secondly, we use techniques of Natural Language Processing(NLP) under Chinese context, including tokenization, Word2vec word embedding and semantic database WordNet, to compute Senti-scores of these news and comments, and then construct the sentimental factor. Here, we build a finance-specific sentimental lexicon so that the sentimental factor can reflect…
In recent years, pretrained language models have revolutionized the NLP world, while achieving state of the art performance in various downstream tasks. However, in many cases, these models do not perform well when labeled data is scarce and the model is expected to perform in the zero or few shot setting. Recently, several works have shown that continual pretraining or performing a second phase of pretraining (inter-training) which is better aligned with the downstream task, can lead to improved results, especially in the scarce data setting. Here, we propose to leverage sentiment-carrying di…
Barcelona Supercomputing Center（巴塞罗那超级计算中心）；LHF Labs（LHF实验室）；Artificial Intelligence Finance Institute（人工智能金融研究所）；NYU Courant（纽约大学库朗数学科学研究所）
We introduce a new language representation model in finance called Financial Embedding Analysis of Sentiment (FinEAS). In financial markets, news and investor sentiment are significant drivers of security prices. Thus, leveraging the capabilities of modern NLP approaches for financial sentiment analysis is a crucial component in identifying patterns and trends that are useful for market participants and regulators. In recent years, methods that use transfer learning from large Transformer-based language models like BERT, have achieved state-of-the-art results in text classification tasks, incl…
Controlling Large Language Models Through Concept Activation Vectors
Key Lab of Intelligent Information Processing of Chinese Academy of Sciences（中国科学院智能信息处理重点实验室）；Key Lab of AI Safety of Chinese Academy of Sciences（中国科学院人工智能安全重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）
As natural language processing methods are increasingly deployed in real-world scenarios such as healthcare, legal systems, and social science, it becomes necessary to recognize the role they potentially play in shaping social biases and stereotypes. Previous work has revealed the presence of social biases in widely used word embeddings involving gender, race, religion, and other social constructs. While some methods were proposed to debias these word-level embeddings, there is a need to perform debiasing at the sentence-level given the recent shift towards new contextualized sentence represen…
Allen Institute for Artificial Intelligence（艾伦人工智能研究所）；University of California, Irvine（加州大学欧文分校）；University of Washington（华盛顿大学）
Much recent work in NLP has documented dataset artifacts, bias, and spurious correlations between input features and output labels. However, how to tell which features have "spurious" instead of legitimate correlations is typically left unspecified. In this work we argue that for complex language understanding tasks, all simple feature correlations are spurious, and we formalize this notion into a class of problems which we call competency problems. For example, the word "amazing" on its own should not give information about a sentiment label independent of the context in which it appears, whi…
Universal Adversarial Suffixes for Language Models Using Reinforcement Learning with Calibrated Reward
Center for Intelligent Cyber Physical Systems, Indian Institute of Technology Guwahati, India（智能网络物理系统中心，印度理工学院古瓦哈提）；Department of Computer Science and Engineering, Indian Institute of Technology Guwahati, India（计算机科学与工程系，印度理工学院古瓦哈提）
