有效分析评论以揭示潜在意图在各个领域制定战略决策方面具有巨大价值。然而，一些挑战阻碍了情感分析过程，包括评论中表现出的词汇多样性、文本中长依赖的存在、遇到未知符号和单词，以及处理不平衡的数据集。此外，现有的情感分析任务主要利用序列模型来编码长依赖文本，并且由于按顺序处理文本，需要更长的执行时间。相比之下，Transformer由于其并行处理特性，需要较少的执行时间。在这项工作中，我们引入了一种新颖的混合深度学习模型RoBERTa-BiLSTM，它将鲁棒优化的BERT预训练方法（RoBERTa）与双向长短期记忆（BiLSTM）网络相结合。RoBERTa用于生成有意义的词嵌入向量，而BiLSTM有效地捕获长依赖文本的上下文语义。RoBERTa-BiLSTM混合模型利用序列模型和Transformer模型的优势，以增强情感分析的性能。我们使用来自IMDb、Twitter US Airline和Sentiment140的数据集进行了实验，以评估所提出的模型与现有最先进方法的对比。我们的实验结果表明，RoBERTa-BiLSTM模型超越了基线模型（例如BERT、RoBERTa-base、RoBERTa-GRU和RoBERTa-LSTM），在Twitter US Airline、IMDb和Sentiment140数据集上分别实现了80.74%、92.36%和82.25%的准确率。此外，该模型在相同数据集上分别实现了80.73%、92.35%和82.25%的F1分数。
英文摘要：
Effectively analyzing the comments to uncover latent intentions holds immense value in making strategic decisions across various domains. However, several challenges hinder the process of sentiment analysis including the lexical diversity exhibited in comments, the presence of long dependencies within the text, encountering unknown symbols and words, and dealing with imbalanced datasets. Moreover, existing sentiment analysis tasks mostly leveraged sequential models to encode the long dependent texts and it requires longer execution time as it processes the text sequentially. In contrast, the Transformer requires less execution time due to its parallel processing nature. In this work, we introduce a novel hybrid deep learning model, RoBERTa-BiLSTM, which combines the Robustly Optimized BERT Pretraining Approach (RoBERTa) with Bidirectional Long Short-Term Memory (BiLSTM) networks. RoBERTa is utilized to generate meaningful word embedding vectors, while BiLSTM effectively captures the contextual semantics of long-dependent texts. The RoBERTa-BiLSTM hybrid model leverages the strengths of both sequential and Transformer models to enhance performance in sentiment analysis. We conducted experiments using datasets from IMDb, Twitter US Airline, and Sentiment140 to evaluate the proposed model against existing state-of-the-art methods. Our experimental findings demonstrate that the RoBERTa-BiLSTM model surpasses baseline models (e.g., BERT, RoBERTa-base, RoBERTa-GRU, and RoBERTa-LSTM), achieving accuracies of 80.74%, 92.36%, and 82.25% on the Twitter US Airline, IMDb, and Sentiment140 datasets, respectively. Additionally, the model achieves F1-scores of 80.73%, 92.35%, and 82.25% on the same datasets, respectively.
A Unified BERT-CNN-BiLSTM Framework for Simultaneous Headline Classification and Sentiment Analysis of Bangla News
Department of CCE, International Islamic University Chittagong（国际伊斯兰大学丘拉图加冈分校计算机工程系）；Department of ICE, Noakhali Science and Technology University（诺阿克利科学与技术大学信息工程系）；Department of CSE, Banladesh University of Engineering and Technology（孟加拉工程与技术大学计算机科学与工程系）；Dept. of CSE, IIUC, Chattogram, Bangladesh（国际伊斯兰大学丘拉图加冈分校计算机科学与工程系）；School of CSE, University of Southern Mississippi（密西西比大学南方分校计算机科学学院）；Department of CSE, Netrokona University（纳特罗克纳大学计算机科学与工程系）；mPower Social Enterprise（mPower社会企业）
Aspect-based Sentiment Analysis (ABSA), aiming at predicting the polarities for aspects, is a fine-grained task in the field of sentiment analysis. Previous work showed syntactic information, e.g. dependency trees, can effectively improve the ABSA performance. Recently, pre-trained models (PTMs) also have shown their effectiveness on ABSA. Therefore, the question naturally arises whether PTMs contain sufficient syntactic information for ABSA so that we can obtain a good ABSA model only based on PTMs. In this paper, we firstly compare the induced trees from PTMs and the dependency parsing trees…
University of Information Technology（信息科技大学）；Vietnam National University（越南国立大学）
Aspect-based sentiment analysis plays an essential role in natural language processing and artificial intelligence. Recently, researchers only focused on aspect detection and sentiment classification but ignoring the sub-task of detecting user opinion span, which has enormous potential in practical applications. In this paper, we present a new Vietnamese dataset (UIT-ViSD4SA) consisting of 35,396 human-annotated spans on 11,122 feedback comments for evaluating the span detection in aspect-based sentiment analysis. Besides, we also propose a novel system using Bidirectional Long Short-Term Memo…
In this paper, we conduct a sentence level sentiment analysis on the product reviews from Amazon and thorough analysis on the model interpretability. For the sentiment analysis task, we use the BiLSTM model with attention mechanism. For the study of interpretability, we consider the attention weights distribution of single sentence and the attention weights of main aspect terms. The model has an accuracy of up to 0.96. And we find that the aspect terms have the same or even more attention weights than the sentimental words in sentences.
Collaborative Filtering(CF) recommender is a crucial application in the online market and ecommerce. However, CF recommender has been proven to suffer from persistent problems related to sparsity of the user rating that will further lead to a cold-start issue. Existing methods address the data sparsity issue by applying token-level sentiment analysis that translate text review into sentiment scores as a complement of the user rating. In this paper, we attempt to optimize the sentiment analysis with advanced NLP models including BERT and RoBERTa, and experiment on whether the CF recommender has…
Arabic Sentiment Analysis with Noisy Deep Explainable Model
Virginia Tech（弗吉尼亚理工大学）；Fraunhofer Institute for Applied Information Technology FIT（弗劳恩霍夫应用信息技术研究所）；University of Siegen（锡根大学）；UITS（达卡大学信息技术学院）；Bonn-Rhein-Sieg University of Applied Sciences（波恩-莱茵-锡根应用科学大学）
Bauman Moscow State Technical University（鲍曼莫斯科国立技术大学）；Lomonosov Moscow State University（罗蒙诺索夫莫斯科国立大学）
In this study, we test standard neural network architectures (CNN, LSTM, BiLSTM) and recently appeared BERT architectures on previous Russian sentiment evaluation datasets. We compare two variants of Russian BERT and show that for all sentiment tasks in this study the conversational variant of Russian BERT performs better. The best results were achieved by BERT-NLI model, which treats sentiment classification tasks as a natural language inference task. On one of the datasets, this model practically achieves the human level.
Chittagong University of Engineering and Technology（吉大港工程技术大学）
The amount of textual data generation has increased enormously due to the effortless access of the Internet and the evolution of various web 2.0 applications. These textual data productions resulted because of the people express their opinion, emotion or sentiment about any product or service in the form of tweets, Facebook post or status, blog write up, and reviews. Sentiment analysis deals with the process of computationally identifying and categorizing opinions expressed in a piece of text, especially in order to determine whether the writer's attitude toward a particular topic is positive,…
In this paper, we describe our system submitted for SemEval 2020 Task 9, Sentiment Analysis for Code-Mixed Social Media Text alongside other experiments. Our best performing system is a Transfer Learning-based model that fine-tunes "XLM-RoBERTa", a transformer-based multilingual masked language model, on monolingual English and Spanish data and Spanish-English code-mixed data. Our system outperforms the official task baseline by achieving a 70.1% average F1-Score on the official leaderboard using the test set. For later submissions, our system manages to achieve a 75.9% average F1-Score on the…
Twitter customer service interactions have recently emerged as an effective platform to respond and engage with customers. In this work, we explore the role of negation in customer service interactions, particularly applied to sentiment analysis. We define rules to identify true negation cues and scope more suited to conversational data than existing general review data. Using semantic knowledge and syntactic structure from constituency parse trees, we propose an algorithm for scope detection that performs comparable to state of the art BiLSTM. We further investigate the results of negation sc…
This paper studies the efficiency of transferring BERT learnings to low complexity models like BiLSTM, BiLSTM with attention and shallow CNNs using sentiment analysis on SST-2 dataset. It also compares the complexity of inference of the BERT model with these lower complexity models and underlines the importance of these techniques in enabling high performance NLP models on edge devices like mobiles, tablets and MCU development boards like Raspberry Pi etc. and enabling exciting new applications.
Sentiment Analysis typically refers to using natural language processing, text analysis and computational linguistics to extract affect and emotion based information from text data. Our work explores how we can effectively use deep neural networks in transfer learning and joint dual input learning settings to effectively classify sentiments and detect hate speech in Hindi and Bengali data. We start by training Word2Vec word embeddings for Hindi \textbf{HASOC dataset} and Bengali hate speech and then train LSTM and subsequently, employ parameter sharing based transfer learning to Bengali sentim…
Prior work on sentiment analysis using weak supervision primarily focuses on different reviews such as movies (IMDB), restaurants (Yelp), products (Amazon).~One under-explored field in this regard is customer chat data for a customer-agent chat in customer support due to the lack of availability of free public data. Here, we perform sentiment analysis on customer chat using weak supervision on our in-house dataset. We fine-tune the pre-trained language model (LM) RoBERTa as a sentiment classifier using weak supervision. Our contribution is as follows:1) We show that by using weak sentiment cla…
Sentiment analysis for code-mixed social media text continues to be an under-explored area. This work adds two common approaches: fine-tuning large transformer models and sample efficient methods like ULMFiT. Prior work demonstrates the efficacy of classical ML methods for polarity detection. Fine-tuned general-purpose language representation models, such as those of the BERT family are benchmarked along with classical machine learning and ensemble methods. We show that NB-SVM beats RoBERTa by 6.2% (relative) F1. The best performing model is a majority-vote ensemble which achieves an F1 of 0.7…
Aspect-based sentiment analysis (ABSA) aims at analyzing the sentiment of a given aspect in a sentence. Recently, neural network-based methods have achieved promising results in existing ABSA datasets. However, these datasets tend to degenerate to sentence-level sentiment analysis because most sentences contain only one aspect or multiple aspects with the same sentiment polarity. To facilitate the research of ABSA, NLPCC 2020 Shared Task 2 releases a new large-scale Multi-Aspect Multi-Sentiment (MAMS) dataset. In the MAMS dataset, each sentence contains at least two different aspects with diff…
University of Colorado Boulder（科罗拉多大学博尔德分校）；University of Colorado Denver（科罗拉多大学丹佛分校）
Public acceptance and support for renewable energy are important determinants of renewable energy policies and market conditions. This paper examines public sentiment toward solar energy in the United States using data from Twitter, a micro-blogging platform in which people post messages, known as tweets. We filtered tweets specific to solar energy and performed a classification task using Robustly optimized Bidirectional Encoder Representations from Transformers (RoBERTa). Analyzing 71,262 tweets during the period of late January to early July 2020, we find public sentiment varies significant…
Advancing Sentiment Analysis in Tamil-English Code-Mixed Texts: Challenges and Transformer-Based Solutions
Instituto Politécnico Nacional (IPN)（墨西哥国立理工学院）；Centro de Investigación en Computación (CIC)（计算机研究中心）；Escuela Superior de Ingeniería Mecánica y Eléctrica (ESIME)（机械与电气工程高等学院）
Wuhan University（武汉大学）；JD Explore Academy（京东探索研究院）；Kunming University of Science and Technology（昆明理工大学）
Aspect-based sentiment analysis (ABSA) is a fine-grained task of sentiment analysis. To better comprehend long complicated sentences and obtain accurate aspect-specific information, linguistic and commonsense knowledge are generally required in this task. However, most current methods employ complicated and inefficient approaches to incorporate external knowledge, e.g., directly searching the graph nodes. Additionally, the complementarity between external knowledge and linguistic information has not been thoroughly studied. To this end, we propose a knowledge graph augmented network KGAN, whic…
The invention of transformer-based models such as BERT, GPT, and RoBERTa has enabled researchers and financial companies to finetune these powerful models and use them in different downstream tasks to achieve state-of-the-art performance. Recently, a lightweight alternative (approximately 0.1% - 3% of the original model parameters) to fine-tuning, known as prefix tuning has been introduced. This method freezes the model parameters and only updates the prefix to achieve performance comparable to full fine-tuning. Prefix tuning enables researchers and financial practitioners to achieve similar r…
Newsletters and social networks can reflect the opinion about the market and specific stocks from the perspective of analysts and the general public on products and/or services provided by a company. Therefore, sentiment analysis of these texts can provide useful information to help investors trade in the market. In this paper, a hierarchical stack of Transformers model is proposed to identify the sentiment associated with companies and stocks, by predicting a score (of data type real) in a range between -1 and +1. Specifically, we fine-tuned a RoBERTa model to process headlines and microblogs…
Tsinghua University（清华大学）；Beijing Forestry University（北京林业大学）；Beijing University of Posts and Telecommunications（北京邮电大学）
Audio Sentiment Analysis is a popular research area which extends the conventional text-based sentiment analysis to depend on the effectiveness of acoustic features extracted from speech. However, current progress on audio sentiment analysis mainly focuses on extracting homogeneous acoustic features or doesn't fuse heterogeneous features effectively. In this paper, we propose an utterance-based deep neural network model, which has a parallel combination of Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) based network, to obtain representative features termed Audio Sentimen…
A hybrid transformer and attention based recurrent neural network for robust and interpretable sentiment analysis of tweets
Khulna University of Engineering & Technology (KUET)（库尔纳工程技术大学）；Advanced Machine Intelligence Research (AMIR) Lab（高级机器智能研究实验室）；American International University-Bangladesh（孟加拉国美国国际大学）；Chowagiken Corp.（Chowagiken公司）；University of Asia Pacific（亚太大学）；University of Aizu（会津大学）
Detecting Abnormal User Feedback Patterns through Temporal Sentiment Aggregation
Khoury College of Computer Science Northeastern University Boston, United States；College of Engineering Northeastern University Boston, United States；College of Professional Studies Northeastern University Boston, United States
Covid-19 has spread across the world and several vaccines have been developed to counter its surge. To identify the correct sentiments associated with the vaccines from social media posts, we fine-tune various state-of-the-art pre-trained transformer models on tweets associated with Covid-19 vaccines. Specifically, we use the recently introduced state-of-the-art pre-trained transformer models RoBERTa, XLNet and BERT, and the domain-specific transformer models CT-BERT and BERTweet that are pre-trained on Covid-19 tweets. We further explore the option of text augmentation by oversampling using L…
HausaMovieReview: A Benchmark Dataset for Sentiment Analysis in Low-Resource African Language
Federal University Dutsin-Ma（杜钦马联邦大学）；Aliko Dangote University of Science and Technology（阿利科·丹格特科技大学）；Northwest University（西北大学）；Bayero University Kano（卡诺巴耶罗大学）
Constructing Colloquial Dataset for Persian Sentiment Analysis of Social Microblogs
Faculty member in ICT Research Institute（ICT研究所教员）；Iran Telecommunication Research Center (ITRC)（伊朗电信研究中心）；Faculty member in Computer Department（计算机系教员）；Mehralborz University（梅赫拉布尔兹大学）；Hazrat-e Masoumeh University（玛苏姆大学）
Expanding Vietnamese SentiWordNet to Improve Performance of Vietnamese Sentiment Analysis Models
University of Engineering and Technology, Vietnam National University（越南国家大学工程技术大学）；University of Economic and Technical Industries（经济技术工业大学）；Hanoi University of Public Health（河内公共卫生大学）
Labelling a large quantity of social media data for the task of supervised machine learning is not only time-consuming but also difficult and expensive. On the other hand, the accuracy of supervised machine learning models is strongly related to the quality of the labelled data on which they train, and automatic sentiment labelling techniques could reduce the time and cost of human labelling. We have compared three automatic sentiment labelling techniques: TextBlob, Vader, and Afinn to assign sentiments to tweets without any human assistance. We compare three scenarios: one uses training and t…
