The internet today has become an unrivalled source of information where people converse on content based websites such as Quora, Reddit, StackOverflow and Twitter asking doubts and sharing knowledge with the world. A major arising problem with such websites is the proliferation of toxic comments or instances of insincerity wherein the users instead of maintaining a sincere motive indulge in spreading toxic and divisive content. The straightforward course of action in confronting this situation is detecting such content beforehand and preventing it from subsisting online. In recent times Transfer Learning in Natural Language Processing has seen an unprecedented growth. Today with the existence of transformers and various state of the art innovations, a tremendous growth has been made in various NLP domains. The introduction of BERT has caused quite a stir in the NLP community. As mentioned, when published, BERT dominated performance benchmarks and thereby inspired many other authors to experiment with it and publish similar models. This led to the development of a whole BERT-family, each member being specialized on a different task. In this paper we solve the Insincere Questions Classification problem by fine tuning four cutting age models viz BERT, RoBERTa, DistilBERT and ALBERT.
PRHLT research center, Universitat Politècnica de València（瓦伦西亚理工大学PRHLT研究中心）；Faculty of Mathematics and Computer Science, University of Bucharest（布加勒斯特大学数学与计算机科学学院）；CIIRC, Czech Technical University（捷克技术大学捷克信息学、机器人与控制论研究所）
The high prevalence of depression in society has given rise to the need for new digital tools to assist in its early detection. To this end, existing research has mainly focused on detecting depression in the domain of social media, where there is a sufficient amount of data. However, with the rise of conversational agents like Siri or Alexa, the conversational domain is becoming more critical. Unfortunately, there is a lack of data in the conversational domain. We perform a study focusing on domain adaptation from social media to the conversational domain. Our approach mainly exploits the lin…
Birmingham City University（伯明翰城市大学）；University of Bristol（布里斯托大学）
In 2016, United Kingdom (UK) citizens voted to leave the European Union (EU), which was officially implemented in 2020. During this period, UK residents experienced a great deal of uncertainty around the UK's continued relationship with the EU. Many people have used social media platforms to express their emotions about this critical event. Sentiment analysis has been recently considered as an important tool for detecting mental well-being in Twitter contents. However, detecting the psychological distress status in political-related tweets is a challenging task due to the lack of explicit sent…
Dedan Kimathi University of Technology（德丹·基马蒂理工大学）
Chest X-rays remains to be the most common imaging modality used to diagnose lung diseases. However, they necessitate the interpretation of experts (radiologists and pulmonologists), who are few. This review paper investigates the use of deep transfer learning techniques to detect COVID-19, pneumonia, and tuberculosis in chest X-ray (CXR) images. It provides an overview of current state-of-the-art CXR image classification techniques and discusses the challenges and opportunities in applying transfer learning to this domain. The paper provides a thorough examination of recent research studies t…
To provide privacy-aware software systems, it is crucial to consider privacy from the very beginning of the development. However, developers do not have the expertise and the knowledge required to embed the legal and social requirements for data protection into software systems. Objective: We present an approach to decrease privacy risks during agile software development by automatically detecting privacy-related information in the context of user story requirements, a prominent notation in agile Requirement Engineering (RE). Methods: The proposed approach combines Natural Language Processing …
University of Georgia（佐治亚大学）；Florida International University（佛罗里达国际大学）
DRDr II is a hybrid of machine learning and deep learning worlds. It builds on the successes of its antecedent, namely, DRDr, that was trained to detect, locate, and create segmentation masks for two types of lesions (exudates and microaneurysms) that can be found in the eyes of the Diabetic Retinopathy (DR) patients; and uses the entire model as a solid feature extractor in the core of its pipeline to detect the severity level of the DR cases. We employ a big dataset with over 35 thousand fundus images collected from around the globe and after 2 phases of preprocessing alongside feature extra…
Zhejiang University of Technology（浙江工业大学）；Nanyang Technological University（南洋理工大学）；Zhejiang University（浙江大学）
Most traditional false data injection attack (FDIA) detection approaches rely on a key assumption, i.e., the power system can be accurately modeled. However, the transmission line parameters are dynamic and cannot be accurately known during operation and thus the involved modeling errors should not be neglected. In this paper, an illustrative case has revealed that modeling errors in transmission lines significantly weaken the detection effectiveness of conventional FDIA approaches. To tackle this issue, we propose an FDIA detection mechanism from the perspective of transfer learning. Specific…
New Knowledge；Algorine, Inc.（Algorine公司）；Binghamton University (SUNY)（纽约州立大学宾汉姆顿分校）
We apply an ensemble pipeline composed of a character-level convolutional neural network (CNN) and a long short-term memory (LSTM) as a general tool for addressing a range of disinformation problems. We also demonstrate the ability to use this architecture to transfer knowledge from labeled data in one domain to related (supervised and unsupervised) tasks. Character-level neural networks and transfer learning are particularly valuable tools in the disinformation space because of the messy nature of social media, lack of labeled data, and the multi-channel tactics of influence campaigns. We dem…
Insight Centre for Data Analytics Dublin City University（都柏林城市大学数据洞察中心）；Dublin City University（都柏林城市大学）
This work presents a comparative study of existing and new techniques to detect knee injuries by leveraging Stanford's MRNet Dataset. All approaches are based on deep learning and we explore the comparative performances of transfer learning and a deep residual network trained from scratch. We also exploit some characteristics of Magnetic Resonance Imaging (MRI) data by, for example, using a fixed number of slices or 2D images from each of the axial, coronal and sagittal planes as well as combining the three planes into one multi-plane network. Overall we achieved a performance of 93.4% AUC on …
Xidian University（西安电子科技大学）；School of Artificial Intelligence（人工智能学院）
We can consider Counterfactuals as belonging in the domain of Discourse structure and semantics, A core area in Natural Language Understanding and in this paper, we introduce an approach to resolving counterfactual detection as well as the indexing of the antecedents and consequents of Counterfactual statements. While Transfer learning is already being applied to several NLP tasks, It has the characteristics to excel in a novel number of tasks. We show that detecting Counterfactuals is a straightforward Binary Classification Task that can be implemented with minimal adaptation on already exist…
It is commonly believed that in transfer learning including more pre-training data translates into better performance. However, recent evidence suggests that removing data from the source dataset can actually help too. In this work, we take a closer look at the role of the source dataset's composition in transfer learning and present a framework for probing its impact on downstream performance. Our framework gives rise to new capabilities such as pinpointing transfer learning brittleness as well as detecting pathologies such as data-leakage and the presence of misleading examples in the source…
Microscopic examination of tissues or histopathology is one of the diagnostic procedures for detecting colorectal cancer. The pathologist involved in such an examination usually identifies tissue type based on texture analysis, especially focusing on tumour-stroma ratio. In this work, we automate the task of tissue classification within colorectal cancer histology samples using deep transfer learning. We use discriminative fine-tuning with one-cycle-policy and apply structure-preserving colour normalization to boost our results. We also provide visual explanations of the deep neural network's …
University of Saskatchewan（萨斯喀彻温大学）；International Road Dynamics（国际道路动力学公司）；National Institute of Technology, Trichy（特里奇国家理工学院）
One of the primary clinical observations for screening the infectious by the novel coronavirus is capturing a chest x-ray image. In most of the patients, a chest x-ray contains abnormalities, such as consolidation, which are the results of COVID-19 viral pneumonia. In this study, research is conducted on efficiently detecting imaging features of this type of pneumonia using deep convolutional neural networks in a large dataset. It is demonstrated that simple models, alongside the majority of pretrained networks in the literature, focus on irrelevant features for decision-making. In this paper,…
Efficient Breast and Ovarian Cancer Classification via ViT-Based Preprocessing and Transfer Learning
Department of Mathematics, University of Texas at Arlington（德克萨斯大学阿灵顿分校数学系）；Department of Data Science and Mathematics, Embry-Riddle Aeronautical University（埃姆伯里-瑞德航空大学数据科学与数学系）
Deep Neural Networks - especially Convolutional Neural Network (ConvNet) has become the state-of-the-art for image classification, pattern recognition and various computer vision tasks. ConvNet has a huge potential in medical domain for analyzing medical data to diagnose diseases in an efficient way. Based on extracted features by ConvNet model from MRI data, early diagnosis is very crucial for preventing progress and treating the Alzheimer's disease. Despite having the ability to deliver great performance, absence of interpretability of the model's decision can lead to misdiagnosis which can …
This paper proposes a methodological approach with a transfer learning scheme for plastic waste bottle detection and instance segmentation using the \textit{mask region proposal convolutional neural network} (Mask R-CNN). Plastic bottles constitute one of the major pollutants posing a serious threat to the environment both in oceans and on land. The automated identification and segregation of bottles can facilitate plastic waste recycling. We prepare a custom-made dataset of 192 bottle images with pixel-by pixel-polygon annotation for the automatic segmentation task. The proposed transfer lear…
University of Toronto（多伦多大学）；Vector Institute（矢量研究所）；Academia Sinica（中央研究院）；Toronto Rehabilitation Institute（多伦多康复研究所）
Machine learning has shown promise for automatic detection of Alzheimer's disease (AD) through speech; however, efforts are hampered by a scarcity of data, especially in languages other than English. We propose a method to learn a correspondence between independently engineered lexicosyntactic features in two languages, using a large parallel corpus of out-of-domain movie dialogue data. We apply it to dementia detection in Mandarin Chinese, and demonstrate that our method outperforms both unilingual and machine translation-based baselines. This appears to be the first study that transfers feat…
Classification is an essential and fundamental task in machine learning, playing a cardinal role in the field of natural language processing (NLP) and computer vision (CV). In a supervised learning setting, labels are always needed for the classification task. Especially for deep neural models, a large amount of high-quality labeled data are required for training. However, when a new domain comes out, it is usually hard or expensive to acquire the labels. Transfer learning could be an option to transfer the knowledge from a source domain to a target domain. A challenge is that these two domain…
Tsinghua-Berkeley Shenzhen Institute（清华-伯克利深圳学院）；University of California, Berkeley（加州大学伯克利分校）；Affiliated Eye Hospital of Nanchang University（南昌大学附属眼科医院）
Retinal vascular diseases affect the well-being of human body and sometimes provide vital signs of otherwise undetected bodily damage. Recently, deep learning techniques have been successfully applied for detection of diabetic retinopathy (DR). The main obstacle of applying deep learning techniques to detect most other retinal vascular diseases is the limited amount of data available. In this paper, we propose a transfer learning technique that aims to utilize the feature similarities for detecting retinal vascular diseases. We choose the well-studied DR detection as a source task and identify…
Convolution Neural Network (ConvNet) offers a high potential to generalize input data. It has been widely used in many application areas, such as visual imagery, where comprehensive learning datasets are available and a ConvNet model can be well trained and perform the required function effectively. ConvNet can also be applied to network intrusion detection. However, the currently available datasets related to the network intrusion are often inadequate, which makes the ConvNet learning deficient, hence the trained model is not competent in detecting unknown intrusions. In this paper, we propos…
Polytechnique Montréal（蒙特利尔理工学院）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；Quebec Artificial Intelligence Institute (Mila)（魁北克人工智能研究所（Mila））
Providing better language tools for low-resource and endangered languages is imperative for equitable growth. Recent progress with massively multilingual pretrained models has proven surprisingly effective at performing zero-shot transfer to a wide variety of languages. However, this transfer is not universal, with many languages not currently understood by multilingual approaches. It is estimated that only 72 languages possess a "small set of labeled datasets" on which we could test a model's performance, the vast majority of languages not having the resources available to simply evaluate per…
Detecting Text Formality: A Study of Text Classification Approaches
Technical University of Munich（慕尼黑工业大学）；Centro Singular de Investigación en Tecnoloxías Intelixentes (CiTIUS)（智能技术单一研究中心）；Universidade de Santiago de Compostela（圣地亚哥德孔波斯特拉大学）；Skolkovo Institute of Science and Technology（斯科尔科沃科技学院）；Artificial Intelligence Research Institute（人工智能研究所）
Ferhat Abbas University, Setif 1（费尔哈特·阿巴斯大学塞提夫第一大学）；Qatar Computing Research Institute（卡塔尔计算研究所）；Hamad Bin Khalifa University (HBKU)（哈马德·本·哈利法大学）
During the last two decades, we have progressively turned to the Internet and social media to find news, entertain conversations and share opinion. Recently, OpenAI has developed a ma-chine learning system called GPT-2 for Generative Pre-trained Transformer-2, which can pro-duce deepfake texts. It can generate blocks of text based on brief writing prompts that look like they were written by humans, facilitating the spread false or auto-generated text. In line with this progress, and in order to counteract potential dangers, several methods have been pro-posed for detecting text written by thes…
International Institute of Information Technology Hyderabad（海得拉巴国际信息技术学院）
Memes are one of the most popular types of content used to spread information online. They can influence a large number of people through rhetorical and psychological techniques. The task, Detection of Persuasion Techniques in Texts and Images, is to detect these persuasive techniques in memes. It consists of three subtasks: (A) Multi-label classification using textual content, (B) Multi-label classification and span identification using textual content, and (C) Multi-label classification using visual and textual content. In this paper, we propose a transfer learning approach to fine-tune BERT…
Hybrid Machine Learning Model for Detecting Bangla Smishing Text Using BERT and Character-Level CNN
Department of Computer Science and Engineering（计算机科学与工程系）；World University of Bangladesh（孟加拉世界大学）；Department of Electrical and Electronics Engineering（电气与电子工程系）；Jashore University of Science and Technology（贾沙尔大学科学与技术学院）；Southern University Bangladesh（孟加拉南部大学）；Institute of Information Technology（信息技术研究所）；Noakhali Science and Technology University（诺阿基利科学与技术大学）
