首页/论文/2006.09161

arXiv 2006.091612020-06cs.CLcs.AIcs.LG

CUHK at SemEval-2020 Task 4: CommonSense Explanation, Reasoning and Prediction with Multi-task Learning

The Chinese University of Hong Kong（香港中文大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；South China Normal University（华南师范大学）

Hongru Wang, Xiangru Tang, Sunny Lai, Kwong Sak Leung, Jia Zhu, Gabriel Pui Cheong Fung, Kam-Fai Wong

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

This paper describes our system submitted to task 4 of SemEval 2020: Commonsense Validation and Explanation (ComVE) which consists of three sub-tasks. The task is to directly validate the given sentence whether or not it makes sense and require the model to explain it. Based on BERTarchitecture with a multi-task setting, we propose an effective and interpretable "Explain, Reason and Predict" (ERP) system to solve the three sub-tasks about commonsense: (a) Validation, (b)Reasoning, and (c) Explanation. Inspired by cognitive studies of common sense, our system first generates a reason or understanding of the sentences and then chooses which one statement makes sense, which is achieved by multi-task learning. During the post-evaluation, our system has reached 92.9% accuracy in subtask A (rank 11), 89.7% accuracy in subtask B (rank 9), andBLEU score of 12.9 in subtask C (rank 8)

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

QiaoNing at SemEval-2020 Task 4: Commonsense Validation and Explanation system based on ensemble of language model

Northeastern University（东北大学）

In this paper, we present language model system submitted to SemEval-2020 Task 4 competition: "Commonsense Validation and Explanation". We participate in two subtasks for subtask A: validation and subtask B: Explanation. We implemented with transfer learning using pretrained language models (BERT, XLNet, RoBERTa, and ALBERT) and fine-tune them on this task. Then we compared their characteristics in this task to help future researchers understand and use these models more properly. The ensembled model better solves this problem, making the model's accuracy reached 95.9% on subtask A, which just…

2020-09-06cs.AIcs.CL

Yuanfudao at SemEval-2018 Task 11: Three-way Attention and Relational Knowledge for Commonsense Machine Comprehension

Yuanfudao Research（猿辅导研究部）

This paper describes our system for SemEval-2018 Task 11: Machine Comprehension using Commonsense Knowledge. We use Three-way Attentive Networks (TriAN) to model interactions between the passage, question and answers. To incorporate commonsense knowledge, we augment the input with relation embedding from the graph of general knowledge ConceptNet (Speer et al., 2017). As a result, our system achieves state-of-the-art performance with 83.95% accuracy on the official test data. Code is publicly available at https://github.com/intfloat/commonsense-rc

2018-05-15cs.CL

LMVE at SemEval-2020 Task 4: Commonsense Validation and Explanation using Pretraining Language Model

Northeastern University（东北大学）

This paper describes our submission to subtask a and b of SemEval-2020 Task 4. For subtask a, we use a ALBERT based model with improved input form to pick out the common sense statement from two statement candidates. For subtask b, we use a multiple choice model enhanced by hint sentence mechanism to select the reason from given options about why a statement is against common sense. Besides, we propose a novel transfer learning strategy between subtasks which help improve the performance. The accuracy scores of our system are 95.6 / 94.9 on official test set and rank 7$^{th}$ / 2$^{nd}$ on Pos…

2020-07-06cs.AIcs.CL

基于大语言模型的零样本常识验证与推理：在SemEval-2020 Task 4数据集上的评估

Zero-Shot Commonsense Validation and Reasoning with Large Language Models: An Evaluation on SemEval-2020 Task 4 Dataset

College of Information Technology（信息技术学院）；Lusail University（卢赛尔大学）；Qatar University（卡塔尔大学）

本研究在SemEval-2020 Task 4数据集上零样本评估多款大语言模型的常识验证与解释能力，发现大模型常识验证接近人类水平，但常识解释能力仍有不足，暴露了因果与推断推理的局限。

2025-02-19cs.AIcs.CL

SemEval-2020 Task 4: Commonsense Validation and Explanation

Zhejiang University（浙江大学）；Sun Yat-Sen University（中山大学）；School of Engineering, Westlake University（西湖大学工学院）；Singapore University of Technology and Design（新加坡科技设计大学）

In this paper, we present SemEval-2020 Task 4, Commonsense Validation and Explanation (ComVE), which includes three subtasks, aiming to evaluate whether a system can distinguish a natural language statement that makes sense to humans from one that does not, and provide the reasons. Specifically, in our first subtask, the participating systems are required to choose from two natural language statements of similar wording the one that makes sense and the one does not. The second subtask additionally asks a system to select the key reason from three options why a given statement does not make sen…

2020-08-03cs.AIcs.CL

ECNU-SenseMaker at SemEval-2020 Task 4: Leveraging Heterogeneous Knowledge Resources for Commonsense Validation and Explanation

East China Normal University（华东师范大学）

This paper describes our system for SemEval-2020 Task 4: Commonsense Validation and Explanation (Wang et al., 2020). We propose a novel Knowledge-enhanced Graph Attention Network (KEGAT) architecture for this task, leveraging heterogeneous knowledge from both the structured knowledge base (i.e. ConceptNet) and unstructured text to better improve the ability of a machine in commonsense understanding. This model has a powerful commonsense inference capability via utilizing suitable commonsense incorporation methods and upgraded data augmentation techniques. Besides, an internal sharing mechanism…

2020-07-28cs.CL

更多相关论文收起更多论文

HFL-RC System at SemEval-2018 Task 11: Hybrid Multi-Aspects Model for Commonsense Reading Comprehension

Joint Laboratory of HIT and iFLYTEK (HFL), iFLYTEK Research（哈尔滨工业大学与科大讯飞联合实验室（HFL）、科大讯飞研究院）；Research Center for Social Computing and Information Retrieval (SCIR),Harbin Institute of Technology（哈尔滨工业大学社会计算与信息检索研究中心（SCIR））；Harbin Institute of Technology（哈尔滨工业大学）；iFLYTEK Research（科大讯飞研究院）

This paper describes the system which got the state-of-the-art results at SemEval-2018 Task 11: Machine Comprehension using Commonsense Knowledge. In this paper, we present a neural network called Hybrid Multi-Aspects (HMA) model, which mimic the human's intuitions on dealing with the multiple-choice reading comprehension. In this model, we aim to produce the predictions in multiple aspects by calculating attention among the text, question and choices, and combine these results for final predictions. Experimental results show that our HMA model could give substantial improvements over the base…

2018-03-15cs.CL

ANA at SemEval-2020 Task 4: mUlti-task learNIng for cOmmonsense reasoNing (UNION)

Alberta Machine Intelligence Institute（阿尔伯塔机器智能研究所）；University of Alberta（阿尔伯塔大学）

In this paper, we describe our mUlti-task learNIng for cOmmonsense reasoNing (UNION) system submitted for Task C of the SemEval2020 Task 4, which is to generate a reason explaining why a given false statement is non-sensical. However, we found in the early experiments that simple adaptations such as fine-tuning GPT2 often yield dull and non-informative generations (e.g. simple negations). In order to generate more meaningful explanations, we propose UNION, a unified end-to-end framework, to utilize several existing commonsense datasets so that it allows a model to learn more dynamics under the…

2020-06-29cs.CL

KaLM at SemEval-2020 Task 4: Knowledge-aware Language Models for Comprehension And Generation

Sun Yat-Sen University（中山大学）

This paper presents our strategies in SemEval 2020 Task 4: Commonsense Validation and Explanation. We propose a novel way to search for evidence and choose the different large-scale pre-trained models as the backbone for three subtasks. The results show that our evidence-searching approach improves model performance on commonsense explanation task. Our team ranks 2nd in subtask C according to human evaluation score.

2020-07-24cs.CL

BUT-FIT at SemEval-2020 Task 4: Multilingual commonsense

Brno University of Technology（布尔诺理工大学）

This paper describes work of the BUT-FIT's team at SemEval 2020 Task 4 - Commonsense Validation and Explanation. We participated in all three subtasks. In subtasks A and B, our submissions are based on pretrained language representation models (namely ALBERT) and data augmentation. We experimented with solving the task for another language, Czech, by means of multilingual models and machine translated dataset, or translated model inputs. We show that with a strong machine translation system, our system can be used in another language with a small accuracy loss. In subtask C, our submission, wh…

2020-08-21cs.CL

CS-NLP team at SemEval-2020 Task 4: Evaluation of State-of-the-art NLP Deep Learning Architectures on Commonsense Reasoning Task

Western Michigan University（西密歇根大学）；Virginia Commonwealth University（弗吉尼亚联邦大学）

In this paper, we investigate a commonsense inference task that unifies natural language understanding and commonsense reasoning. We describe our attempt at SemEval-2020 Task 4 competition: Commonsense Validation and Explanation (ComVE) challenge. We discuss several state-of-the-art deep learning architectures for this challenge. Our system uses prepared labeled textual datasets that were manually curated for three different natural language inference subtasks. The goal of the first subtask is to test whether a model can distinguish between natural language statements that make sense and those…

2020-07-18cs.CL

SemEval-2024 Task 9: BRAINTEASER：一个挑战常识的新任务

SemEval-2024 Task 9: BRAINTEASER: A Novel Task Defying Common Sense

Information Sciences Institute, Viterbi School of Engineering, University of Southern California（南加州大学维特比工程学院信息科学研究所）；Department of Computer Science, Faculty of Science, Vrije Universiteit Amsterdam（阿姆斯特丹自由大学理学院计算机科学系）；Tencent AI Lab（腾讯AI实验室）

本文介绍了SemEval-2024 Task 9: BRAINTEASER，首个评估系统横向思维推理能力的任务，通过拆分基准支持微调并分析182名参与者的483份提交结果，探讨当前模型的横向思维能力并展望未来研究方向。

2024-04-22cs.AIcs.CLcs.LG

Autoencoding Language Model Based Ensemble Learning for Commonsense Validation and Explanation

Posts and Telecommunications Institute of Technology（邮电技术学院）

An ultimate goal of artificial intelligence is to build computer systems that can understand human languages. Understanding commonsense knowledge about the world expressed in text is one of the foundational and challenging problems to create such intelligent systems. As a step towards this goal, we present in this paper ALMEn, an Autoencoding Language Model based Ensemble learning method for commonsense validation and explanation. By ensembling several advanced pre-trained language models including RoBERTa, DeBERTa, and ELECTRA with Siamese neural networks, our method can distinguish natural l…

2022-04-07cs.AIcs.CL

IIE-NLP-NUT at SemEval-2020 Task 4: Guiding PLM with Prompt Template Reconstruction Strategy for ComVE

Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；University of Chinese Academy of Sciences（中国科学院大学）

This paper introduces our systems for the first two subtasks of SemEval Task4: Commonsense Validation and Explanation. To clarify the intention for judgment and inject contrastive information for selection, we propose the input reconstruction strategy with prompt templates. Specifically, we formalize the subtasks into the multiple-choice question answering format and construct the input with the prompt templates, then, the final prediction of question answering is considered as the result of subtasks. Experimental results show that our approaches achieve significant performance compared with t…

2020-07-02cs.CL

Incorporating Relation Knowledge into Commonsense Reading Comprehension with Multi-task Learning

Alibaba DAMO Academy（阿里巴巴达摩院）

This paper focuses on how to take advantage of external relational knowledge to improve machine reading comprehension (MRC) with multi-task learning. Most of the traditional methods in MRC assume that the knowledge used to get the correct answer generally exists in the given documents. However, in real-world task, part of knowledge may not be mentioned and machines should be equipped with the ability to leverage external knowledge. In this paper, we integrate relational knowledge into MRC model for commonsense reasoning. Specifically, based on a pre-trained language model (LM). We design two a…

2019-09-05cs.CL

通过幽默和谜语数据增强语言模型的横向思维以完成BRAINTEASER任务

Augmenting Lateral Thinking in Language Models with Humor and Riddle Data for the BRAINTEASER Task

Amazon Web Services（亚马逊网络服务）

通过引入幽默和谜语数据增强语言模型，提升其在BRAINTEASER任务中的横向推理能力，提高句子谜题和词语谜题的准确率。

2026-02-23cs.AIcs.CLcs.IR