An ultimate goal of artificial intelligence is to build computer systems that can understand human languages. Understanding commonsense knowledge about the world expressed in text is one of the foundational and challenging problems to create such intelligent systems. As a step towards this goal, we present in this paper ALMEn, an Autoencoding Language Model based Ensemble learning method for commonsense validation and explanation. By ensembling several advanced pre-trained language models including RoBERTa, DeBERTa, and ELECTRA with Siamese neural networks, our method can distinguish natural language statements that are against commonsense (validation subtask) and correctly identify the reason for making against commonsense (explanation selection subtask). Experimental results on the benchmark dataset of SemEval-2020 Task 4 show that our method outperforms state-of-the-art models, reaching 97.9% and 95.4% accuracies on the validation and explanation selection subtasks, respectively.
Probing Ethical Framework Representations in Large Language Models: Structure, Entanglement, and Methodological Challenges
School of Computer and Communication Sciences, École Polytechnique Fédérale de Lausanne（瑞士联邦理工学院计算机与通信科学学院）；Digital Humanities Lab, École Polytechnique Fédérale de Lausanne（瑞士联邦理工学院数字人文实验室）
SemEval-2020 Task 4: Commonsense Validation and Explanation
Zhejiang University（浙江大学）；Sun Yat-Sen University（中山大学）；School of Engineering, Westlake University（西湖大学工学院）；Singapore University of Technology and Design（新加坡科技设计大学）
Large Language Models are Better Reasoners with Self-Verification
The Laboratory of Cognition and Decision Intelligence for Complex Systems, IA, CAS（中国科学院自动化研究所复杂系统认知与决策智能实验室）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；College of Electrical and Information Engineering, Hunan University（湖南大学电气与信息工程学院）；Unisound（云知声）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
CUHK at SemEval-2020 Task 4: CommonSense Explanation, Reasoning and Prediction with Multi-task Learning
The Chinese University of Hong Kong（香港中文大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；South China Normal University（华南师范大学）
本文描述了我们的系统提交给SemEval 2020的任务4：常识验证和解释（ComVE），其中包括三个子任务。基于BERT架构的多任务环境，提出了一个有效的、可解释的“解释、推理和预测”（Explain，Reason and Predict，ERP）系统来解决与常识相关的三个子任务：（a）验证、（b）推理和（c）解释。受常识认知研究的启发，我们的系统首先生成句子的原因或理解，然后选择哪一个陈述是有意义的，这是通过多任务学习实现的。在后评估中，我们的系统在子任务A中达到了92.9%的准确率（排名11），在子任务B中达到了89.7%的准确率（排名9），在子任务C中达到了12.9的BLEU得分（排名8）。
DoTA: Weight-Decomposed Tensor Adaptation for Large Language Models
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；University of International Business and Economics, China（对外经济贸易大学）；XiaoMi AI Lab, China（小米AI实验室）
