Many recent studies have shown that for models trained on datasets for natural language inference (NLI), it is possible to make correct predictions by merely looking at the hypothesis while completely ignoring the premise. In this work, we manage to derive adversarial examples in terms of the hypothesis-only bias and explore eligible ways to mitigate such bias. Specifically, we extract various phrases from the hypotheses (artificial patterns) in the training sets, and show that they have been strong indicators to the specific labels. We then figure out `hard' and `easy' instances from the original test sets whose labels are opposite to or consistent with those indications. We also set up baselines including both pretrained models (BERT, RoBERTa, XLNet) and competitive non-pretrained models (InferSent, DAM, ESIM). Apart from the benchmark and baselines, we also investigate two debiasing approaches which exploit the artificial pattern modeling to mitigate such hypothesis-only bias: down-sampling and adversarial training. We believe those methods can be treated as competitive baselines in NLI debiasing tasks.
Institute of Computing Technology（计算技术研究所）；Chinese Academy of Science（中国科学院）；Peking University（北京大学）；Fudan University（复旦大学）
In this work, we propose a generalized likelihood ratio method capable of training the artificial neural networks with some biological brain-like mechanisms,.e.g., (a) learning by the loss value, (b) learning via neurons with discontinuous activation and loss functions. The traditional back propagation method cannot train the artificial neural networks with aforementioned brain-like learning mechanisms. Numerical results show that the robustness of various artificial neural networks trained by the new method is significantly improved when the input data is affected by both the natural noises a…
Natural Language Inference (NLI) or Recognizing Textual Entailment (RTE) is the task of predicting the entailment relation between a pair of sentences (premise and hypothesis). This task has been described as a valuable testing ground for the development of semantic representations, and is a key component in natural language understanding evaluation benchmarks. Models that understand entailment should encode both, the premise and the hypothesis. However, experiments by Poliak et al. revealed a strong preference of these models towards patterns observed only in the hypothesis, based on a 10 dat…
From Aleatoric to Epistemic: Exploring Uncertainty Quantification Techniques in Artificial Intelligence
University of Liverpool（利物浦大学）；University of Edinburgh（爱丁堡大学）；The University of Texas at Dallas（德州大学达拉斯分校）；Georgia Institute of Technology（佐治亚理工学院）；Emory University（埃默里大学）；Simon Fraser University（西蒙菲莎大学）；Kyoto University（京都大学）；Zhejiang University（浙江大学）；National Taiwan Normal University（国立台湾师范大学）；Indiana University（印第安纳大学）；Purdue University（普渡大学）；AppCubic（AppCubic公司）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；The Hong Kong University of Science and Technology（香港科技大学）
Case Western Reserve University（凯斯西储大学）；East Carolina University（东卡罗来纳大学）
The emergence of artificial intelligence has incited a paradigm shift across the spectrum of human endeavors, with ChatGPT serving as a catalyst for the transformation of various established domains, including but not limited to education, journalism, security, and ethics. In the post-pandemic era, the widespread adoption of remote work has prompted the educational sector to reassess conventional pedagogical methods. This paper is to scrutinize the underlying psychological principles of ChatGPT, delve into the factors that captivate user attention, and implicate its ramifications on the future…
Exploring Major Transitions in the Evolution of Biological Cognition With Artificial Neural Networks
Institute for Human-Centered AI, Helmholtz Zentrum Munich（以人为中心的人工智能研究所，慕尼黑亥姆霍兹研究中心）；Leverhulme Centre for the Future of Intelligence, University of Cambridge（未来智能研究中心，剑桥大学）；School of Natural Sciences, Macquarie University（自然科学学院，麦考瑞大学）；School of Philosophy, The Australian National University（哲学学院，澳大利亚国立大学）；Department of History and Philosophy of Science, University of Cambridge（历史与哲学科学系，剑桥大学）；School of Environmental and Life Sciences, University of Hull（环境与生命科学学院，霍尔姆大学）；Centre for Data Science, AI, and Modelling, University of Hull（数据科学、人工智能与建模中心，霍尔姆大学）
MOOSE-Chem2: Exploring LLM Limits in Fine-Grained Scientific Hypothesis Discovery via Hierarchical Search
Nanyang Technological University（南洋理工大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；University of Science and Technology of China（中国科学技术大学）；Wuhan University（武汉大学）；National University of Singapore（新加坡国立大学）；University of New South Wales（新南威尔士大学）；MiroMind；The Chinese University of Hong Kong（香港中文大学）
University of Illinois at Urbana-Champaign, USA（伊利诺伊大学厄巴纳-香槟分校，美国）；University of California at San Diego, USA（加利福尼亚大学圣地亚哥分校，美国）；IBM-Illinois Center for Cognitive Computing Systems Research (C3SR), USA（IBM-伊利诺伊认知计算系统研究中心（C3SR），美国）；IBM Thomas J. Watson Research Center, USA（IBM托马斯·J·沃森研究中心，美国）
We introduce and study semantic capacity of terms. For example, the semantic capacity of artificial intelligence is higher than that of linear regression since artificial intelligence possesses a broader meaning scope. Understanding semantic capacity of terms will help many downstream tasks in natural language processing. For this purpose, we propose a two-step model to investigate semantic capacity of terms, which takes a large text corpus as input and can evaluate semantic capacity of terms if the text corpus can provide enough co-occurrence information of terms. Extensive experiments in thr…
The Software Naturalness hypothesis argues that programming languages can be understood through the same techniques used in natural language processing. We explore this hypothesis through the use of a pre-trained transformer-based language model to perform code analysis tasks. Present approaches to code analysis depend heavily on features derived from the Abstract Syntax Tree (AST) while our transformer-based language models work on raw source code. This work is the first to investigate whether such language models can discover AST features automatically. To achieve this, we introduce a sequen…
Amazon Alexa AI（亚马逊Alexa人工智能部门）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）
Voice Assistants such as Alexa, Siri, and Google Assistant typically use a two-stage Spoken Language Understanding pipeline; first, an Automatic Speech Recognition (ASR) component to process customer speech and generate text transcriptions, followed by a Natural Language Understanding (NLU) component to map transcriptions to an actionable hypothesis. An end-to-end (E2E) system that goes directly from speech to a hypothesis is a more attractive option. These systems were shown to be smaller, faster, and better optimized. However, they require massive amounts of end-to-end training data and in a…
Nagoya University of Business and Commerce（名古屋商科大学）
This study explores an interesting pattern emerging from research that combines artificial intelligence with sound symbolism. In these studies, supervised machine learning algorithms are trained to classify samples based on the sounds of referent names. Machine learning algorithms are efficient learners of sound symbolism, but they tend to bias one category over the other. The pattern is this: when a category arguably represents greater threat, the algorithms tend to overpredict to that category. A hypothesis, framed by error management theory, is presented that proposes that this may be evide…
Exploring Perspectives on the Impact of Artificial Intelligence on the Creativity of Knowledge Work: Beyond Mechanised Plagiarism and Stochastic Parrots
Microsoft Research（微软研究院）；University of Cambridge（剑桥大学）；University College London（伦敦大学学院）
Since the advent of Large Language Models (LLMs), efforts have largely focused on improving their instruction-following and deductive reasoning abilities, leaving open the question of whether these models can truly discover new knowledge. In pursuit of artificial general intelligence (AGI), there is a growing need for models that not only execute commands or retrieve information but also learn, reason, and generate new knowledge by formulating novel hypotheses and theories that deepen our understanding of the world. Guided by Peirce's framework of abduction, deduction, and induction, this surv…
Charles University（查理大学）；Faculty of Mathematics and Physics（数学与物理学院）；Institute of Formal and Applied Linguistics（形式与应用语言学研究所）
Artificial neural networks are a state-of-the-art solution for many problems in natural language processing. What can we learn about language and meaning from the way artificial neural networks represent it? Word representations obtained from the Skip-gram variant of the word2vec model exhibit interesting semantic properties. This is usually explained by referring to the general distributional hypothesis, which states that the meaning of the word is given by the contexts where it occurs. We propose a more specific approach based on Frege's holistic and functional approach to meaning. Taking Tu…
