Research on interference has provided evidence that the formation of dependencies between non-adjacent words relies on a cue-based retrieval mechanism. Two different models can account for one of the main predictions of interference, i.e., a slowdown at a retrieval site, when several items share a feature associated with a retrieval cue: Lewis and Vasishth's (2005) activation-based model and McElree's (2000) direct access model. Even though these two models have been used almost interchangeably, they are based on different assumptions and predict differences in the relationship between reading times and response accuracy. The activation-based model follows the assumptions of ACT-R, and its retrieval process behaves as a lognormal race between accumulators of evidence with a single variance. Under this model, accuracy of the retrieval is determined by the winner of the race and retrieval time by its rate of accumulation. In contrast, the direct access model assumes a model of memory where only the probability of retrieval varies between items; in this model, differences in latencies are a by-product of the possibility and repairing incorrect retrievals. We implemented both models in a Bayesian hierarchical framework in order to evaluate them and compare them. We show that some aspects of the data are better fit under the direct access model than under the activation-based model. We suggest that this finding does not rule out the possibility that retrieval may be behaving as a race model with assumptions that follow less closely the ones from the ACT-R framework. We show that by introducing a modification of the activation model, i.e, by assuming that the accumulation of evidence for retrieval of incorrect items is not only slower but noisier (i.e., different variances for the correct and incorrect items), the model can provide a fit as good as the one of the direct access model.
Modelling dependency completion in sentence comprehension as a Bayesian hierarchical mixture process: A case study involving Chinese relative clauses
University of Potsdam(波茨坦大学);École Nationale de la Statistique et de l’administration économique(国家统计与经济管理学院);CNRS(法国国家科学研究中心);Université Paris-Dauphine(巴黎九大);PSL Research University(巴黎文理研究大学)
Feature overwriting as a finite mixture process: Evidence from comprehension data
University of Potsdam(波茨坦大学);Centre de Recherche en Mathématiques de la Décision(决策数学研究中心);CNRS(法国国家科学研究中心);Université Paris-Dauphine(巴黎第九大学);PSL Research University(巴黎文理研究大学)
众所周知,不合语法的句子“橱柜的钥匙在桌子上”会导致一种语法上的错觉。正如Jaeger等人在荟萃分析中所讨论的,2017年,在上面的协议吸引句子中,与同样不合语法的句子“The key to the cabinet are on the table”相比,观察到动词are的阅读时间更快。对这种促进效应的一种解释是特征渗透:橱柜上的复数特征延伸到中心名词键,导致错觉。另一种解释是基于线索的检索(Lewis & Vasishth,2005),它假设非主语名词柜被误检索,由于部分特征匹配时,在辅助的依赖完成过程中启动了一个记忆访问的主题与复数标记。我们提出的证据又一个解释所观察到的便利。因为第二个句子中有两个名词的数量相同,所以在一定程度上,这两个名词可能更难区分,导致阅读上面第一个句子中的动词的时间变慢;这是Nairne,1990的特征性解释。我们表明,特征提取的建议可以实现为一个有限的混合过程。我们重新分析了10个已发表的数据集,拟合分层贝叶斯混合模型,这些数据假设两个混合物分布。我们发现,在十项研究中,有九项研究中,与特征渗透和基于线索的检索帐户相对应的混合分布比特征渗透和基于线索的检索帐户都更适合。
我们的目标是跨语言机器阅读理解(MRC)的任务,在直接zero-shot设置,通过合并的句法特征,从普遍的相似性(UD),我们使用的关键特征是每个句子内的句法关系。虽然以前的工作已经证明了有效的语法引导的MRC模型,我们建议采用句间的句法关系,除了基本的句内关系,进一步利用在多句输入的MRC任务的句法依赖。在我们的方法中,我们建立了句子间依赖图(ISDG)连接依赖树,以形成跨句子的全局句法关系。然后,我们提出了ISDG编码器,编码的全球依赖图,通过一跳和多跳依赖路径明确地解决句子间的关系。在三个多语言MRC数据集(XQuAD,MLQA,TyDiQA-GoldP)上的实验表明,我们的编码器只在英语上训练,能够提高所有14个测试集的zero-shot性能,覆盖8种语言,平均提高高达3.8 F1 / 5.2 EM,某些语言的5.2 F1 / 11.2 EM。进一步的分析表明,这种改进可以归因于对跨语言一致性句法路径的关注。
A Sentence Cloze Dataset for Chinese Machine Reading Comprehension
Research Center for SCIR, Harbin Institute of Technology(哈尔滨工业大学SCIR研究中心);State Key Laboratory of Cognitive Intelligence, iFLYTEK Research, China(科大讯飞研究中心认知智能国家重点实验室);iFLYTEK AI Research (Hebei), Langfang, China(科大讯飞(河北)人工智能研究院,廊坊)
Accounting for Agreement Phenomena in Sentence Comprehension with Transformer Language Models: Effects of Similarity-based Interference on Surprisal and Attention
Question Answering from Unstructured Text by Retrieval and Comprehension
Sony Corporation(索尼公司);Carnegie Mellon University(卡内基梅隆大学)
开放域问答(QA)系统必须与外部知识源(例如网页)交互以查找相关信息。然而,像维基百科这样的信息源与知识库(KBs)相比结构不好并且难以利用。在这项工作中,我们提出了一个两步的方法来从非结构化文本的问答,包括检索步骤和理解步骤。对于理解,我们提出了一个基于RNN的注意力模型,该模型具有一种新的混合机制,用于从检索到的文章或固定词汇中选择答案。为了检索,我们引入了一个手工制作的模型和一个神经模型来对相关文章进行排名。我们在W IKI M OVIES数据集上实现了最先进的性能,将误差降低了40%。我们的实验结果进一步证明了每个引入的组件的重要性。
Step out of KG: Knowledge Graph Completion via Knowledgeable Retrieval and Reading Comprehension
Tsinghua University(清华大学);Renmin University of China(中国人民大学);BNRist(北京国家信息科学与技术研究中心);Institute for Artificial Intelligence, Tsinghua University(清华大学人工智能研究院)
Underutilization of Syntactic Processing by Chinese Learners of English in Comprehending English Sentences, Evidenced from Adapted Garden-Path Ambiguity Experiment
When the LM misunderstood the human chuckled: Analyzing garden path effects in humans and language models
Blavatnik School of Computer Science, Tel Aviv University(特拉维夫大学布拉瓦尼克计算机科学学院);Department of Linguistics, Tel Aviv University(特拉维夫大学语言学系);Sagol School of Neuroscience, Tel Aviv University(特拉维夫大学萨戈尔神经科学学院)
Measuring Meaning Composition in the Human Brain with Composition Scores from Large Language Models
National Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室);Department of Linguistics and Translation, City University of Hong Kong(香港城市大学语言及翻译学系)
Comparing Human and Language Models Sentence Processing Difficulties on Complex Structures
Blavatnik School of Computer Science, Tel Aviv University, Israel(巴尔-艾塔夫大学计算机科学学院);Department of Linguistics, Tel Aviv University, Israel(巴尔-艾塔夫大学语言学系);Sagol School of Neuroscience, Tel Aviv University, Israel(巴尔-艾塔夫大学神经科学学院)
ParetoRAG: Leveraging Sentence-Context Attention for Robust and Efficient Retrieval-Augmented Generation
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cybersecurity, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院);Alibaba Group(阿里巴巴集团)