"You are grounded!": Latent Name Artifacts in Pre-trained Language Models
Allen Institute for Artificial Intelligence（艾伦人工智能研究所）；Paul G. Allen School of Computer Science & Engineering, University of Washington（华盛顿大学保罗·G·艾伦计算机科学与工程学院）；University of Maryland, College Park（马里兰大学学院市分校）
预训练的语言模型（LM）可能会将源自其训练语料库的偏见延续到下游模型。我们专注于与给定名称的表示相关的工件（例如，Donald），其取决于语料库，可以与特定实体相关联，如由下一个令牌预测（例如，特朗普）。虽然在某些情况下是有帮助的，但接地也发生在不明确或不适当的情况下。例如，“Donald is a”的结尾与其他名字的结尾有很大不同，并且往往具有高于平均水平的负面情绪。我们展示了潜在的影响下游任务与阅读理解探针名称扰动改变模型的答案。作为一线希望，我们的实验表明，在不同语料库上进行额外的预训练可能会减轻这种偏见。
英文摘要：
Pre-trained language models (LMs) may perpetuate biases originating in their training corpus to downstream models. We focus on artifacts associated with the representation of given names (e.g., Donald), which, depending on the corpus, may be associated with specific entities, as indicated by next token prediction (e.g., Trump). While helpful in some contexts, grounding happens also in under-specified or inappropriate contexts. For example, endings generated for `Donald is a' substantially differ from those of other names, and often have more-than-average negative sentiment. We demonstrate the potential effect on downstream tasks with reading comprehension probes where name perturbation changes the model answers. As a silver lining, our experiments suggest that additional pre-training on different corpora may mitigate this bias.
Beyond NL2Code: A Structured Survey of Multimodal Code Intelligence
Meituan（美团）；The University of Hong Kong（香港大学）；The Chinese University of Hong Kong（香港中文大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Nanjing University（南京大学）；Harbin Institute of Technology（哈尔滨工业大学）；Australian Institute for Machine Learning, Adelaide University（阿德莱德大学澳大利亚机器学习研究所）；Ludwig Maximilian University of Munich（慕尼黑大学）；University of Science and Technology of China（中国科学技术大学）；Queen Mary University of London（伦敦玛丽女王大学）
AgentSnare: Learning to Delay, Divert, and Defuse Autonomous Web Penetration Agents
Zhejiang University（浙江大学）；Tsinghua University（清华大学）；The University of Hong Kong（香港大学）；Hainan University（海南大学）；Anhui University（安徽大学）；The Hong Kong Polytechnic University（香港理工大学）
ZARA: Training-Free Motion Time-Series Reasoning via Evidence-Grounded LLM Agents
University of New South Wales, Sydney（新南威尔士大学悉尼校区）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Hong Kong University of Science and Technology（香港科技大学）
Learning the Loop, Not Just the Page: Execution-Grounded Loop Learning for Web Generation
Tsinghua University（清华大学）；East China Normal University（华东师范大学）；Tongji University（同济大学）；Institute of Artificial Intelligence, Beihang University（北京航空航天大学人工智能研究院）；National University of Singapore（新加坡国立大学）
Semantic categories of artifacts and animals reflect efficient coding
Edmond and Lily Safra Center for Brain Sciences, Hebrew University（希伯来大学埃德蒙与莉莉·萨夫拉脑科学中心）；Department of Linguistics, University of California, Berkeley（加州大学伯克利分校语言学系）；Cognitive Science Program, University of California, Berkeley（加州大学伯克利分校认知科学项目）；Benin School of Computer Science and Engineering, Hebrew University（希伯来大学贝宁计算机科学与工程学院）；School of Psychological Sciences, University of Melbourne（墨尔本大学心理科学学院）
