CommentsAccepted (main track) at NeurIPS 2025. 44 pages, 17 figures, 22 tables. Published in Advances in Neural Information Processing Systems, vol. 38
Journal refAdvances in Neural Information Processing Systems, vol. 38, pp. 88811-88861, 2025
RoPE Distinguishes Neither Positions Nor Tokens in Long Contexts, Provably
RoPE在长上下文中无法区分位置或令牌,证明性分析
Yufeng Du, Phillip Harris, Minyang Tian, Eliu A Huerta, Srikanth Ronanki, Subendhu Rongali, Aram Galstyan, Hao Peng
机构
*
University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
;
University of Bonn(波恩大学)
;
Argonne National Laboratory(阿贡国家实验室)
;
Amazon AGI(亚马逊人工智能研究院)
CommentsAccepted for a presentation at ICML 2024 and an oral presentation at NeurIPS 2023 Workshop on Causal Representation Learning. Code is available at https://github.com/KihoPark/linear_rep_geometry
Journal refIn Proceedings of the 41st International Conference on Machine Learning (ICML), 2024
TILT: Target-induced loss tilting under covariate shift
TILT:在协变量偏移下由目标诱导的损失倾斜
Kakei Yamamoto, Martin J. Wainwright
机构
*
Lab for Information and Decision Systems(信息与决策系统实验室)
;
Statistics and Data Science Center(统计与数据科学中心)
;
EECS, Massachusetts Institute of Technology(麻省理工学院电子工程与计算机科学系)
;
Mathematics and EECS, Massachusetts Institute of Technology(数学与电子工程与计算机科学系, 麻省理工学院)
Jan Arne Telle, Brigt Håvardstun, Jose Hernandez-Orallo
机构
*
Department of Informatics University of Bergen(卑尔根大学信息学院)
;
University of Bergen(卑尔根大学)
;
VRAIN - Universitat Politecnica de Valencia(瓦伦西亚理工大学VRAIN实验室)
;
Universitat Politecnica de Valencia(瓦伦西亚理工大学)
;
Leverhulme Centre for the Future of Intelligence - University of Cambridge(剑桥大学未来智能中心)
LLMs as Implicit Imputers: Uncertainty Should Scale with Missing Information
大型语言模型作为隐式填补器:不确定性应与缺失信息成比例
Stef van Buuren
机构
*
TNO - Netherlands Organization for Applied Scientific Research(荷兰应用科学研究院)
;
Dept. of Methodology and Statistics, University of Utrecht(乌得勒支大学方法学与统计学系)
Implicit Behavioral Decoding from Next-Step Spike Forecasts at Population Scale
从群体尺度上的下一步尖峰预测隐式行为解码
John R. Minnick, Jesus Gonzalez-Ferrer, Kamran Hussain, Jinghui Geng, Ash Robbins, Mohammed A. Mostajo-Radji, David Haussler, Jason Eshraghian, Mircea Teodorescu
机构
*
Department of Electrical and Computer Engineering, University of California, Santa Cruz, CA, USA(加州大学圣克ruz分校电气与计算机工程系)
;
UC Santa Cruz Genomics Institute, University of California, Santa Cruz, CA, USA(加州大学圣克ruz分校基因组研究所)
;
Department of Biomolecular Engineering, University of California, Santa Cruz, CA, USA(加州大学圣克ruz分校生物分子工程系)
;
Department of Applied Mathematics, University of California, Santa Cruz, CA, USA(加州大学圣克ruz分校应用数学系)
;
Department of Computer Science and Engineering, University of California, Santa Cruz, CA, USA(加州大学圣克ruz分校计算机科学与工程系)
SpikeProphecy: A Large-Scale Benchmark for Autoregressive Neural Population Forecasting
SpikeProphecy:大规模用于自回归神经群体预测的基准测试
John R. Minnick, Jinghui Geng, Kamran Hussain, Jesus Gonzalez-Ferrer, Ash Robbins, Mohammed A. Mostajo-Radji, David Haussler, Jason K. Eshraghian, Mircea Teodorescu
机构
*
Department of Electrical and Computer Engineering, University of California, Santa Cruz, CA, USA(加州大学圣克ruz分校电气与计算机工程系)
;
UC Santa Cruz Genomics Institute, University of California, Santa Cruz, CA, USA(加州大学圣克ruz分校基因组研究所)
;
Department of Computer Science and Engineering, University of California, Santa Cruz, CA, USA(加州大学圣克ruz分校计算机科学与工程系)
;
Department of Applied Mathematics, University of California, Santa Cruz, CA, USA(加州大学圣克ruz分校应用数学系)
;
Department of Biomolecular Engineering, University of California, Santa Cruz, CA, USA(加州大学圣克ruz分校生物分子工程系)