A basic problem in machine learning is to find a mapping $f$ from a low dimensional latent space $\mathcal{Y}$ to a high dimensional observation space $\mathcal{X}$. Modern tools such as deep neural networks are capable to represent general non-linear mappings. A learner can easily find a mapping which perfectly fits all the observations. However, such a mapping is often not considered as good, because it is not simple enough and can overfit. How to define simplicity? We try to make a formal definition on the amount of information imposed by a non-linear mapping $f$. Intuitively, we measure the local discrepancy between the pullback geometry and the intrinsic geometry of the latent space. Our definition is based on information geometry and is independent of the empirical observations, nor specific parameterizations. We prove its basic properties and discuss relationships with related machine learning methods.
Universal Preprocessing Operators for Embedding Knowledge Graphs with Literals
Peoples' Friendship University of Russia (RUDN University)(俄罗斯人民友谊大学(卢蒙巴大学));Joint Institute for Nuclear Research(联合核研究所);Vrije Universiteit Amsterdam(阿姆斯特丹自由大学);University of Skövde(舍夫德大学);University of Mannheim(曼海姆大学)
Jailbreaking LLMs' Safeguard with Universal Magic Words for Text Embedding Models
Dept. of Comp. Sci. and Tech., Inst. for AI, Tsinghua-Bosch Joint ML Center, THBI Lab, BNRist Center, Tsinghua University, Beijing, 100084, China(计算机科学与技术系、人工智能研究所、清华-博世联合机器学习中心、THBI实验室、BNRist中心、清华大学、北京);Department of Mathematical Sciences, Tsinghua University(数学科学系、清华大学);BIMSA, Beijing, China(BIMSA、北京、中国)
A Time-Series Foundation Model by Universal Delay Embedding
Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences(中国科学院大学杭州高等研究院);Shanghai Jiao Tong University(上海交通大学);Fudan University(复旦大学);Ocean University of China(中国海洋大学);South China University of Technology(华南理工大学)
Generalizing Knowledge Graph Embedding with Universal Orthogonal Parameterization
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);Key Laboratory of Trustworthy Distributed Computing and Service (MOE), Beijing University of Posts and Telecommunications(北京邮电大学);School of Computer Science and Technology, Dalian University of Technology(大连理工大学计算机科学与技术学院)
HomID : Benchmarking Intrinsic Dimension Estimators on Homogenous Manifolds with Anisotropic Embeddings
Joint Center for Quantum Information and Computer Science (QuICS), NIST & UMD College Park(联合量子信息与计算机科学中心(QuICS)、国家标准与技术研究院(NIST)及马里兰大学 College Park 分校);Joint Quantum Institute (JQI), NIST & UMD College Park(联合量子研究所(JQI)、国家标准与技术研究院(NIST)及马里兰大学 College Park 分校)
The Shape of Learning: Anisotropy and Intrinsic Dimensions in Transformer-Based Models
Skoltech(斯科尔科沃科技学院);AIRI(人工智能研究所);Lomonosov Moscow State University(罗蒙诺索夫莫斯科国立大学);HSE University(高等经济大学);SberAI(萨伯人工智能公司);Samara National Research University(萨马拉国立研究大学)