The rapidly growing ecosystem of Large Language Models (LLMs) makes it increasingly challenging to manage and utilize the vast and dynamic pool of models effectively. We propose LOCUS, a method that produces low-dimensional vector embeddings that compactly represent a language model's capabilities across queries. LOCUS is an attention-based approach that generates embeddings by a deterministic forward pass over query encodings and evaluation scores via an encoder model, enabling seamless incorporation of new models to the pool and refinement of existing model embeddings without having to perform any retraining. We additionally train a correctness predictor that uses model embeddings and query encodings to achieve state-of-the-art routing accuracy on unseen queries. Experiments show that LOCUS needs up to 4.8x fewer query evaluation samples than baselines to produce informative and robust embeddings. Moreover, the learned embedding space is geometrically meaningful: proximity reflects model similarity, enabling a range of downstream applications including model comparison and clustering, model portfolio selection, and resilient proxies of unavailable models.
VERSE: Visual Embedding Reduction and Space Exploration. Clustering-Guided Insights for Training Data Enhancement in Visually-Rich Document Understanding
Institute for Research in Technology, ICAI School of Engineering, Comillas Pontifical University（技术研究所、ICAI工程学院、科利马斯天主教大学）
Answer Probing-Guided Search for Diverse Solution Exploration of LLMs
University of Science and Technology of China（中国科学技术大学）；Zhongguancun Academy（中关村学院）；Alibaba Group（阿里巴巴集团）；Shanghai Jiao Tong University（上海交通大学）；Tsinghua University（清华大学）
SeLaR: Selective Latent Reasoning in Large Language Models
Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology（广东省超高清沉浸式媒体技术重点实验室）；Shenzhen Graduate School, Peking University（北京大学深圳研究生院）
CHARME: A chain-based reinforcement learning approach for the minor embedding problem
Department of Computer and Information Science and Engineering, University of Florida（计算机与信息科学与工程系，佛罗里达大学）；Theoritical Division, Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室理论部）
Cluster Exploration using Informative Manifold Projections
Department of Computer Science, University of Liverpool, Liverpool, UK（利物浦大学计算机科学系）；Department of Chemistry, University of Liverpool, Liverpool, UK（利物浦大学化学系）；Leverhulme Research Centre for Functional Materials Design, University of Liverpool, Liverpool, UK（利物浦大学功能性材料设计勒弗赫姆研究中心）
