Rep2Text: Decoding Full Text from a Single LLM Token Representation
New Jersey Institute of Technology(新泽西理工学院);National University of Singapore(国立新加坡大学);Cisco Research(思科研究);Wake Forest University(威克森林大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳))
Haiyan Zhao, Zirui He, Yiming Tang, Fan Yang, Ali Payani, Dianbo Liu, Mengnan Du
Large language models represent text as hidden numerical patterns, but what information these representations actually contain remains a mystery. Can we recover the original text from just one of them? In this paper, we introduce Rep2Text, a method designed to answer this question by reconstructing text from a single hidden representation. We find that these representations preserve much more information than we might expect. For short texts of about 16 words, Rep2Text can recover roughly half of the original words while preserving their overall meaning. As the text gets longer, recovering the exact wording becomes harder, but the main ideas can still be recovered. These findings offer a new way to understand what language models store internally and highlight potential privacy risks, as hidden representations may reveal parts of the original input.
Gumiho: A Hybrid Architecture to Prioritize Early Tokens in Speculative Decoding
Advanced Micro Devices, Inc., Beijing, China(Advanced Micro Devices公司,北京);Department of Electrical and Electronic Engineering, The University of Hong Kong(香港大学电子与电气工程系);Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University(西安交通大学人工智能与机器人研究所)
SimSD: Simple Speculative Decoding in Diffusion Language Models
University of California San Diego(加州大学圣地亚哥分校);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Google(谷歌);University of California Santa Barbara(加州大学圣芭芭拉分校)
TETRIS: Optimal Draft Token Selection for Batch Speculative Decoding
Singapore-MIT Alliance for Research and Technology(新加坡-麻省理工联合研究技术联盟);Dept. of Computer Science, National University of Singapore(新加坡国立大学计算机科学系);CSAIL, Massachusetts Institute of Technology(麻省理工学院计算机科学与人工智能实验室)
HiViS: Hiding Visual Tokens from the Drafter for Speculative Decoding in Vision-Language Models
C 2 DL, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);AIRIA;City University of Hong Kong(香港城市大学)
Stability-Weighted Decoding for Diffusion Language Models
Department of XXX, University of YYY, Location, Country(XXX系,YYY大学,Location,Country);School of ZZZ, Institute of WWW, Location, Country(ZZZ学院,WWW研究所,Location,Country);Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University, Hong Kong, China(数据科学与人工智能系,香港理工大学,香港,中国)
Beyond Tokens: Semantic-Aware Speculative Decoding for Efficient Inference by Probing Internal States
Centre for Software Excellence, Huawei, Canada(华为加拿大软件 excellence 中心);Department of Computer Science, University of Manitoba, Canada(曼尼托巴大学计算机科学系);School of Computing, Queen’s University, Canada(皇后大学计算学院)
SpecVLM: Enhancing Speculative Decoding of Video LLMs via Verifier-Guided Token Pruning
The State Key Laboratory of Blockchain and Data Security(区块链与数据安全国家重点实验室);Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security(杭州高新技术区(滨江)区块链与数据安全研究院);Department of Computing, The Hong Kong Polytechnic University(香港理工大学计算机系);School of Computer Science, Beijing University of Posts and Telecommunications(北京邮电大学计算机学院)
Beyond Truncation: Rethinking LLM Decoding as Ensemble Pruning
Institute of Statistics and Big Data, Renmin University of China(中国人民大学统计与大数据研究院);Big Data and Responsible Artificial Intelligence for National Governance, Renmin University of China(中国人民大学大数据与国家治理研究院);Center for Applied Statistics, Institute of Statistics and Big Data, Renmin University of China(中国人民大学统计与大数据研究院应用统计中心)