Perplexity is a widely adopted metric for assessing the predictive quality of large language models (LLMs) and often serves as a reference metric for downstream evaluations. However, recent evidence shows that perplexity can be unreliable, especially when irrelevant long inputs are used, raising concerns for both benchmarking and system deployment. While prior efforts have employed selective input filtering and curated datasets, the impact of input length on perplexity has not been systematically studied from a systems perspective and input length has rarely been treated as a first-class system variable affecting both fairness and efficiency. In this work, we close this gap by introducing LengthBenchmark, a system-conscious evaluation framework that explicitly integrates input length, evaluation protocol design, and system-level costs, evaluating representative LLMs under two scoring protocols (direct accumulation and fixed window sliding) across varying context lengths. Unlike prior work that focuses solely on accuracy-oriented metrics, LengthBenchmark additionally measures latency, memory footprint, and evaluation cost, thereby linking predictive metrics to deployment realities. We further incorporate quantized variants not as a main contribution, but as robustness checks, showing that length-induced biases persist across both full-precision and compressed models. This design disentangles the effects of evaluation logic, quantization, and input length, and demonstrates that length bias is a general phenomenon that undermines fair cross-model comparison. Our analysis yields two key observations: (i) sliding window evaluation consistently inflates performance on short inputs, and (ii) both full-precision and quantized models appear to realise gains as the evaluated segment length grows.
Rethinking Creativity Evaluation: A Critical Analysis of Existing Creativity Evaluations
Graduate Institute of Communication Engineering, National Taiwan University（台湾大学通信工程研究院）；Computer Science Department, University of California, Los Angeles（加州大学洛杉矶分校计算机科学系）；Graduate Institute of Networking and Multimedia, National Taiwan University（台湾大学网络与多媒体研究院）；Department of Electrical Engineering, National Taiwan University（台湾大学电子工程系）
Is my model perplexed for the right reason? Contrasting LLMs' Benchmark Behavior with Token-Level Perplexity
College of Informatics, University of Amsterdam（阿姆斯特丹大学信息学学院）；Department of Medical Informatics, Amsterdam University Medical Center（阿姆斯特丹大学医学中心医学信息学系）；ILLC, University of Amsterdam（阿姆斯特丹大学逻辑、语言与计算研究所）
The Lottery LLM Hypothesis, Rethinking What Abilities Should LLM Compression Preserve?
The Hong Kong University of Science and Technology（香港科技大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；National University of Singapore（新加坡国立大学）
On the Fallacy of Global Token Perplexity in Spoken Language Model Evaluation
Carnegie Mellon University（卡内基梅隆大学）；National Taiwan University（国立台湾大学）；Toyota Technological Institute at Chicago（芝加哥丰田技术研究所）；Massachusetts Institute of Technology（麻省理工学院）
VRPO: Rethinking Value Modeling for Robust RL under Noisy Supervision in LLM Post-Training
College of Computer Science and Artificial Intelligence, Fudan University（复旦大学计算机科学与人工智能学院）；Honor Device Co., Ltd（荣耀设备有限公司）；Institute of Trustworthy Embodied AI, Fudan University（复旦大学可信具身人工智能研究院）；Shanghai Key Laboratory of Multimodal Embodied AI（上海多模态具身人工智能重点实验室）
Rethinking the Understanding Ability across LLMs through Mutual Information
Department of Industrial and Systems Engineering（工业与系统工程系）；University of Houston（休斯敦大学）；Bakar Computational Health Sciences Institute（巴卡计算健康科学研究所）；UCSF
Rethinking the Evaluation for Conversational Recommendation in the Era of Large Language Models
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；School of Information, Renmin University of China（中国人民大学信息学院）；School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机科学与工程学院）
ExpliCa: Evaluating Explicit Causal Reasoning in Large Language Models
CoLing Lab, Department of Philology, Literature, and Linguistics, University of Pisa, Italy（皮尔森大学哲学、文学与语言学系协作语言实验室）；Department of Informatics, University of Pisa, Italy（皮尔森大学信息学系）；Department of Chinese and Bilingual Studies, The Hong Kong Polytechnic University（香港理工大学中文与双语研究系）
Context-Aware Membership Inference Attacks against Pre-trained Large Language Models
Mohamed bin Zayed University of Artificial Intelligence（莫扎伊德·本·扎耶德人工智能大学）；Brave Software（Brave软件公司）；Imperial College London（帝国理工学院）；National University of Singapore（新加坡国立大学）
University of Manchester, UK（曼彻斯特大学，英国）；Featurespace, a Visa Solution（Visa解决方案的Featurespace）；Risk and Security AI Lab, Visa Inc., UK（Visa公司的风险与安全AI实验室，英国）；University of Cambridge, UK（剑桥大学，英国）
ROMER: Expert Replacement and Router Calibration for Robust MoE LLMs on Analog Compute-in-Memory Systems
The Department of Electrical and Computer Engineering, The University of Hong Kong（香港大学电子与计算机工程系）；The School of Integrated Circuit Science and Engineering, Beihang University（北航集成电路科学与工程学院）
