Large Language Models (LLMs) have demonstrated remarkable capabilities across various domains, including software development, education, and technical assistance. Among these, software development is one of the key areas where LLMs are increasingly adopted. However, when hardware constraints are considered-for instance, in physical computing, where software must interact with and control physical hardware -their effectiveness has not been fully explored. To address this gap, we introduce \textsc{PCEval} (Physical Computing Evaluation), the first benchmark in physical computing that enables a fully automatic evaluation of the capabilities of LLM in both the logical and physical aspects of the projects, without requiring human assessment. Our evaluation framework assesses LLMs in generating circuits and producing compatible code across varying levels of project complexity. Through comprehensive testing of 13 leading models, \textsc{PCEval} provides the first reproducible and automatically validated empirical assessment of LLMs' ability to reason about fundamental hardware implementation constraints within a simulation environment. Our findings reveal that while LLMs perform well in code generation and logical circuit design, they struggle significantly with physical breadboard layout creation, particularly in managing proper pin connections and avoiding circuit errors. \textsc{PCEval} advances our understanding of AI assistance in hardware-dependent computing environments and establishes a foundation for developing more effective tools to support physical computing education.
Evaluating LLMs' Divergent Thinking Capabilities for Scientific Idea Generation with Minimal Context
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学人工智能学院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Bank of China（中国银行）；School of Engineering Science, University of Chinese Academy of Sciences（中国科学院大学工程科学学院）；State Key Laboratory of Nonlinear Mechanics, Institute of Mechanics, Chinese Academy of Sciences（中国科学院力学研究所非线性力学重点实验室）
StatABench: Dataset and Framework for Evaluating Statistical Analysis Capabilities of LLMs
Southern University of Science and Technology（南方科技大学）；Alibaba Group（阿里巴巴集团）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；The University of Hong Kong（香港大学）
DICE-BENCH: Evaluating the Tool-Use Capabilities of Large Language Models in Multi-Round, Multi-Party Dialogues
IPAI, Seoul National University（首尔国立大学IPAI）；Department of Intelligence and Information, Seoul National University（首尔国立大学智能与信息系）；Korea University（韩国大学）；AIGEN Sciences（AIGEN科学）；Cornell University（康奈尔大学）
LR^2Bench: Evaluating Long-chain Reflective Reasoning Capabilities of Large Language Models via Constraint Satisfaction Problems
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Zhongguancun Academy（中关村学院）；Wuhan AI Research（武汉人工智能研究院）
EmotionQueen: A Benchmark for Evaluating Empathy of Large Language Models
Fudan University（复旦大学）；School of Computer Science, Fudan University（复旦大学计算机科学学院）；Beijing Yuandian Technology（北京原点科技）；School of Engineering and Applied Science, University of Pennsylvania（宾夕法尼亚大学工程与应用科学学院）
From Capabilities to Performance: Evaluating Key Functional Properties of LLM Architectures in Penetration Testing
Bradley Department of Electrical and Computer Engineering, Virginia Tech（弗吉尼亚理工大学电气与计算机工程系布雷拉德部门）；National Security Institute, Virginia Tech（弗吉尼亚理工大学国家安全研究所）
OralMLLM-Bench: Evaluating Cognitive Capabilities of Multimodal Large Language Models in Dental Practice
Beijing Stomatological Hospital（北京口腔医院）；School of Stomatology, Capital Medical University（首都医科大学口腔医学院）；University of Minnesota（明尼苏达大学）；The Hong Kong Polytechnic University（香港理工大学）
AstroMMBench: A Benchmark for Evaluating Multimodal Large Language Models Capabilities in Astronomy
University of Chinese Academy of Sciences（中国科学院大学）；National Astronomical Observatories, Chinese Academy of Sciences（中国科学院国家天文台）；Zhejiang Laboratory（之江实验室）；Tianjin University（天津大学）
EarthSE: A Benchmark for Evaluating Earth Scientific Exploration Capability of LLMs
Shanghai Jiao Tong University（上海交通大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；The Hong Kong Polytechnic University（香港理工大学）；The Chinese University of Hong Kong（香港中文大学）
