Action Quality Assessment (AQA) has broad applications in physical therapy, sports coaching, and competitive judging. Although Vision Language Models (VLMs) hold considerable promise for AQA, their actual performance in this domain remains largely uncharacterised. We present a comprehensive evaluation of state-of-the-art VLMs across activity domains (e.g. fitness, figure skating, diving), tasks, representations, and prompting strategies. Baseline results reveal that Gemini 3.1 Pro, Qwen3-VL and InternVL3.5 models perform only marginally above random chance, and although strategies such as incorporation of skeleton information, grounding instructions, reasoning structures and in-context learning lead to isolated gains, none is consistently effective. Analysis of prediction distributions uncovers two systematic biases: a tendency to predict correct execution regardless of visual evidence, and a sensitivity to superficial linguistic framing. Reformulating tasks contrastively to mitigate these biases yields minimal improvement, suggesting that the models' limitations go beyond these biases, pointing to a fundamental difficulty with fine-grained movement quality assessment. Our findings establish a rigorous baseline for future VLM-based AQA research and provide an actionable outline for failure modes requiring mitigation prior to reliable real-world deployment.
VisionQ: VLM-as-a-Judge Taxonomy, Dataset and Benchmark for Qualitative Analysis in Computer Vision
University of Information Technology, VNU-HCM(越南国立胡志明市大学信息技术大学);University College Cork(科克大学);Hanoi University of Science and Technology(河内科技大学);Trinity College Dublin(都柏林圣三一大学)
Can Vision-Language Models Judge Olympic Diving? From Reasoning to Scores in Zero-Shot Action Quality Assessment
ESPOL Polytechnic University(ESPOL理工大学);University of Granada(格拉纳达大学);Universidad de Las Palmas de Gran Canaria(拉斯帕尔马斯·德·大加那利大学);Michigan Technological University(密歇根理工大学)
CPJ: Explainable Agricultural Pest Diagnosis via Caption-Prompt-Judge with LLM-Judged Refinement
Business School, Shandong University of Technology, Shandong, China(山东科技大学商学院);Institute of International Language Services Studies, Macau Millennium College, Macau SAR, China(国际语言服务研究所,澳门 Millennium 学院)
Cross-Cultural Expert-Level Art Critique Evaluation with Vision-Language Models
DJCAD, University of Dundee, United Kingdom(邓迪大学DJCAD研究中心,英国);Hebei Academy of Fine Art, China(河北美术院,中国);Computer Science, Nanjing University, China(南京大学计算机科学系,中国);Department of Mechanical Engineering, School of Engineering, University of Birmingham(伯明翰大学工程学院机械工程系)
Agri-CPJ: A Training-Free Explainable Framework for Agricultural Pest Diagnosis Using Caption-Prompt-Judge and LLM-as-a-Judge
Business School, Shandong University of Technology(山东理工大学商学院);Faculty of Data Science, City University of Macau(澳门城市大学数据科学学院);Guangdong Institute of Intelligent Science and Technology(广东智能科学与技术研究院);Macau Millennium College(澳门 millennium 学院);Department of Computer and Information Science, University of Macau(澳门大学计算机与信息科学系)
Does AI See like Art Historians? Interpreting How Vision Language Models Recognize Artistic Style
Columbia University, Department of Computer Science(哥伦比亚大学计算机科学系);Columbia University, Department of Art History & Archaeology(哥伦比亚大学艺术史与考古系);University of Texas at Austin(德克萨斯大学奥斯汀分校);UNC Chapel Hill(北卡罗来纳大学教堂山分校)
Rethinking LLM-as-a-Judge: Representation-as-a-Judge with Small Language Models via Semantic Capacity Asymmetry
Ping An Technology (Shenzhen) Co., Ltd.(平安科技(深圳)有限公司);University of Pittsburgh(匹兹堡大学);University of Maryland, College Park(马里兰大学学院公园分校);University of Connecticut(康涅狄格大学)
3D-DefectBench: A Controlled Factorial Study of Vision-Language Model Evaluation Pipelines for Fine-Grained 3D Generation Defects
Roblox Corporation(罗布乐思公司);Berkeley AI Research Lab & Department of Industrial Engineering and Operations Research, University of California, Berkeley(加州大学伯克利分校伯克利人工智能研究实验室及工业工程与运筹学系);Computer Science Department, Stanford University(斯坦福大学计算机科学系);Division of Biostatistics, University of California, Berkeley(加州大学伯克利分校生物统计学系)
Diagnosing the Reliability of LLM-as-a-Judge via Item Response Theory
Department of Artificial Intelligence, Chung-Ang University, Seoul, Republic of Korea(Chung-Ang 大学人工智能系);Department of Industrial Engineering, Seoul National University, Seoul, Republic of Korea(首尔国立大学工业工程系)
Preference Leakage: A Contamination Problem in LLM-as-a-judge
Arizona State University(亚利桑那州立大学);University of California, Los Angeles(加州大学洛杉矶分校);University of Notre Dame(诺丁汉大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
The Catastrophic Paradox of Human Cognitive Frameworks in Large Language Model Evaluation: A Comprehensive Empirical Analysis of the CHC-LLM Incompatibility