Digital Environment Research Institute, Queen Mary University of London（伦敦玛丽女王大学数字环境研究所）；Centre for Bioengineering, Queen Mary University of London（伦敦玛丽女王大学生物工程中心）；UCL Hawkes Institute, University College London（伦敦大学学院霍克斯研究所）
Shaheer U. Saeed, Zion Tse, Gregory Slabaugh, Daniel C. Alexander, Yipeng Hu
Modern machines can solve equations, follow logic and reason through language, yet they still struggle with problems that require visual non-verbal reasoning. This is evident even in advanced systems such as DeepSeek, ChatGPT and Gemini, which remain limited on visual tasks such as completing abstract patterns or delineating ambiguous regions-of-interest. Unlike explicit intermediate steps in verbal reasoning which can be verified through logic or automated tests, visual reasoning involves implicit intermediate steps with no clear methods for verification. Non-verbal reasoning is therefore challenging to model and, as a result, remains a fundamental limitation of current machine intelligence. Here we introduce GenVerOpt, a general framework that unifies generation, verification and optimisation into a reasoning pipeline for machines. By generating candidate solutions, verifying their correctness, and iteratively refining them, GenVerOpt enables per-sample optimisation rather than relying solely on dataset-level trends captured during model training. This process mirrors human problem-solving where multiple possibilities are imagined, evaluated, and refined until a correct solution emerges. Across diverse domains, GenVerOpt demonstrates capabilities beyond existing systems: it surpasses humans on abstract IQ puzzles, matches clinicians in prostate cancer segmentation, exceeds experts in liver vessel reconstruction and outperforms state-of-the-art approaches in tumour boundary alignment. These results show that machine reasoning can extend beyond language, establishing visual non-verbal reasoning as a foundation for more flexible and generalisable intelligence.
We aim to automatically identify human action reasons in online videos. We focus on the widespread genre of lifestyle vlogs, in which people perform actions while verbally describing them. We introduce and make publicly available the WhyAct dataset, consisting of 1,077 visual actions manually annotated with their reasons. We describe a multimodal model that leverages visual and textual information to automatically infer the reasons corresponding to an action presented in the video.
ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generation
University of Maryland, College Park（马里兰大学学院公园分校）；University of Pennsylvania（宾夕法尼亚大学）；The Hong Kong University of Science and Technology（香港科学与技术大学）；University of Michigan（密歇根大学）；University of Southern California（南加州大学）
MMTIT-Bench: A Multilingual and Multi-Scenario Benchmark with Cognition-Perception-Reasoning Guided Text-Image Machine Translation
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；Tencent（腾讯）；Nankai University（南开大学）；University of Chinese Academy of Sciences（中国科学院大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Nanjing University of Science and Technology（南京理工大学）
Reasoning in machine vision by learning fast and slow thinking
Centre for Bioengineering, Queen Mary University of London（伦敦玛丽女王大学生物工程中心）；Digital Environment Research Institute, Queen Mary University of London（伦敦玛丽女王大学数字环境研究所）；School of Engineering and Materials Science, Queen Mary University of London（伦敦玛丽女王大学工程与材料科学学院）；UCL Hawkes Institute, University College London（伦敦大学学院UCL霍克斯研究所）；Department of Medical Physics and Biomedical Engineering, University College London（伦敦大学学院医学物理与生物医学工程系）；Centre for Urology Imaging, Prostate, AI and Surgical Studies (COMPASS) Research Group, Division of Surgery and Interventional Science, University College London（伦敦大学学院外科与介入科学部泌尿影像、前列腺、人工智能与手术研究（COMPASS）组中心）；Department of Urology, Comprehensive Cancer Center, Medical University of Vienna（维也纳医科大学综合癌症中心泌尿科）；Division of Surgery and Interventional Science, University College London（伦敦大学学院外科与介入科学部）；Department of Computer Science, University College London（伦敦大学学院计算机科学系）
Indian Institute of Technology Kharagpur（印度理工学院克勒格布尔分校）
Human vision is able to compensate imperfections in sensory inputs from the real world by reasoning based on prior knowledge about the world. Machine learning has had a significant impact on computer vision due to its inherent ability in handling imprecision, but the absence of a reasoning framework based on domain knowledge limits its ability to interpret complex scenarios. We propose semi-lexical languages as a formal basis for dealing with imperfect tokens provided by the real world. The power of machine learning is used to map the imperfect tokens into the alphabet of the language and symb…
Without relevant human priors, neural networks may learn uninterpretable features. We propose Dynamics of Attention for Focus Transition (DAFT) as a human prior for machine reasoning. DAFT is a novel method that regularizes attention-based reasoning by modelling it as a continuous dynamical system using neural ordinary differential equations. As a proof of concept, we augment a state-of-the-art visual reasoning model with DAFT. Our experiments reveal that applying DAFT yields similar performance to the original model while using fewer reasoning steps, showing that it implicitly learns to skip …
Brown University（布朗大学）；Adobe Research（奥多比研究院）；KAIST（韩国科学技术院）
We present the Shape Part Slot Machine, a new method for assembling novel 3D shapes from existing parts by performing contact-based reasoning. Our method represents each shape as a graph of ``slots,'' where each slot is a region of contact between two shape parts. Based on this representation, we design a graph-neural-network-based model for generating new slot graphs and retrieving compatible parts, as well as a gradient-descent-based optimization scheme for assembling the retrieved parts into a complete shape that respects the generated slot graph. This approach does not require any semantic…
Institute of Computer Science, Foundation for Research and Technology, Hellas（希腊研究与技术基金会计算机科学研究所）；Aristotle University of Thessaloniki（塞萨洛尼基亚里士多德大学）
Object perception is a fundamental sub-field of Computer Vision, covering a multitude of individual areas and having contributed high-impact results. While Machine Learning has been traditionally applied to address related problems, recent works also seek ways to integrate knowledge engineering in order to expand the level of intelligence of the visual interpretation of objects, their properties and their relations with their environment. In this paper, we attempt a systematic investigation of how knowledge-based methods contribute to diverse object perception tasks. We review the latest achie…
There has been a surge in the number of Machine Learning methods to analyze products kept on retail shelves images. Deep learning based computer vision methods can be used to detect products on retail shelves and then classify them. However, there are different sized variants of products which look exactly the same visually and the method to differentiate them is to look at their relative sizes with other products on shelves. This makes the process of deciphering the sized based variants from each other using computer vision algorithms alone impractical. In this work, we propose methods to asc…
UCLA Center for Vision, Cognition, Learning and Autonomy（加州大学洛杉矶分校视觉、认知、学习与自主性中心）；International Center for AI and Robot Autonomy (CARA)（人工智能与机器人自主性国际中心（CARA））
Dramatic progress has been witnessed in basic vision tasks involving low-level perception, such as object recognition, detection, and tracking. Unfortunately, there is still an enormous performance gap between artificial vision systems and human intelligence in terms of higher-level vision problems, especially ones involving reasoning. Earlier attempts in equipping machines with high-level reasoning have hovered around Visual Question Answering (VQA), one typical task associating vision and language understanding. In this work, we propose a new dataset, built in the context of Raven's Progress…
Recent advances in natural language processing and computer vision have led to AI models that interpret simple scenes at human levels. Yet, we do not have a complete understanding of how humans and AI models differ in their interpretation of more complex scenes. We created a dataset of complex scenes that contained human behaviors and social interactions. AI and humans had to describe the scenes with a sentence. We used a quantitative metric of similarity between scene descriptions of the AI/human and ground truth of five other human descriptions of each scene. Results show that the machine/hu…
Singapore University of Technology and Design（新加坡科技设计大学）；Nanyang Technological University（南洋理工大学）；Singapore Polytechnic（新加坡理工学院）
We introduce algebraic machine reasoning, a new reasoning framework that is well-suited for abstract reasoning. Effectively, algebraic machine reasoning reduces the difficult process of novel problem-solving to routine algebraic computation. The fundamental algebraic objects of interest are the ideals of some suitably initialized polynomial ring. We shall explain how solving Raven's Progressive Matrices (RPMs) can be realized as computational problems in algebra, which combine various well-known algebraic subroutines that include: Computing the Gröbner basis of an ideal, checking for ideal con…
