Existing illumination estimation methods primarily recover lighting distributions for rendering, but typically leave the contributions of individual physical light sources at specific scene locations implicit. We address this gap by leveraging the scene understanding capabilities of vision-language models (VLMs) to connect local illumination with its physical sources. We introduce LAMP, a framework for point-conditioned, source-aware illumination reasoning from a single RGB image with marked query points. LAMP constructs a shared inventory of active light sources, ranks contributing sources for each query, and predicts their 3D query-to-source directions. We supervise these relationships using procedurally controlled indoor scenes, reusing independently rendered source responses to derive contribution targets across lighting configurations and obtaining exact directions from source and query positions. On the Infinigen test set, fine-tuning Qwen3-VL-8B increases dominant-source 3D PCK@0.5m from 0% to 70.6%, with 94.5% of directions within 12.5 degrees of ground truth among correctly localized sources. Combined with a lightweight network predicting dense RGB irradiance, LAMP achieves probe-rendering quality comparable to prior lighting-estimation methods while additionally localizing individual physical sources in 3D and explicitly attributing local illumination to them.
SUNY at Buffalo（布法罗纽约州立大学）；Tianjin University（天津大学）
Deep convolutional neural networks (CNN) have recently been shown to generate promising results for aesthetics assessment. However, the performance of these deep CNN methods is often compromised by the constraint that the neural network only takes the fixed-size input. To accommodate this requirement, input images need to be transformed via cropping, warping, or padding, which often alter image composition, reduce image resolution, or cause image distortion. Thus the aesthetics of the original images is impaired because of potential loss of fine grained details and holistic image layout. Howev…
Engine-Native Editable 3D World Reconstruction with Objects and Lighting
Tsinghua University（清华大学）；Nankai University（南开大学）；University of Electronic Science and Technology of China（电子科技大学）；Sun Yat-sen University（中山大学）；Nanyang Technological University（南洋理工大学）
University of Central Florida（中佛罗里达大学）；Adobe Research（奥多比研究院）
Compositing-aware object search aims to find the most compatible objects for compositing given a background image and a query bounding box. Previous works focus on learning compatibility between the foreground object and background, but fail to learn other important factors from large-scale data, i.e. geometry and lighting. To move a step further, this paper proposes GALA (Geometry-and-Lighting-Aware), a generic foreground object search method with discriminative modeling on geometry and lighting compatibility for open-world image compositing. Remarkably, it achieves state-of-the-art results o…
Tsinghua University（清华大学）；Meta Reality Labs（Meta现实实验室）；Guangdong OPPO Mobile Telecommunications Corp., Ltd（广东OPPO移动通信有限公司）
Single-image human relighting aims to relight a target human under new lighting conditions by decomposing the input image into albedo, shape and lighting. Although plausible relighting results can be achieved, previous methods suffer from both the entanglement between albedo and lighting and the lack of hard shadows, which significantly decrease the realism. To tackle these two problems, we propose a geometry-aware single-image human relighting framework that leverages single-image geometry reconstruction for joint deployment of traditional graphics rendering and neural rendering techniques. F…
Predicting panoramic indoor lighting from a single perspective image is a fundamental but highly ill-posed problem in computer vision and graphics. To achieve locale-aware and robust prediction, this problem can be decomposed into three sub-tasks: depth-based image warping, panorama inpainting and high-dynamic-range (HDR) reconstruction, among which the success of panorama inpainting plays a key role. Recent methods mostly rely on convolutional neural networks (CNNs) to fill the missing contents in the warped panorama. However, they usually achieve suboptimal performance since the missing cont…
We propose a framework, called LiftedGAN, that disentangles and lifts a pre-trained StyleGAN2 for 3D-aware face generation. Our model is "3D-aware" in the sense that it is able to (1) disentangle the latent space of StyleGAN2 into texture, shape, viewpoint, lighting and (2) generate 3D components for rendering synthetic images. Unlike most previous methods, our method is completely self-supervised, i.e. it neither requires any manual annotation nor 3DMM model for training. Instead, it learns to generate images as well as their 3D components by distilling the prior knowledge in StyleGAN2 with a…
Single Document Image Highlight Removal via A Large-Scale Real-World Dataset and A Location-Aware Network
College of Computer Science, Jilin University（吉林大学计算机科学学院）；Institute of Information Science, Academia Sinica（中国科学院信息科学研究院）；National Yang Ming Chiao Tung University（阳明交通大学）；National Taiwan University（台湾大学）
NVIDIA（英伟达）；University of Toronto（多伦多大学）；Vector Institute（向量研究所）
We consider the challenging problem of outdoor lighting estimation for the goal of photorealistic virtual object insertion into photographs. Existing works on outdoor lighting estimation typically simplify the scene lighting into an environment map which cannot capture the spatially-varying lighting effects in outdoor scenes. In this work, we propose a neural approach that estimates the 5D HDR light field from a single image, and a differentiable object insertion formulation that enables end-to-end training with image-based losses that encourage realism. Specifically, we design a hybrid lighti…
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Beijing Jiaotong University（北京交通大学）；University of California San Diego（加利福尼亚大学圣迭戈分校）；Cardiff University（卡迪夫大学）；City University of Hong Kong（香港城市大学）；The Hong Kong University of Science and Technology（香港科技大学）；University of Chinese Academy of Sciences（中国科学院大学）；School of Computer Science, Peking University（北京大学计算机学院）
Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；The Chinese University of Hong Kong（香港中文大学）；Nanyang Technological University（南洋理工大学）
The advancement of generative radiance fields has pushed the boundary of 3D-aware image synthesis. Motivated by the observation that a 3D object should look realistic from multiple viewpoints, these methods introduce a multi-view constraint as regularization to learn valid 3D radiance fields from 2D images. Despite the progress, they often fall short of capturing accurate 3D shapes due to the shape-color ambiguity, limiting their applicability in downstream tasks. In this work, we address this ambiguity by proposing a novel shading-guided generative implicit model that is able to learn a stark…
Moving Light Adaptive Colonoscopy Reconstruction via Illumination-Attenuation-Aware 3D Gaussian Splatting
Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology（武汉光电国家研究中心，华中科技大学）；School of Computer Science, Wuhan University（武汉大学计算机学院）；Wuhan United Imaging Healthcare Surgical Technology Co., Ltd（武汉联影医疗手术技术有限公司）
Lighting is a determining factor in photography that affects the style, expression of emotion, and even quality of images. Creating or finding satisfying lighting conditions, in reality, is laborious and time-consuming, so it is of great value to develop a technology to manipulate illumination in an image as post-processing. Although previous works have explored techniques based on the physical viewpoint for relighting images, extensive supervisions and prior knowledge are necessary to generate reasonable images, restricting the generalization ability of these works. In contrast, we take the v…
LISA: Language-guided Interference-aware Spatial-Frequency Attention for Driver Gaze Estimation
School of Computer Science (National Pilot Software Engineering School), Beijing University of Posts and Telecommunications（北京邮电大学计算机科学学院（国家级试点软件工程学院））；Key Laboratory of Trustworthy Distributed Computing and Service (BUPT), Ministry of Education（教育部可信分布式计算与服务重点实验室（BUPT））；School of Electrical Engineering, Guangxi University（广西大学电气工程学院）；Zhejiang University（浙江大学）
We present a scheme for fast environment light estimation from the RGBD appearance of individual objects and their local image areas. Conventional inverse rendering is too computationally demanding for real-time applications, and the performance of purely learning-based techniques may be limited by the meager input data available from individual objects. To address these issues, we propose an approach that takes advantage of physical principles from inverse rendering to constrain the solution, while also utilizing neural networks to expedite the more computationally expensive portions of its p…
Spectrum Aware Illumination Estimation Using Multispectral Image
Department of Electronic and Electrical Engineering, Ewha W. University（成均馆大学电子与电气工程系）；Telechips；Samsung Advanced Institute of Technology（三星先进技术研究所）；Department of Design, Ewha W. University（成均馆大学设计系）
A Luminance-Aware Multi-Scale Network for Polarization Image Fusion with a Multi-Scene Dataset
Guangdong-HongKong-Macao Joint Laboratory for Intelligent Micro-Nano Optoelectronic Technology, School of Physics and Optoelectronic Engineering, Foshan University（广东-香港-澳门智能微纳光电子技术联合实验室，物理与光电工程学院，佛山大学）；State Key Laboratory of Dynamic Measurement Technology, North University of China（动态测试技术国家重点实验室，中北大学）；School of Information Engineering and Automation, Kunming University of Science and Technology（信息工程与自动化学院，昆明理工大学）
Object detection is essential to safe autonomous or assisted driving. Previous works usually utilize RGB images or LiDAR point clouds to identify and localize multiple objects in self-driving. However, cameras tend to fail in bad driving conditions, e.g. bad weather or weak lighting, while LiDAR scanners are too expensive to get widely deployed in commercial applications. Radar has been drawing more and more attention due to its robustness and low cost. In this paper, we propose a scene-aware radar learning framework for accurate and robust object detection. First, the learning framework conta…
University of Bath（巴斯大学）；Adobe Research（Adobe研究院）；UCL（伦敦大学学院）
We present BlockGAN, an image generative model that learns object-aware 3D scene representations directly from unlabelled 2D images. Current work on scene representation learning either ignores scene background or treats the whole scene as one object. Meanwhile, work that considers scene compositionality treats scene objects only as image patches or 2D layers with alpha maps. Inspired by the computer graphics pipeline, we design BlockGAN to learn to first generate 3D features of background and foreground objects, then combine them into 3D features for the wholes cene, and finally render them i…
Indoor scenes typically exhibit complex, spatially-varying appearance from global illumination, making inverse rendering a challenging ill-posed problem. This work presents an end-to-end, learning-based inverse rendering framework incorporating differentiable Monte Carlo raytracing with importance sampling. The framework takes a single image as input to jointly recover the underlying geometry, spatially-varying lighting, and photorealistic materials. Specifically, we introduce a physically-based differentiable rendering layer with screen-space ray tracing, resulting in more realistic specular …
LiftFeat: 3D Geometry-Aware Local Feature Matching
School of Computer Science, Wuhan University（武汉大学计算机学院）；SF Technology（SF科技）；School of Computer Science, Central China Normal University（中央财经大学计算机学院）；Institute for Infocomm Research, A*STAR（资讯研究院）
