Robust Low-Light Human Pose Estimation through Illumination-Texture Modulation
Nanjing University of Posts and Telecommunications（南京邮电大学）；Surrey Institute for People-Centred Artificial Intelligence（萨里以人为本人工智能研究所）；University of Surrey（萨里大学）
As critical visual details become obscured, the low visibility and high ISO noise in extremely low-light images pose a significant challenge to human pose estimation. Current methods fail to provide high-quality representations due to reliance on pixel-level enhancements that compromise semantics and the inability to effectively handle extreme low-light conditions for robust feature learning. In this work, we propose a frequency-based framework for low-light human pose estimation, rooted in the "divide-and-conquer" principle. Instead of uniformly enhancing the entire image, our method focuses on task-relevant information. By applying dynamic illumination correction to the low-frequency components and low-rank denoising to the high-frequency components, we effectively enhance both the semantic and texture information essential for accurate pose estimation. As a result, this targeted enhancement method results in robust, high-quality representations, significantly improving pose estimation performance. Extensive experiments demonstrating its superiority over state-of-the-art methods in various challenging low-light scenarios.
Accurate and fast extraction of the foreground object is one of the most significant issues to be solved due to its important meaning for object tracking and recognition in video surveillance. Although many foreground object detection methods have been proposed in the recent past, it is still regarded as a tough problem due to illumination variations and dynamic backgrounds challenges. In this paper, we propose a robust foreground object detection method with two aspects of contributions. First, we propose a robust texture operator named Robust Local Binary Similarity Pattern (RLBSP), which sh…
Text as Illumination: Spatial Contrastive Retinex Learning for Language-guided Medical Image Segmentation
School of Software Technology & DUT-RU International School of Information Science and Engineering, Dalian University of Technology（大连理工大学软件学院 & 大连理工大学-俄罗斯国际信息科学与工程学院）；School of Future Technology, Dalian University of Technology（大连理工大学未来技术学院）；Central Hospital of Dalian University of Technology（大连理工大学中心医院）；Cancer Hospital of Dalian University of Technology（大连理工大学肿瘤医院）；College of Computer Science and Engineering, Shandong University of Science and Technology（山东科技大学计算机科学与工程学院）
Wuhan University of Science and Technology（武汉科技大学）；Institute of Robotics and Intelligent Systems, Wuhan University of Science and Technology（武汉科技大学机器人与智能系统研究院）；Institute for Infocomm Research（信息通信研究院）
In order to improve the accuracy of face recognition under varying illumination conditions, a local texture enhanced illumination normalization method based on fusion of differential filtering images (FDFI-LTEIN) is proposed to weaken the influence caused by illumination changes. Firstly, the dynamic range of the face image in dark or shadowed regions is expanded by logarithmic transformation. Then, the global contrast enhanced face image is convolved with difference of Gaussian filters and difference of bilateral filters, and the filtered images are weighted and merged using a coefficient sel…
Self-supervised learning aims to learn image feature representations without the usage of manually annotated labels. It is often used as a precursor step to obtain useful initial network weights which contribute to faster convergence and superior performance of downstream tasks. While self-supervision allows one to reduce the domain gap between supervised and unsupervised learning without the usage of labels, the self-supervised objective still requires a strong inductive bias to downstream tasks for effective transfer learning. In this work, we present our material and texture based self-supe…
Nankai University（南开大学）；Taishan University（泰山学院）；Tianjin University（天津大学）；Inception Institute of Artificial Intelligence（起源人工智能研究院）；Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；Shenzhen Institute of Future Media Technology（深圳未来媒体技术研究院）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Retinex theory is developed mainly to decompose an image into the illumination and reflectance components by analyzing local image derivatives. In this theory, larger derivatives are attributed to the changes in reflectance, while smaller derivatives are emerged in the smooth illumination. In this paper, we utilize exponentiated local derivatives (with an exponent γ) of an observed image to generate its structure map and texture map. The structure map is produced by been amplified with γ > 1, while the texture map is generated by been shrank with γ < 1. To this end, we design exponential filte…
GTFMN: Guided Texture and Feature Modulation Network for Low-Light Image Enhancement and Super-Resolution
1 Graduate School of Engineering, Tohoku University, Sendai, Japan 2 Dept. of Radiology \& Biomedical Imaging, Yale University, New Haven, CT, USA 3 Dept. of Computer Science；Information Engineering, National Taiwan University, Taipei, Taiwan 4 Graduate Institute of Communication Engineering, National Taiwan University, Taipei, Taiwan 5 Research Center for Information Technology Innovation, Academia Sinica, Taipei, Taiwan
MaterialMVP: Illumination-Invariant Material Generation via Multi-view PBR Diffusion
Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）；Tencent Hunyuan（腾讯文元）；The Hong Kong University of Science and Technology（香港科技大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））
A simple and effective low-light image enhancement method based on a noise-aware texture-preserving retinex model is proposed in this work. The new method, called NATLE, attempts to strike a balance between noise removal and natural texture preservation through a low-complexity solution. Its cost function includes an estimated piece-wise smooth illumination map and a noise-free texture-preserving reflectance map. Afterwards, illumination is adjusted to form the enhanced image together with the reflectance map. Extensive experiments are conducted on common low-light image enhancement datasets t…
High-Fidelity Facial Albedo Estimation via Texture Quantization
University of Technology Sydney（悉尼科技大学）；Shanghai Jiao Tong University（上海交通大学）；DeepGlint（旷视科技（北京））；Insightface；Zhejiang University（浙江大学）；Imperial College London（伦敦帝国学院）
Shenzhen Institute of Advanced Technology (SIAT), Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
Reconstructing high-fidelity 3D facial texture from a single image is a quite challenging task due to the lack of complete face information and the domain gap between the 3D face and 2D image. Further, obtaining re-renderable 3D faces has become a strongly desired property in many applications, where the term 're-renderable' demands the facial texture to be spatially complete and disentangled with environmental illumination. In this paper, we propose a new self-supervised deep learning framework for reconstructing high-quality and re-renderable facial albedos from single-view images in-the-wil…
Imperceptible Physical Attack against Face Recognition Systems via LED Illumination Modulation
Jinan University（暨南大学）；Zhejiang University（浙江大学）；The University of Hong Kong（香港大学）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；Peng Cheng Laboratory（鹏城实验室）
Najafabad Branch, Islamic Azad University（伊斯兰阿扎德大学纳杰法巴德分校）
Texture classification is an active topic in image processing which plays an important role in many applications such as image retrieval, inspection systems, face recognition, medical image processing, etc. There are many approaches extracting texture features in gray-level images such as local binary patterns, gray level co-occurrence matrices, statistical features, skeleton, scale invariant feature transform, etc. The texture analysis methods can be categorized in 4 groups titles: statistical methods, structural methods, filter-based and model based approaches. In many related researches, au…
Recovering the radiometric properties of a scene (i.e., the reflectance, illumination, and geometry) is a long-sought ability of computer vision that can provide invaluable information for a wide range of applications. Deciphering the radiometric ingredients from the appearance of a real-world scene, as opposed to a single isolated object, is particularly challenging as it generally consists of various objects with different material compositions exhibiting complex reflectance and light interactions that are also part of the illumination. We introduce the first method for radiometric scene dec…
Continuous Splatting meets Retinex: Continuous Gaussian Splatting and Implicit Reflectance Modeling for Low-Light Image Enhancement
College of Mechanical and Vehicle Engineering, Chongqing University（重庆大学机械与运载工程学院）；School of Engineering Science, University of Science and Technology of China（中国科学技术大学工程科学学院）；National Innovation Center of Intelligent and Connected Vehicles（国家智能网联汽车创新中心）；School of Vehicle and Mobility, Tsinghua University（清华大学车辆与运载学院）
University of São Paulo（圣保罗大学）；ENSTA-ParisTech（巴黎高等先进技术学校）；Université de Paris-Saclay（巴黎-萨克雷大学）
To recognize textures many methods have been developed along the years. However, texture datasets may be hard to be classified due to artefacts such as a variety of scale, illumination and noise. This paper proposes the application of binary distance transform on the original dataset to add information to texture representation and consequently improve recognition. Texture images, usually in grayscale, suffers a binarization prior to distance transform and one of the resulted images are combined with original texture to improve the amount of information. Four datasets are used to evaluate our …
We present TexMesh, a novel approach to reconstruct detailed human meshes with high-resolution full-body texture from RGB-D video. TexMesh enables high quality free-viewpoint rendering of humans. Given the RGB frames, the captured environment map, and the coarse per-frame human mesh from RGB-D tracking, our method reconstructs spatiotemporally consistent and detailed per-frame meshes along with a high-resolution albedo texture. By using the incident illumination we are able to accurately estimate local surface geometry and albedo, which allows us to further use photometric constraints to adapt…
Within the domain of texture classification, a lot of effort has been spent on local descriptors, leading to many powerful algorithms. However, preprocessing techniques have received much less attention despite their important potential for improving the overall classification performance. We address this question by proposing a novel, simple, yet very powerful biologically-inspired filtering (BF) which simulates the performance of human retina. In the proposed approach, given a texture image, after applying a DoG filter to detect the "edges", we first split the filtered image into two "maps" …
Nanjing University of Science and Technology（南京理工大学）；Tencent AI Lab（腾讯人工智能实验室）
We present a large-scale facial UV-texture dataset that contains over 50,000 high-quality texture UV-maps with even illuminations, neutral expressions, and cleaned facial regions, which are desired characteristics for rendering realistic 3D face models under different lighting conditions. The dataset is derived from a large-scale face image dataset namely FFHQ, with the help of our fully automatic and robust UV-texture production pipeline. Our pipeline utilizes the recent advances in StyleGAN-based facial image editing approaches to generate multi-view normalized face images from single-image …
Shenyang Institute of Automation, Chinese Academy of Sciences（中国科学院沈阳自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）
In this paper, we propose a novel, effective and fast method to obtain a color illumination invariant and shadow-free image from a single outdoor image. Different from state-of-the-art methods for shadow-free image that either need shadow detection or statistical learning, we set up a linear equation set for each pixel value vector based on physically-based shadow invariants, deduce a pixel-wise orthogonal decomposition for its solutions, and then get an illumination invariant vector for each pixel value vector on an image. The illumination invariant vector is the unique particular solution of…
Universidad Nacional Autónoma de México（墨西哥国立自治大学）；Instituto de Óptica（光学研究所）
In many image processing applications, such as segmentation and classification, the selection of robust features descriptors is crucial to improve the discrimination capabilities in real world scenarios. In particular, it is well known that image textures constitute power visual cues for feature extraction and classification. In the past few years the local binary pattern (LBP) approach, a texture descriptor method proposed by Ojala et al., has gained increased acceptance due to its computational simplicity and more importantly for encoding a powerful signature for describing textures. However…
Illumination estimation is often used in mixed reality to re-render a scene from another point of view, to change the color/texture of an object, or to insert a virtual object consistently lit into a real video or photograph. Specifically, the estimation of a point light source is required for the shadows cast by the inserted object to be consistent with the real scene. We tackle the problem of illumination retrieval given an RGBD image of the scene as an inverse problem: we aim to find the illumination that minimizes the photometric error between the rendered image and the observation. In par…
InterLight: Leveraging Intrinsic Illumination Priors for Low-Light Image Enhancement
National Engineering Research Center for Multimedia Software, School of Computer Science, Wuhan University（武汉大学计算机学院多媒体软件国家工程研究中心）；Department of Computer Science, University of Macau（澳门大学计算机科学系）；Department of Computer Vision, Mohamed bin Zayed University of Artificial Intelligence（马尔代夫穆罕默德·本·扎耶德人工智能大学计算机视觉系）；School of Information Engineering, Guangdong University of Technology（广东技术大学信息工程学院）
University of Oulu（奥卢大学）；Shenzhen University（深圳大学）
In this paper, we propose a novel local feature, called Local Orientation Adaptive Descriptor (LOAD), to capture regional texture in an image. In LOAD, we proposed to define point description on an Adaptive Coordinate System (ACS), adopt a binary sequence descriptor to capture relationships between one point and its neighbors and use multi-scale strategy to enhance the discriminative power of the descriptor. The proposed LOAD enjoys not only discriminative power to capture the texture information, but also has strong robustness to illumination variation and image rotation. Extensive experiment…
VFACamou: View-Fused Adversarial Camouflage for Environment-Adaptive Physical Evasion
State Key Laboratory of Intelligent Vehicle Safety Technology（智能汽车安全技术国家重点实验室）；School of Cyber Science and Engineering, Huazhong University of Science and Technology（华中科技大学网络空间安全学院）；School of Computer Science and Technology, Huazhong University of Science and Technology（华中科技大学计算机科学与技术学院）；School of Software Engineering, Huazhong University of Science and Technology（华中科技大学软件学院）；Hebei Energy College of Vocation And Technology（河北能源职业技术学院）
Technion - Israel Institute of Technology（以色列理工学院）
In this paper we introduce a novel notion of separation surfaces for image decomposition. A surface is embedded in the spectral total-variation (TV) three dimensional domain and encodes a spatially-varying separation scale. The method allows good separation of textures with gradually varying pattern-size, pattern-contrast or illumination. The recently proposed total variation spectral framework is used to decompose the image into a continuum of textural scales. A desired texture, within a scale range, is found by fitting a surface to the local maximal responses in the spectral domain. A band a…
University College London（伦敦大学学院）；Adobe Research（Adobe研究院）；Imperial College London（帝国理工学院）
We learn a latent space for easy capture, consistent interpolation, and efficient reproduction of visual material appearance. When users provide a photo of a stationary natural material captured under flashlight illumination, first it is converted into a latent material code. Then, in the second step, conditioned on the material code, our method produces an infinite and diverse spatial field of BRDF model parameters (diffuse albedo, normals, roughness, specular albedo) that subsequently allows rendering in complex scenes and illuminations, matching the appearance of the input photograph. Techn…
Indian Statistical Institute（印度统计研究所）；National Institute of Technology, Rourkela（鲁尔凯拉国家技术学院）；Indian Institute of Information Technology, Sri City（斯里城印度信息技术学院）
Methods based on local image features have recently shown promise for texture classification tasks, especially in the presence of large intra-class variation due to illumination, scale, and viewpoint changes. Inspired by the theories of image structure analysis, this paper presents a simple, efficient, yet robust descriptor namely local jet pattern (LJP) for texture classification. In this approach, a jet space representation of a texture image is derived from a set of derivatives of Gaussian (DtGs) filter responses up to second order, so called local jet vectors (LJV), which also satisfy the …
Structured illumination microscopy (SIM) is a very important super-resolution microscopy technique, which provides high speed super-resolution with about two-fold spatial resolution enhancement. Several attempts aimed at improving the performance of SIM reconstruction algorithm have been reported. However, most of these highlight only one specific aspect of the SIM reconstruction -- such as the determination of the illumination pattern phase shift accurately -- whereas other key elements -- such as determination of modulation factor, estimation of object power spectrum, Wiener filtering freque…
LightQANet: Quantized and Adaptive Feature Learning for Low-Light Image Enhancement
College of Computer Science and Software Engineering, Shenzhen University（深圳大学计算机科学与软件工程学院）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算机与数据科学学院）；Computer Vision Institute, College of Computer Science and Software Engineering, Shenzhen University（深圳大学计算机视觉研究院）；School of AI and Advanced Computing, Xi’an Jiaotong-Liverpool University（西安交通大学利物浦大学人工智能与先进计算学院）；School of Mathematics and Statistics, Changsha University of Science and Technology（长沙理工大学数学与统计学院）；School of Computer Science, Sichuan Normal University（四川师范大学计算机科学学院）；Department of Computer Science, University of Nottingham Ningbo China（宁波大学计算机科学系）
Towards Controllable Low-Light Image Enhancement: A Continuous Multi-illumination Dataset and Efficient State Space Framework
Department of Computer and Information Science, University of Macau（澳门大学计算机与信息科学系）；School of Automation, China University of Geosciences（中国地质大学自动化学院）；Department of Electromechanical Engineering, University of Macau（澳门大学机电工程系）
MoCTEFuse: Illumination-Gated Mixture of Chiral Transformer Experts for Multi-Level Infrared and Visible Image Fusion
School of Optics and Photonics, Beijing Institute of Technology（光学与光电学院，北京理工大学）；School of Computer Science and Technology, Beijing Institute of Technology（计算机科学与技术学院，北京理工大学）
University of Oulu（奥卢大学）；Beijing University of Posts and Telecommunications（北京邮电大学）
Dynamic texture and scene classification are two fundamental problems in understanding natural video content. Extracting robust and effective features is a crucial step towards solving these problems. However the existing approaches suffer from the sensitivity to either varying illumination, or viewpoint changing, or even camera motion, and/or the lack of spatial information. Inspired by the success of deep structures in image classification, we attempt to leverage a deep structure to extract feature for dynamic texture and scene classification. To tackle with the challenges in training a deep…
