Deep models are capable of fitting complex high dimensional functions while usually yielding large computation load. There is no way to speed up the inference process by classical lookup tables due to the high-dimensional input and limited memory size. Recently, a novel architecture (PointNet) for point clouds has demonstrated that it is possible to obtain a complicated deep function from a set of 3-variable functions. In this paper, we exploit this property and apply a lookup table to encode these 3-variable functions. This method ensures that the inference time is only determined by the memory access no matter how complicated the deep function is. We conduct extensive experiments on ModelNet and ShapeNet datasets and demonstrate that we can complete the inference process in 1.5 ms on an Intel i7-8700 CPU (single core mode), 32x speedup over the PointNet architecture without any performance degradation.
School of Electronics and Communication Engineering, Shenzhen Campus of Sun Yat-sen University（中山大学电子与通信工程学院深圳校区）；Aviation University of Air Force（空军航空大学）；College of Integrated Circuits, Shenzhen Campus of Sun Yat-sen University（中山大学集成电路学院深圳校区）；University of Tokyo（东京大学）；College of Electronic Science and Technology, National University of Defense Technology（国防科学技术大学电子科学与技术学院）
Low-cardinality activations permit an algorithm based on fetching the inference values from pre-calculated lookup tables instead of calculating them every time. This algorithm can have extensions, some of which offer abilities beyond those of the currently used algorithms. It also allows for a simpler and more effective CNN-specialized hardware.
The histogram of oriented gradients (HOG) is a widely used feature descriptor in computer vision for the purpose of object detection. In the paper, a modified HOG descriptor is described, it uses a lookup table and the method of integral image to speed up the detection performance by a factor of 5~10. By exploiting the special hardware features of a given platform(e.g. a digital signal processor), further improvement can be made to the HOG descriptor in order to have real-time object detection and tracking.
Optimizing 4D Lookup Table for Low-light Video Enhancement via Wavelet Priori
College of Computer Science and Engineering, Chongqing University of Technology（重庆理工大学计算机科学与工程学院）；Chinese University of Hong Kong（香港中文大学）；College of Computer Science, Chongqing University（重庆大学计算机学院）
UHD-MFF: Shattering Barriers in Multi-Focus Ultra-High-Definition Image Fusion via Learnable Lookup Tables
Electronic Information School, Wuhan University（武汉大学电子信息学院）；School of Robotics, Wuhan University（武汉大学机器人学院）；School of Automation, Southeast University（东南大学自动化学院）
Fast Cubical Persistent Homology on 2D and 3D Images via Union-Find, Pruning, and Lookup Tables
Helmholtz AI（海德堡人工智能研究所）；Helmholtz Munich（海德堡慕尼黑研究所）；École des Ponts ParisTech（巴黎科技大学）；ENS Paris-Saclay（巴黎-萨克勒大学）；Institute of AI for Health, Helmholtz Munich（健康人工智能研究所，海德堡慕尼黑研究所）
Recently, deep learning-based image enhancement algorithms achieved state-of-the-art (SOTA) performance on several publicly available datasets. However, most existing methods fail to meet practical requirements either for visual perception or for computation efficiency, especially for high-resolution images. In this paper, we propose a novel real-time image enhancer via learnable spatial-aware 3-dimentional lookup tables(3D LUTs), which well considers global scenario and local spatial information. Specifically, we introduce a light weight two-head weight predictor that has two outputs. One is …
LoopLUT: 3D Lookup Tables with Progressive Region Refinement for Real-Time 4K Image Enhancement
Universiti Sains Malaysia（马来西亚理科大学）；National University of Defense Technology（国防科技大学）；Sun Yat-sen University（中山大学）；Shandong Normal University（山东师范大学）；Shandong University of Finance and Economics（山东财经大学）
Lightweight and Fast Real-time Image Enhancement via Decomposition of the Spatial-aware Lookup Tables
IPAI, Seoul National University, Seoul, Korea（IPAI，首尔国立大学，首尔，韩国）；LG Electronics, Seoul, Korea（LG电子，首尔，韩国）；Department of ECE, INMC, Seoul National University, Seoul, Korea（电子工程系，INMC，首尔国立大学，首尔，韩国）
Image-adaptive lookup tables (LUTs) have achieved great success in real-time image enhancement tasks due to their high efficiency for modeling color transforms. However, they embed the complete transform, including the color component-independent and the component-correlated parts, into only a single type of LUTs, either 1D or 3D, in a coupled manner. This scheme raises a dilemma of improving model expressiveness or efficiency due to two factors. On the one hand, the 1D LUTs provide high computational efficiency but lack the critical capability of color components interaction. On the other, th…
Sobey Digital Technology Co., Ltd（索贝数码科技股份有限公司）；Peng Cheng Laboratory（鹏城实验室）
Video photorealistic style transfer is desired to generate videos with a similar photorealistic style to the style image while maintaining temporal consistency. However, existing methods obtain stylized video sequences by performing frame-by-frame photorealistic style transfer, which is inefficient and does not ensure the temporal consistency of the stylized video. To address this issue, we use neural network-based 3D Lookup Tables (LUTs) for the photorealistic transfer of videos, achieving a balance between efficiency and effectiveness. We first train a neural network for generating photoreal…
Shanghai Jiao Tong University（上海交通大学）；Alibaba Group（阿里巴巴集团）；Dalian University of Technology（大连理工大学）
The 3D Lookup Table (3D LUT) is a highly-efficient tool for real-time image enhancement tasks, which models a non-linear 3D color transform by sparsely sampling it into a discretized 3D lattice. Previous works have made efforts to learn image-adaptive output color values of LUTs for flexible enhancement but neglect the importance of sampling strategy. They adopt a sub-optimal uniform sampling point allocation, limiting the expressiveness of the learned LUTs since the (tri-)linear interpolation between uniform sampling points in the LUT transform might fail to model local non-linearities of the…
Microsoft Research Asia（微软亚洲研究院）；Xi’an Jiaotong University（西安交通大学）；Microsoft Research（微软研究院）
Image enhancement aims at improving the aesthetic visual quality of photos by retouching the color and tone, and is an essential technology for professional digital photography. Recent years deep learning-based image enhancement algorithms have achieved promising performance and attracted increasing popularity. However, typical efforts attempt to construct a uniform enhancer for all pixels' color transformation. It ignores the pixel differences between different content (e.g., sky, ocean, etc.) that are significant for photographs, causing unsatisfactory results. In this paper, we propose a no…
The Hong Kong Polytechnic University（香港理工大学）；DJI Innovations Co., Ltd（大疆创新科技有限公司）
Recent years have witnessed the increasing popularity of learning based methods to enhance the color and tone of photos. However, many existing photo enhancement methods either deliver unsatisfactory results or consume too much computational and memory resources, hindering their application to high-resolution images (usually with more than 12 megapixels) in practice. In this paper, we learn image-adaptive 3-dimensional lookup tables (3D LUTs) to achieve fast and robust photo enhancement. 3D LUTs are widely used for manipulating color and tone of photos, but they are usually manually tuned and …
