To improve the viewer's Quality of Experience (QoE) and optimize computer graphics applications, 3D model quality assessment (3D-QA) has become an important task in the multimedia area. Point cloud and mesh are the two most widely used digital representation formats of 3D models, the visual quality of which is quite sensitive to lossy operations like simplification and compression. Therefore, many related studies such as point cloud quality assessment (PCQA) and mesh quality assessment (MQA) have been carried out to measure the visual quality degradations of 3D models. However, a large part of previous studies utilize full-reference (FR) metrics, which indicates they can not predict the quality level with the absence of the reference 3D model. Furthermore, few 3D-QA metrics consider color information, which significantly restricts their effectiveness and scope of application. In this paper, we propose a no-reference (NR) quality assessment metric for colored 3D models represented by both point cloud and mesh. First, we project the 3D models from 3D space into quality-related geometry and color feature domains. Then, the 3D natural scene statistics (3D-NSS) and entropy are utilized to extract quality-aware features. Finally, machine learning is employed to regress the quality-aware features into visual quality scores. Our method is validated on the colored point cloud quality assessment database (SJTU-PCQA), the Waterloo point cloud assessment database (WPC), and the colored mesh quality assessment database (CMDM). The experimental results show that the proposed method outperforms most compared NR 3D-QA metrics with competitive computational resources and greatly reduces the performance gap with the state-of-the-art FR 3D-QA metrics. The code of the proposed model is publicly available now to facilitate further research.
ONERA（法国航空航天实验室）；Université Paris-Saclay（巴黎萨克雷大学）；University of Toulouse（图卢兹大学）
This paper introduces an Online Localisation and Colored Mesh Reconstruction (OLCMR) ROS perception architecture for ground exploration robots aiming to perform robust Simultaneous Localisation And Mapping (SLAM) in challenging unknown environments and provide an associated colored 3D mesh representation in real time. It is intended to be used by a remote human operator to easily visualise the mapped environment during or after the mission or as a development base for further researches in the field of exploration robotics. The architecture is mainly composed of carefully-selected open-source …
PointDreamer: Zero-shot 3D Textured Mesh Reconstruction from Colored Point Cloud
Huazhong University of Science and Technology（华中科技大学）；School of Computer Science and Technology（计算机科学与技术学院）；Guangdong HUST Industrial Technology Research Institute（广东华中科技大学产业技术研究院）；School of Computer Science and Engineering（计算机科学与工程学院）；South China University of Technology（华南理工大学）；Pazhou Laboratory（琶洲实验室）
Jagiellonian University（雅盖隆大学）；Warsaw University of Technology（华沙理工大学）
Designing a 3D game scene is a tedious task that often requires a substantial amount of work. Typically, this task involves synthesis, coloring, and placement of 3D models within the game scene. To lessen this workload, we can apply machine learning to automate some aspects of the game scene development. Earlier research has already tackled automated generation of the game scene background with machine learning. However, model auto-coloring remains an underexplored problem. The automatic coloring of a 3D model is a challenging task, especially when dealing with the digital representation of a …
Qingdao University（青岛大学）；New York University（纽约大学）；Shandong University（山东大学）；De Montfort University（德蒙福特大学）
With the wide applications of colored point cloud in many fields, point cloud perceptual quality assessment plays a vital role in the visual communication systems owing to the existence of quality degradations introduced in various stages. However, the existing point cloud quality assessments ignore the mechanism of human visual system (HVS) which has an important impact on the accuracy of the perceptual quality assessment. In this paper, a progressive knowledge transfer based on human visual perception mechanism for perceptual quality assessment of point clouds (PKT-PCQA) is proposed. The PKT…
Point cloud is one of the most widely used digital formats of 3D models, the visual quality of which is quite sensitive to distortions such as downsampling, noise, and compression. To tackle the challenge of point cloud quality assessment (PCQA) in scenarios where reference is not available, we propose a no-reference quality assessment metric for colored point cloud based on captured video sequences. Specifically, three video sequences are obtained by rotating the camera around the point cloud through three specific orbits. The video sequences not only contain the static views but also include…
Mobile robots that manipulate their environments require high-accuracy scene understanding at close range. Typically this understanding is achieved with RGBD cameras, but the evaluation process for selecting an appropriate RGBD camera for the application is minimally quantitative. Limited manufacturer-published metrics do not translate to observed quality in real-world cluttered environments, since quality is application-specific. To bridge the gap, we present a method for quantitatively measuring depth quality using a set of extendable 3D printed fixtures that approximate real-world condition…
Perceptual Visual Quality Assessment: Principles, Methods, and Future Directions
Cardiff University（卡迪夫大学）；University of Klagenfurt（克拉根福大学）；Shanghai Jiao Tong University（上海交通大学）；University of Nantes（南特大学）；University of Texas at Austin（德克萨斯大学奥斯汀分校）
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；FZI Research Center for Information Technology（FZI信息技术研究中心）
The Large-scale 3D reconstruction, texturing and semantic mapping are nowadays widely used for automated driving vehicles, virtual reality and automatic data generation. However, most approaches are developed for RGB-D cameras with colored dense point clouds and not suitable for large-scale outdoor environments using sparse LiDAR point clouds. Since a 3D surface can be usually observed from multiple camera images with different view poses, an optimal image patch selection for the texturing and an optimal semantic class estimation for the semantic mapping are still challenging. To address these…
Univ Lyon, CNRS, LIRIS（里昂大学，法国国家科学研究中心，LIRIS）；Multimedia Research Center, University of Alberta（阿尔伯塔大学多媒体研究中心）
Objective visual quality assessment of 3D models is a fundamental issue in computer graphics. Quality assessment metrics may allow a wide range of processes to be guided and evaluated, such as level of detail creation, compression, filtering, and so on. Most computer graphics assets are composed of geometric surfaces on which several texture images can be mapped to 11 make the rendering more realistic. While some quality assessment metrics exist for geometric surfaces, almost no research has been conducted on the evaluation of texture-mapped 3D models. In this context, we present a new subject…
Non-Invasive 3D Wound Measurement with RGB-D Imaging
Institute of Medical Informatics, University of Lübeck（吕贝克大学医学信息学研究所）；Queensland University of Technology（昆士兰技术大学）；Imaging and Computer Vision Group, CSIRO Data61（CSIRO Data61成像与计算机视觉小组）
DHQA-4D: Perceptual Quality Assessment of Dynamic 4D Digital Human
Institute of Image Communication and Network Engineering（图像通信与网络工程研究所）；Shanghai Jiao Tong University（上海交通大学）；USC-SJTU Institute of Cultural and Creative Industry（USC-SJTU文化产业与创意产业研究院）
We present a deep learning pipeline that leverages network self-prior to recover a full 3D model consisting of both a triangular mesh and a texture map from the colored 3D point cloud. Different from previous methods either exploiting 2D self-prior for image editing or 3D self-prior for pure surface reconstruction, we propose to exploit a novel hybrid 2D-3D self-prior in deep neural networks to significantly improve the geometry quality and produce a high-resolution texture map, which is typically missing from the output of commodity-level 3D scanners. In particular, we first generate an initi…
Universitat Pompeu Fabra（庞培法布拉大学）；Normandie Univ（诺曼底大学）；UNICAEN（卡昂大学）；CHU de Caen Normandie（卡昂诺曼底大学医院）；Inserm Comete（法国国家健康与医学研究院Comete实验室）；GIP Cyceron（Cyceron集团利益公共机构）；Philips Research（飞利浦研究院）；ICREA（加泰罗尼亚高等研究院）；IDIBAPS（巴塞罗那生物医学研究所）；KULeuven（鲁汶大学）
3D echocardiography is an increasingly popular tool for assessing cardiac remodelling in the right ventricle (RV). It allows quantification of the cardiac chambers without any geometric assumptions, which is the main weakness of 2D echocardiography. However, regional quantification of geometry and function is limited by the lower spatial and temporal resolution and the scarcity of identifiable anatomical landmarks. We developed a technique for regionally assessing the 3 relevant RV regions: apical, inlet and outflow. The method's inputs are end-diastolic (ED) and end-systolic (ES) segmented 3D…
In this letter, we propose a novel LiDAR-Inertial-Visual sensor fusion framework termed R3LIVE, which takes advantage of measurement of LiDAR, inertial, and visual sensors to achieve robust and accurate state estimation. R3LIVE is contained of two subsystems, the LiDAR-inertial odometry (LIO) and visual-inertial odometry (VIO). The LIO subsystem (FAST-LIO) takes advantage of the measurement from LiDAR and inertial sensors and builds the geometry structure of (i.e. the position of 3D points) global maps. The VIO subsystem utilizes the data of visual-inertial sensors and renders the map's textur…
SEMT: Static-Expansion-Mesh Transformer Network Architecture for Remote Sensing Image Captioning
Austrian Institute of Technology（奥地利技术研究院）；University of Technology of Troyes（图卢兹技术大学）；Ho Chi Minh University of Technology（胡志明技术大学）；Ton Duc Thang University（-ton Duc Thang大学）
