Open Semantic Mapping (OSM) is a key technology in robotic perception, combining semantic segmentation and SLAM techniques. This paper introduces a dynamically configurable and highly automated LLM/LVLM-powered pipeline for evaluating OSM solutions called OSMa-Bench (Open Semantic Mapping Benchmark). The study focuses on evaluating state-of-the-art semantic mapping algorithms under varying indoor lighting conditions, a critical challenge in indoor environments. We introduce a novel dataset with simulated RGB-D sequences and ground truth 3D reconstructions, facilitating the rigorous analysis of mapping performance across different lighting conditions. Through experiments on leading models such as ConceptGraphs, BBQ, and OpenScene, we evaluate the semantic fidelity of object recognition and segmentation. Additionally, we introduce a Scene Graph evaluation method to analyze the ability of models to interpret semantic structure. The results provide insights into the robustness of these models, forming future research directions for developing resilient and adaptable robotic systems. Project page is available at https://be2rlab.github.io/OSMa-Bench/.
MV-CoLight: Efficient Object Compositing with Consistent Lighting and Shadow Generation
Shanghai Jiao Tong University（上海交通大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Nanjing University（南京大学）；The Chinese University of Hong Kong（香港中文大学）；University of Science and Technology of China（中国科学技术大学）；The University of Hong Kong（香港大学）
Train Offline, Test Online: A Real Robot Learning Benchmark
Carnegie Mellon University（卡内基梅隆大学）；Meta AI；University of California, Berkeley（加州大学伯克利分校）；University of Washington（华盛顿大学）；New York University（纽约大学）；Stanford University（斯坦福大学）
Lumen: Consistent Video Relighting and Harmonious Background Replacement with Video Generative Models
Peking University（北京大学）；Kunbyte AI（昆字科技）；University of Chinese Academy of Sciences（中国科学院大学）；Zhejiang University（浙江大学）；Hangzhou Normal University（杭州师范大学）
Institute for Intelligent Systems, Esslingen University of Applied Sciences（埃斯林根应用科学大学智能系统研究所）；Department of Computer Science, University of Freiburg（弗莱堡大学计算机科学系）
Single Document Image Highlight Removal via A Large-Scale Real-World Dataset and A Location-Aware Network
College of Computer Science, Jilin University（吉林大学计算机科学学院）；Institute of Information Science, Academia Sinica（中国科学院信息科学研究院）；National Yang Ming Chiao Tung University（阳明交通大学）；National Taiwan University（台湾大学）
Institute for Intelligent Systems, Esslingen University of Applied Sciences（智能系统研究所，埃斯林根应用科学大学）；Department of Computer Science, University of Freiburg（计算机科学系，弗赖堡大学）；ANDREAS STIHL AG & Co. KG（安德烈亚斯·施塔尔有限公司）
CeyMo: See More on Roads -- A Novel Benchmark Dataset for Road Marking Detection
University of Moratuwa（莫拉图瓦大学）
在本文中，我们介绍了一种新的道路标记基准数据集的道路标记检测，解决现有的公开可用的数据集的局限性，如缺乏挑战性的情况下，突出车道标记，评估脚本不可用，缺乏注释格式和较低的分辨率。我们的数据集由2887张图像组成，其中4706个道路标记实例属于11个类别。这些图像具有1920 x 1080的高分辨率，可以捕捉各种交通、照明和天气条件。我们在多边形、边界框和像素级分割掩码中提供道路标记注释，以促进各种道路标记检测算法。我们提供的评估指标和评估脚本，将进一步促进直接比较新的道路标记检测方法与现有的方法。此外，我们评估使用实例分割和目标检测为基础的道路标记检测任务的方法的有效性。两个实例分割模型和两个对象检测器模型的速度和准确性得分作为我们的基准数据集的性能基线。数据集和评估脚本可在https://github.com/oshadajay/CeyMo上公开获得。
HG-Lane: High-Fidelity Generation of Lane Scenes under Adverse Weather and Lighting Conditions without Re-annotation
Shanghai Jiao Tong University（上海交通大学）；School of Artificial Intelligence, Henan University（河南大学人工智能学院）；University of Science and Technology of China（中国科学技术大学）；AnnLab, Institute of Semiconductors, Chinese Academy of Sciences（中国科学院半导体研究所 AnnLab）；Nanyang Technological University（南洋理工大学）
BALTIC: A Benchmark and Cross-Domain Strategy for 3D Reconstruction Across Air and Underwater Domains Under Varying Illumination
School of Engineering & Physical Sciences, Heriot-Watt University（赫瑞瓦德大学工程与物理科学学院）；Marine Data Science Group, Christian Albrechts University of Kiel（基尔大学海洋数据科学组）；Frontier Robotics, The National Robotarium（前沿机器人，国家机器人中心）
Deep Depth Estimation from Thermal Image: Dataset, Benchmark, and Challenges
School of Computer Science, Carnegie Mellon University（卡内基梅隆大学计算机科学学院）；School of Computer Science and Engineering, Pusan National University（釜山国立大学计算机科学与工程学院）
RAD: A Realistic Multi-View Benchmark for Pose-Agnostic Anomaly Detection
Massachusetts Institute of Technology（麻省理工学院）；Peking University（北京大学）；Carnegie Mellon University（卡内基梅隆大学）；Great Bay University（大湾大学）；Harvard University（哈佛大学）；Tsinghua University（清华大学）；Nanjing University（南京大学）；Deakin University（德克萨斯大学）
ENC-Bench: A Benchmark for Evaluating Multimodal Large Language Models in Electronic Navigational Chart Understanding
National University of Defense Technology（国防科技大学）；Intelligent Game and Decision Lab（智能游戏与决策实验室）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
