DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Vision
Purdue University(普渡大学);Adobe Inc.(Adobe公司);Rutgers University(罗格斯大学);Google Inc.(谷歌公司);Huazhong University of Science and Technology(华中科技大学);Wormpex AI Research(Wormpex AI研究院)
We have witnessed significant progress in deep learning-based 3D vision, ranging from neural radiance field (NeRF) based 3D representation learning to applications in novel view synthesis (NVS). However, existing scene-level datasets for deep learning-based 3D vision, limited to either synthetic environments or a narrow selection of real-world scenes, are quite insufficient. This insufficiency not only hinders a comprehensive benchmark of existing methods but also caps what could be explored in deep learning-based 3D analysis. To address this critical gap, we present DL3DV-10K, a large-scale scene dataset, featuring 51.2 million frames from 10,510 videos captured from 65 types of point-of-interest (POI) locations, covering both bounded and unbounded scenes, with different levels of reflection, transparency, and lighting. We conducted a comprehensive benchmark of recent NVS methods on DL3DV-10K, which revealed valuable insights for future research in NVS. In addition, we have obtained encouraging results in a pilot study to learn generalizable NeRF from DL3DV-10K, which manifests the necessity of a large-scale scene-level dataset to forge a path toward a foundation model for learning 3D representation. Our DL3DV-10K dataset, benchmark results, and models will be publicly accessible at https://dl3dv-10k.github.io/DL3DV-10K/.
MAPS: A Synthetic Dataset for Probing Vision Models in a Controlled 3D Scene Space
FIAS & Institute of Computer Science Goethe University Frankfurt(FIAS与计算机科学研究所弗赖堡大学);Mila & Department of Biology York University(Mila与生物学系约克大学);Institute of Computer Science Goethe University Frankfurt(计算机科学研究所弗赖堡大学)
Vision-Language Integration for Zero-Shot Scene Understanding in Real-World Environments
Nitte (Deemed to Be University), Nitte Institute of Professional Education(尼特大学(视为大学),尼特专业教育学院);Ballari Institute of Technology and Management(巴拉里技术与管理学院)
Learning with Nested Scene Modeling and Cooperative Architecture Search for Low-Light Vision
Dalian University of Technology(大连理工大学)
低光照场景下拍摄的图像往往会受到可见度低、偏色和噪声等严重影响,这些因素不仅影响图像质量,而且会降低下游低光照视觉(LLV)应用的性能。已经提出了各种深度学习方法来增强低光图像的视觉质量。然而,这些方法大多依赖于重要的建筑工程,以获得适当的低光模型,往往遭受高计算负担。此外,将这些增强技术扩展到处理其他LLV仍然具有挑战性。为了部分解决上述问题,我们建立了Retinex启发的Unrolling with Architecture Search(RUAS),这是一个通用的学习框架,它不仅可以解决弱光增强任务,而且还可以灵活地处理其他更具挑战性的下游视觉应用。具体来说,我们首先建立一个嵌套的优化配方,连同展开策略,探索一系列LLV任务的基本原则。此外,我们构建了一个可区分的策略,以合作搜索特定的场景和任务架构的RUAS。最后但并非最不重要的是,我们演示了如何将RUAS应用于低级和高级LLV应用程序(例如,增强、检测和分割)。大量的实验验证了RUAS的灵活性,有效性和效率。
3D Vision and Language Pretraining with Large-Scale Synthetic Data
Wangxuan Institute of Computer Technology, Peking University(北京大学王选计算机研究所);National Institute of Health Data Science, Peking University(北京大学健康数据科学研究院);National Key Laboratory of General Artificial Intelligence, Peking University(北京大学通用人工智能国家重点实验室);State Key Laboratory of General Artificial Intelligence, BIGAI(北京智源人工智能研究院通用人工智能国家重点实验室)
SceneTAP: Scene-Coherent Typographic Adversarial Planner against Vision-Language Models in Real-World Environments
Agency for Science, Technology and Research (A*STAR)(新加坡科技研究局);College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院);University of Alberta(阿尔伯塔大学);Tianjin University(天津大学)
MVL-Loc: Leveraging Vision-Language Model for Generalizable Multi-Scene Camera Relocalization
School of Automation Science and Engineering, South China University of Technology(自动化科学与工程学院,华南理工大学);Thrust of Artificial Intelligence, The Hong Kong University of Science and Technology (Guangzhou)(人工智能方向,香港科学与技术大学(广州))
SceneBench: A Hierarchical Benchmark for Vision-Language Understanding of 3D Scenes
Nepal Applied Mathematics and Informatics Institute for research (NAAMII)(尼泊尔应用数学与信息学研究所);INSAIT, Sofia University(索非亚大学INSAIT);State University of New York at Stony Brook(纽约州立大学石溪分校)
RE-VLM: Event-Augmented Vision-Language Model for Scene Understanding
School of Artificial Intelligence, Beijing University of Posts and Telecommunications(北京邮电大学人工智能学院);State Key Laboratory of General Artificial Intelligence, BIGAI(通用人工智能国家重点实验室,BIGAI)
Structured Scene Memory for Vision-Language Navigation
Beijing Institute of Technology(北京理工大学);ETH Zurich(苏黎世联邦理工学院);Salesforce Research(Salesforce研究院);Inception Institute of Artificial Intelligence(Inception人工智能研究院)
Why We Look Where We Look: Emergent Human-like Fixations of a Foveated Visual Language Model Maximizing Scene Understanding
Psychological & Brain Sciences, University of California, Santa Barbara(加州大学圣芭芭拉分校心理学与脑科学系);Electrical and Computer Engineering, University of California, Santa Barbara(加州大学圣芭芭拉分校电气与计算机工程系);Computer Science, University of California, Santa Barbara(加州大学圣芭芭拉分校计算机科学系)
TBN-ViT: Temporal Bilateral Network with Vision Transformer for Video Scene Parsing
Ant Group(蚂蚁集团)
随着自动驾驶技术的飞速发展,野外环境下的视频场景解析是一个具有挑战性和重要意义的课题。数据集Video Scene Parsing in the Wild(VSPW)包含修剪良好的长时间,密集注释和高分辨率剪辑。在VSPW的基础上,设计了一个带有视觉Transformer的时间双边网络.我们首先设计了一个空间路径与卷积生成低级别的功能,可以保持空间信息。同时,采用具有视觉Transformer的上下文路径来获得足够的上下文信息。此外,一个时间上下文模块被设计来利用帧间上下文信息。最后,所提出的方法可以实现49.85\%的平均交集超过并集(mIoU)的VSPW2021挑战测试数据集。
Hierarchical Question-Answering for Driving Scene Understanding Using Vision-Language Models
Center for ICT and Automotive Convergence, Kyungpook National University(信息与汽车融合研究中心,庆北国立大学);School of Electronic and Electrical Engineering, Kyungpook National University(电子电气工程学院,庆北国立大学)
JaWildText: A Benchmark for Vision-Language Models on Japanese Scene Text Understanding
Institute of Science Tokyo, Tokyo, Japan(东京科学大学,日本东京);Research and Development Center for Large Language Models, National Institute of Informatics, Tokyo, Japan(国立信息学研究所大型语言模型研发中心,日本东京)
Vision-Language Pre-training with Object Contrastive Learning for 3D Scene Understanding
Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);College of Computer Science and Software Engineering, Shenzhen University(深圳大学计算机与软件学院);Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));Research Center of Artificial Intelligence, Peng Cheng Laboratory(鹏城实验室人工智能研究中心)
From Pixels to Graphs: Open-Vocabulary Scene Graph Generation with Vision-Language Models
School of Information Science and Technology, ShanghaiTech University(上海科技大学信息科学与技术学院);Shanghai AI Laboratory(上海人工智能实验室);Shanghai Engineering Research Center of Intelligent Vision and Imaging(上海智能视觉与成像工程技术研究中心)
Two-Stream Interactive Joint Learning of Scene Parsing and Geometric Vision Tasks
College of Electronic and Information Engineering, Tongji University(同济大学电子信息学院);Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University(同济大学智能自主系统研究所);College of Computer Science and Technology, Tongji University(同济大学计算机科学与技术学院);Department of Vehicle Control System and Software Development, NIO(蔚来汽车车辆控制系统与软件开发部门);Department of Automotive Engineering, Jilin University(吉林大学汽车工程学院);Key Laboratory of Embedded System and Service Computing (Ministry of Education), Tongji University(同济大学嵌入式系统与服务计算重点实验室);College of Electronic and Information Engineering, Shanghai Institute of Intelligent Science and Technology(上海智能科学与技术研究院电子信息学院);Shanghai Key Laboratory of Intelligent Autonomous Systems(上海智能自主系统重点实验室)
The ParallelEye Dataset: Constructing Large-Scale Artificial Scenes for Traffic Vision Research
Beijing Institute of Technology(北京理工大学);The State Key Laboratory for Management and Control of Complex Systems, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所复杂系统管理与控制国家重点实验室);Qingdao Academy of Intelligent Industries(青岛智能产业研究院);University of Science and Technology of China(中国科学技术大学)