In this paper, we introduce Uni3D-LLM, a unified framework that leverages a Large Language Model (LLM) to integrate tasks of 3D perception, generation, and editing within point cloud scenes. This framework empowers users to effortlessly generate and modify objects at specified locations within a scene, guided by the versatility of natural language descriptions. Uni3D-LLM harnesses the expressive power of natural language to allow for precise command over the generation and editing of 3D objects, thereby significantly enhancing operational flexibility and controllability. By mapping point cloud into the unified representation space, Uni3D-LLM achieves cross-application functionality, enabling the seamless execution of a wide array of tasks, ranging from the accurate instantiation of 3D objects to the diverse requirements of interactive design. Through a comprehensive suite of rigorous experiments, the efficacy of Uni3D-LLM in the comprehension, generation, and editing of point cloud has been validated. Additionally, we have assessed the impact of integrating a point cloud perception module on the generation and editing processes, confirming the substantial potential of our approach for practical applications.
Unifying Speech Editing Detection and Content Localization via Prior-Enhanced Audio LLMs
Key Laboratory of Aerospace Information Security(航空信息安全与可信计算重点实验室);School of Cyber Science and Engineering(网络安全工程学院);Wuhan University(武汉大学);Independent Researcher(独立研究员);School of Computer Science and Technology(计算机科学与技术学院);Anhui University(安徽大学);Communication University of China(中国通信大学);Beihang University(北京航空航天大学)
BMIKE-53: Investigating Cross-Lingual Knowledge Editing with In-Context Learning
LMU Munich(慕尼黑大学);Munich Center for Machine Learning(慕尼黑机器学习中心);Technical University of Munich(慕尼黑技术大学);Bosch Center for Artificial Intelligence(博世人工智能中心);University of Oxford(牛津大学);CISPA Helmholtz Center for Information Security(CISPA海德堡信息安全中心)
UniVLR: Unifying Text and Vision in Visual Latent Reasoning for Multimodal LLMs
University of Science and Technology of China(中国科学技术大学);Tsinghua University(清华大学);National University of Singapore(新加坡国立大学);Zhongguancun Academy(中关村学院)
HealthGPT: A Medical Large Vision-Language Model for Unifying Comprehension and Generation via Heterogeneous Knowledge Adaptation
Zhejiang University(浙江大学);University of Electronic Science and Technology of China(电子科技大学);Alibaba(阿里巴巴);The Hong Kong University of Science and Technology(香港科技大学);National University of Singapore(新加坡国立大学)
The Chinese University of Hong Kong Shanghai AI Laboratory(香港中文大学上海人工智能实验室);Shanghai AI Laboratory(上海人工智能实验室);Shanghai Jiao Tong University(上海交通大学);Shanghai Innovation Institute(上海创新研究院)
From Structure to Synergy: A Survey of Vision-Language Perception Paradigm Evolution in Multimodal Large Language Models
School of Computer Science, Sichuan University(四川大学计算机学院);School of Electronic and Computer Engineering, Peking University Shenzhen Graduate School(北京大学深圳研究生院电子与计算机工程学院);Institute of Artificial Intelligence (TeleAI), China Telecom and Northwestern Polytechnical University(中国电信与西北工业大学人工智能研究院(TeleAI))
A Simple Baseline for Unifying Understanding, Generation, and Editing via Vanilla Next-token Prediction
Key Lab of High Confidence Software Technologies (Peking University), Ministry of Education, China(高可信软件技术重点实验室(北京大学),教育部,中国);School of Computer Science, Peking University, Beijing, China(北京大学计算机学院,北京,中国);School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing, China(中国科学院大学计算机科学与技术学院,北京,中国)
HairCLIPv2: Unifying Hair Editing via Proxy Feature Blending
University of Science and Technology of China(中国科学技术大学);Microsoft Cloud AI(微软云人工智能部门);City University of Hong Kong(香港城市大学);Xi’an Jiaotong University(西安交通大学)
Sensor calibration, which can be intrinsic or extrinsic, is an essential step to achieve the measurement accuracy required for modern perception and navigation systems deployed on autonomous robots. To date, intrinsic calibration models for spinning LiDARs have been based on hypothesized based on their physical mechanisms, resulting in anywhere from three to ten parameters to be estimated from data, while no phenomenological models have yet been proposed for solid-state LiDARs. Instead of going down that road, we propose to abstract away from the physics of a LiDAR type (spinning vs solid-stat…
Pointer-CAD: Unifying B-Rep and Command Sequences via Pointer-based Edges & Faces Selection
Transcengram;Beihang University(北京航空航天大学);The University of Hong Kong(香港大学);Shenzhen Loop Area Institute(深圳河套学院);ShanghaiTech University(上海科技大学);DeepSeek;University of California, Berkeley(加州大学伯克利分校)
Unifying Specialized Visual Encoders for Video Language Models
Department of Computer Science, Princeton University, Princeton, NJ, United States(普林斯顿大学计算机科学系);Salesforce Research, Palo Alto, CA, United States(Salesforce研究)
ETH Zurich(苏黎世联邦理工学院);University of Tübingen(图宾根大学);Sydney AI Center(悉尼人工智能中心);The University of Sydney(悉尼大学);Monash University(莫纳什大学);Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所)
UniHead: Unifying Multi-Perception for Detection Heads
Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);School of Information and Electronics, Beijing Institute of Technology(北京理工大学信息与电子学院);Durham University(杜伦大学);Tencent Jarvis Lab(腾讯杰维斯实验室)
SpatialReward: Bridging the Perception Gap in Online RL for Image Editing via Explicit Spatial Reasoning
Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));The Hong Kong University of Science and Technology(香港科学与技术大学);Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生学院,清华大学)
We present a conceptually simple, flexible, and universal visual perception head for variant visual tasks, e.g., classification, object detection, instance segmentation and pose estimation, and different frameworks, such as one-stage or two-stage pipelines. Our approach effectively identifies an object in an image while simultaneously generating a high-quality bounding box or contour-based segmentation mask or set of keypoints. The method, called UniHead, views different visual perception tasks as the dispersible points learning via the transformer encoder architecture. Given a fixed spatial c…
MooD: Perception-Enhanced Efficient Affective Image Editing via Continuous Valence-Arousal Modeling
School of Cyber Science and Engineering, Zhengzhou University(郑州大学信息科学与工程学院);School of Computer Science and Technology, Xi’an Jiaotong University(西安交通大学计算机科学与技术学院);School of Journalism and New Media, Xi’an Jiaotong University(西安交通大学新闻与传播学院);Research Center for Space Computing System, Zhejiang Lab(浙江实验室空间计算系统研究中心)
Mixture-of-Thought-Tokens: Unifying Perception and Reasoning for Free-form Multimodal Grounding
Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University(西安交通大学人工智能与机器人研究所);Xingchen AGI Lab, China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd(星辰通用人工智能实验室,中国电信人工智能技术(北京)有限公司);University of Science and Technology Beijing(北京科技大学);Beijing University of Posts and Telecommunications(北京邮电大学);Shanghai Jiao Tong University(上海交通大学)
EditRefiner: A Human-Aligned Agentic Framework for Image Editing Refinement
Shanghai Jiao Tong University(上海交通大学);Vivo Mobile Communication Co., Ltd(Vivo移动通信有限公司);University of Electronic Science and Technology of China(电子科学与技术大学)