arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.07064cs.CV
收藏

OmniFysics:通过多模态信号处理和网络优化实现物理智能进化

OmniFysics: Towards Physical Intelligence Evolution via Omni-Modal Signal Processing and Network Optimization

College of Intelligent Robotics and Advanced Manufacturing, Fudan University(复旦大学智能机器人与先进制造学院);Fysics AI

Minghao Han, Dingkang Yang, Yue Jiang, Yizhou Liu, Lihua Zhang

AI总结:

OmniFysics通过统一图像、音频、视频和文本的多模态信号处理,结合动态物理数据引擎和物理知识注入,实现网络化AI系统的自主优化,提升物理理解能力。

摘要:

网络化AI系统的自主进化依赖于可靠的环境感知,但当前模型的物理理解仍脆弱,因为关键物理信号在大规模数据中视觉模糊且稀疏。为此,我们提出了OmniFysics,一个紧凑的多模态网络,统一处理图像、音频、视频和文本的信号处理与理解。通过构建动态物理数据引擎,FysicsAny通过层次化检索和物理定律约束的信号验证,生成物理基础的监督。同时,FysicsOmniCap利用高级音频视觉跨模态信号处理,从网络视频中提炼高保真数据对,强调动态物理线索。通过分阶段多模态对齐和进化指令微调优化OmniFysics网络,整合潜在空间流匹配生成和自适应意图路由器。实验表明,这种进化优化范式不仅在标准多模态基准上取得竞争性表现,还显著提升了物理导向的评估。

英文摘要:

The autonomous evolution of networked AI systems relies heavily on robust environmental perception. However, physical understanding remains brittle in current models because key physical signals are visually ambiguous and sparsely represented in web-scale data. To bridge the gap between data-centric learning and knowledge-based physical rules, we present OmniFysics, a compact omni-modal network that unifies signal processing and understanding across images, audio, video, and text. To enable autonomous optimization and inject explicit physical knowledge, we construct a dynamic physical data engine. Within this engine, FysicsAny acts as an adaptive mechanism that produces physics-grounded supervision by mapping salient objects to verified physical attributes via hierarchical retrieval and physics-law-constrained signal verification. Concurrently, FysicsOmniCap distills web videos utilizing advanced audio-visual cross-modal signal processing, generating high-fidelity data pairs that emphasize dynamic physical cues. We optimize the OmniFysics network through staged multimodal alignment and evolutive instruction tuning, integrating latent-space flow matching for generation and an adaptive intent router for efficient execution. Experiments demonstrate that this evolutive optimization paradigm not only achieves competitive performance on standard multimodal benchmarks but also significantly advances physics-oriented evaluations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑