发表机构
Sony Group Corporation（索尼集团公司）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
Passing是一个交互式视听装置，通过将单轨电车录像重建为时空体并沿非线性轨迹重采样，生成无尽旅程；利用观众存在检测影响视频转换，并由AI模型SpecMaskFoley生成同步声音，探讨人类意图、机器推理与观众解读之间的创作能动性协商。
AI 中文摘要
本文介绍了Passing，一个交互式视听装置，通过将一段连续的单轨电车窗外录像重建为时空体，从而生成一段无尽的旅程。该作品并非线性地回放录像，而是沿着非线性轨迹重新采样其空间和时间结构，产生一个不断掠过的景观，其深度、速度和时序变得不稳定。一个基于摄像头的观众存在检测系统估计观众是否在观看区域内，并利用这一存在状态来影响渲染视频序列之间的转换。生成的视频流被输入到SpecMaskFoley中，这是一个实时视频到音频的合成模型，为重新配置的图像生成同步的声音景观。该模型并非用于重建客观正确的音轨，而是作为一个推测性的听众，为一个其常规空间和时间前提已被破坏的世界提出一种可能的听觉解读。Passing将创作能动性分配给了艺术家（他定义了时空重建的规则）、AI模型（它将涌现的视觉流解释为声音）以及观众（其具身存在影响着视听轨迹）。通过这种结构，作品探讨了作者身份和聆听如何在人类意图、机器推理和观众解读之间进行协商。作品页面：https://ryufurusawa.com/passing
英文摘要
This paper introduces Passing, an interactive audiovisual installation that generates an endless journey from a single continuous monorail-window recording by reconstructing it as a spatiotemporal volume. Rather than replaying the footage linearly, the work resamples its spatial and temporal structure along nonlinear trajectories, producing a continuously passing landscape whose depth, speed, and temporal order become unstable. A camera-based viewer-presence detection system estimates whether a viewer is present in the viewing zone and uses this presence state to influence transitions among rendered video sequences. The resulting video stream is fed into SpecMaskFoley, a real-time video-to-audio synthesis model that generates a synchronized soundscape for the reconfigured image. The model is not used to reconstruct an objectively correct soundtrack, but functions as a speculative listener, proposing a possible auditory interpretation of a world whose conventional spatial and temporal premises have been disrupted. Passing distributes creative agency across the artist, who defines the rules of spacetime reconstruction; the AI model, which interprets the emergent visual flow as sound; and the audience, whose embodied presence influences the audiovisual trajectory. Through this structure, the work investigates how authorship and listening may be negotiated among human intention, machine inference, and audience interpretation. Artwork page: https://ryufurusawa.com/passing
CommentsAccepted to the NeurIPS 2026 Creative AI Track
