Following the burgeoning interest in implicit neural representation, Neural Light Field (NeLF) has been introduced to predict the color of a ray directly. Unlike Neural Radiance Field (NeRF), NeLF does not create a point-wise representation by predicting color and volume density for each point in space. However, the current NeLF methods face a challenge as they need to train a NeRF model first and then synthesize over 10K views to train NeLF for improved performance. Additionally, the rendering quality of NeLF methods is lower compared to NeRF methods. In this paper, we propose G-NeLF, a versatile grid-based NeLF approach that utilizes spatial-aware features to unleash the potential of the neural network's inference capability, and consequently overcome the difficulties of NeLF training. Specifically, we employ a spatial-aware feature sequence derived from a meticulously crafted grid as the ray's representation. Drawing from our empirical studies on the adaptability of multi-resolution hash tables, we introduce a novel grid-based ray representation for NeLF that can represent the entire space with a very limited number of parameters. To better utilize the sequence feature, we design a lightweight ray color decoder that simulates the ray propagation process, enabling a more efficient inference of the ray's color. G-NeLF can be trained without necessitating significant storage overhead and with the model size of only 0.95 MB to surpass previous state-of-the-art NeLF. Moreover, compared with grid-based NeRF methods, e.g., Instant-NGP, we only utilize one-tenth of its parameters to achieve higher performance. Our code will be released upon acceptance.
Cosh-DiT: Co-Speech Gesture Video Synthesis via Hybrid Audio-Visual Diffusion Transformers
King Abdullah University of Science and Technology(阿卜杜拉国王科技大学);Baidu(百度);Tokyo Institute of Technology(东京工业大学);Tsinghua University(清华大学);University of Science and Technology of China(中国科学技术大学);Nanyang Technological University(南洋理工大学)
PoseCrafter: Extreme Pose Estimation with Hybrid Video Synthesis
School of Computer Science, Northwestern Polytechnical University(西北工业大学计算机学院);School of Computing, National University of Singapore(新加坡国立大学计算机学院);School of Computing and Data Science, The University of Hong Kong(香港大学计算机与数据科学学院);School of Astronautics, Northwestern Polytechnical University(西北工业大学航天学院)
HAGAN: Hybrid Augmented Generative Adversarial Network for Medical Image Synthesis
Beijing University of Posts and Telecommunications(北京邮电大学);Dalian University of Technology(大连理工大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Intelligence Science and Technology, Nanjing University(南京大学智能科学与技术学院)
HDL: Hybrid Deep Learning for the Synthesis of Myocardial Velocity Maps in Digital Twins for Cardiac Analysis
National Heart and Lung Institute, Imperial College London(帝国理工学院国家心肺研究所);Royal Brompton Hospital(皇家布朗普顿医院);University of the Basque Country UPV/EHU(巴斯克大学);TECNALIA, Basque Research and Technology Alliance(巴斯克研究与技术联盟TECNALIA);Beihang University(北京航空航天大学);Shenzhen Second People’s Hospital(深圳市第二人民医院)
HC$^3$L-Diff: Hybrid conditional latent diffusion with high frequency enhancement for CBCT-to-CT synthesis
National University of Singapore(新加坡国立大学);Central South University(中南大学);National Cancer Centre Singapore(新加坡国家癌症中心);Duke-NUS Medical School(杜克-新加坡国立大学医学院);Nanyang Technological University(南洋理工大学);The Institute for Digital Medicine (WisDM), National University of Singapore(新加坡国立大学数字医学研究所(WisDM));The N.1 Institute for Health (N.1), National University of Singapore(新加坡国立大学N.1健康研究所);Yong Loo Lin School of Medicine, National University of Singapore(新加坡国立大学杨潞龄医学院);The Bia-Echo Asia Centre for Reproductive Longevity and Equality (ACRLE), National University of Singapore(新加坡国立大学Bia-Echo亚洲生殖长寿与平等中心);College of Design and Engineering, National University of Singapore(新加坡国立大学设计与工程学院)
From Satellite to Street: A Hybrid Framework Integrating Stable Diffusion and PanoGAN for Consistent Cross-View Synthesis
Department of Information and Computer Science, King Fahd University of Petroleum and Minerals(信息与计算机科学系,国王法赫德石油与矿物大学);NCMI, CSIRO(CSIRO国家科学机构);Department of Computer Science, National University of Computer and Emerging Sciences (FAST-NUCES)(计算机科学系,国家计算机与新兴科学大学(FAST-NUCES));Faculty of Engineering and IT, University of Technology Sydney(工程与信息技术学院,技术大学悉尼);Faculty of Science, University of Ontario Institute of Technology(科学学院, Ontario Institute of Technology大学)
针对卫星到街景跨视角合成中的视角与外观差异,本文提出融合 Stable Diffusion、条件 GAN 与双分支多阶段训练的混合框架,在 CVUSA 上提升几何一致性与视觉真实感。