An Efficient COarse-to-fiNE Alignment Framework @ Ego4D Natural Language Queries Challenge 2022
City University of Hong Kong(香港城市大学);Microsoft Research Asia(微软亚洲研究院);National University of Singapore(新加坡国立大学);Beihang University(北京航空航天大学);Singapore Management University(新加坡管理大学);Sun Yat-sen University(中山大学)
Zhijian Hou, Wanjun Zhong, Lei Ji, Difei Gao, Kun Yan, Wing-Kwong Chan, Chong-Wah Ngo, Zheng Shou, Nan Duan
备注:
Technical report for ECCV 2022 Ego4D workshop, 4 pages, 2 figures, 2 tables. arXiv admin note: substantial text overlap with arXiv:2209.10918
This technical report describes the CONE approach for Ego4D Natural Language Queries (NLQ) Challenge in ECCV 2022. We leverage our model CONE, an efficient window-centric COarse-to-fiNE alignment framework. Specifically, CONE dynamically slices the long video into candidate windows via a sliding window approach. Centering at windows, CONE (1) learns the inter-window (coarse-grained) semantic variance through contrastive learning and speeds up inference by pre-filtering the candidate windows relevant to the NL query, and (2) conducts intra-window (fine-grained) candidate moments ranking utilizing the powerful multi-modal alignment ability of the contrastive vision-text pre-trained model EgoVLP. On the blind test set, CONE achieves 15.26 and 9.24 for R1@IoU=0.3 and R1@IoU=0.5, respectively.
BRIDGE: Multimodal-to-Text Retrieval via Reinforcement-Learned Query Alignment
High institute for computer & information systems(高等计算机与信息系统学院);Chungbuk National University(忠北大学);Assiut University(艾斯尤特大学);University of Innsbruck(因斯布鲁克大学)
RealignDiff: Boosting Text-to-Image Diffusion Model with Coarse-to-fine Semantic Re-alignment
Pengcheng Laboratory(鹏城实验室);Sun Yat-sen University(中山大学);Australian Artificial Intelligence Institute, University of Technology Sydney(悉尼科技大学澳大利亚人工智能研究所);College of Information Technology (CIT), United Arab Emirates University (UAEU)(阿联酋大学信息技术学院)
CONE: An Efficient COarse-to-fiNE Alignment Framework for Long Video Temporal Grounding
City University of Hong Kong(香港城市大学);Microsoft Research Asia(微软亚洲研究院);National University of Singapore(新加坡国立大学);Beihang University(北京航空航天大学);Singapore Management University(新加坡管理大学);Sun Yat-sen University(中山大学)
Investigating Fine- and Coarse-grained Structural Correspondences Between Deep Neural Networks and Human Object Image Similarity Judgments Using Unsupervised Alignment
Graduate School of Arts and Science, University of Tokyo(东京大学艺术与科学研究生院)
Coarse-Fine View Attention Alignment-Based GAN for CT Reconstruction from Biplanar X-Rays
Beijing United Imaging Research Institute of Intelligent Imaging(北京联影智能影像研究院);United Imaging Intelligence Beijing(联影智能(北京));Peking University Third Hospital(北京大学第三医院);Beijing University of Posts and Telecommunications(北京邮电大学)
Focus Your Distribution: Coarse-to-Fine Non-Contrastive Learning for Anomaly Detection and Localization
SenseTime Research(商汤科技研究院);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);University of California, Los Angeles(加州大学洛杉矶分校)
Cross-domain Object Detection through Coarse-to-Fine Feature Adaptation
Beihang University(北京航空航天大学);Beijing Advanced Innovation Center for Big Data and Brain Computing(大数据与脑计算北京高等创新中心);State Key Laboratory of Software Development Environment(软件开发环境国家重点实验室);School of Computer Science and Engineering(计算机学院)
A Modified Cross Correlation Algorithm for Reference-free Image Alignment of Non-Circular Projections in Single-Particle Electron Microscopy
Department of Mechanical Engineering, University of Texas at Dallas Richardson, TX, 75080, USA(德克萨斯大学达拉斯分校机械工程系);Department of Mechanical Engineering, Johns Hopkins University Baltimore, MD, 21218, USA(约翰霍普金斯大学机械工程系)
Back into Plato's Cave: Examining Cross-modal Representational Convergence at Scale
UC Berkeley(伯克利大学);Technical University Munich, MCML(慕尼黑技术大学);University of Tübingen, Tübingen AI Center(图宾根大学);Toyota Technical Institute at Chicago(芝加哥丰田技术研究所)
Weakly-Supervised 3D Visual Grounding based on Visual Language Alignment
College of Computer Science and Software Engineering, Shenzhen University(深圳大学计算机科学与软件工程学院);Meituan Inc.(美团公司);College of Electronics and Information Engineering, Shenzhen University(深圳大学电子与信息工程学院);Guangdong Key Laboratory of Intelligent Information Processing(广东省智能信息处理重点实验室)
Remote Sensing Large Vision-Language Model: Semantic-augmented Multi-level Alignment and Semantic-aware Expert Modeling
Integrated Vision and Language Lab., School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST)(整合视觉与语言实验室,电气工程学院,韩国科学技术院(KAIST))
FINER: MLLMs Hallucinate under Fine-grained Negative Queries
Technical University of Munich(慕尼黑技术大学);Munich Center for Machine Learning(慕尼黑机器学习中心);Helmholtz Munich(海德堡-慕尼黑亥姆霍尔茨中心);Google(谷歌);LTCI, Télécom Paris, Institut Polytechnique de Paris(LTCI,巴黎电信学院,巴黎理工学院)