Video captioning aims to generate natural language sentences that describe the given video accurately. Existing methods obtain favorable generation by exploring richer visual representations in encode phase or improving the decoding ability. However, the long-tailed problem hinders these attempts at low-frequency tokens, which rarely occur but carry critical semantics, playing a vital role in the detailed generation. In this paper, we introduce a novel Refined Semantic enhancement method towards Frequency Diffusion (RSFD), a captioning model that constantly perceives the linguistic representation of the infrequent tokens. Concretely, a Frequency-Aware Diffusion (FAD) module is proposed to comprehend the semantics of low-frequency tokens to break through generation limitations. In this way, the caption is refined by promoting the absorption of tokens with insufficient occurrence. Based on FAD, we design a Divergent Semantic Supervisor (DSS) module to compensate for the information loss of high-frequency tokens brought by the diffusion process, where the semantics of low-frequency tokens is further emphasized to alleviate the long-tailed problem. Extensive experiments indicate that RSFD outperforms the state-of-the-art methods on two benchmark datasets, i.e., MSR-VTT and MSVD, demonstrate that the enhancement of low-frequency tokens semantics can obtain a competitive generation effect. Code is available at https://github.com/lzp870/RSFD.
Re$^3$Cap: Retrieval-Guided Refinement for Image Captioning Enhancement via Reinforcement Learning
Taobao & Tmall Group of Alibaba(阿里巴巴淘宝和天猫集团);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)
OVC-Net: Object-Oriented Video Captioning with Temporal Graph and Detail Enhancement
School of Artificial Intelligence Beijing Univerisity of Posts\ Telecommunications Beijing, China Email;Department of Electrical;School of Artificial Intelligence Beijing Univerisity of Posts
Pseudo-labeling with Keyword Refining for Few-Supervised Video Captioning
School of Computer Science and Technology, Hangzhou Dianzi University(杭州电子科技大学计算机科学与技术学院);School of Software Technology, Zhejiang University(浙江大学软件学院);College of Computer Science, Zhejiang University(浙江大学计算机学院)
Player-Centric Multimodal Prompt Generation for Large Language Model Based Identity-Aware Basketball Video Captioning
Beijing University of Technology(北京理工大学);Shanghai Jiao Tong University(上海交通大学);Chinese Academy of Sciences(中国科学院);The Hong Kong Polytechnic University(香港理工大学)
M-A-P;Interdisciplinary Centre for Security, Reliability and Trust (SnT), Université du Luxembourg(卢森堡大学安全、可靠性与跨学科中心(SnT));University of Manchester(曼彻斯特大学);Zhejiang University(浙江大学);The University of New South Wales(新南威尔士大学);USTC(中国科学技术大学);Dartmouth College(达特茅斯学院);Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);NTU(南洋理工大学)
Revisiting CroPA: A Reproducibility Study and Enhancements for Cross-Prompt Adversarial Transferability in Vision-Language Models
Mehta Family School of Data Science and Artificial Intelligence(梅hta家族数据科学与人工智能学院);Indian Institute of Technology, Roorkee(印度理工学院罗奥克学院);Department of Civil Engineering(土木工程系);Department of Electronics and Communication(电子与通信系)
Mask Approximation Net: A Novel Diffusion Model Approach for Remote Sensing Change Captioning
School of Computer Science and Technology and the Ministry of Education Key Lab for Intelligent Networks and Network Security, Xi’an Jiaotong University(计算机科学与技术学院和教育部智能网络与网络安全重点实验室,西安交通大学);Aerospace Information Research Institute, Chinese Academy of Sciences(航天信息研究所,中国科学院);Space Engineering University(航天工程大学);School of Mathematics and Statistics, Guangdong University of Technology(数学与统计学院,广东工业大学);Helmholtz-Zentrum Dresden-Rossendorf(德累斯顿-罗斯托克亥姆霍兹中心)
UBLLIE: Unified Backlight and Low-Light Image Enhancement
King Fahd University of Petroleum and Minerals(法赫德国王石油与矿产大学);Ontario Tech University(安大略理工大学);University of Western Australia(西澳大利亚大学);University of Canberra(堪培拉大学)
Curvelet-Based Frequency-Aware Feature Enhancement for Deepfake Detection
Department of Computer Science, College of Science, University of Zakho(扎赫大学计算机科学系);PRIME Lab, Scientific Research Center, University of Zakho(扎赫大学PRIME实验室)
A Non-Uniform Low-Light Image Enhancement Method with Multi-Scale Attention Transformer and Luminance Consistency Loss
Beijing University of Posts and Telecommunications(北京邮电大学);China Electric Power Research Institute Company Limited(中国电力科学研究院有限公司);Tianjin University(天津大学);State Grid Shanxi Marketing Service Center(国网山西省电力公司营销服务中心)
UGAE: Unified Geometry and Attribute Enhancement for G-PCC Compressed Point Clouds
School of Control Science and Engineering, Shandong University(控制科学与工程学院,山东大学);Key Laboratory of Machine Intelligence and System Control, Ministry of Education(机器智能与系统控制重点实验室,教育部);School of Engineering and Sustainable Development, De Montfort University(工程与可持续发展学院,德蒙福特大学);Institute of NUIST-MetaX for Graphics Processing Unit, Nanjing University of Information Science and Technology(信息科学技术大学NUIST-MetaX图形处理单元研究所)
Boosting Fidelity for Pre-Trained-Diffusion-Based Low-Light Image Enhancement via Condition Refinement
Department of Computer Science and Engineering, The Chinese University of Hong Kong(计算机科学与工程系,香港中文大学);Snap Research;HKUST (GZ)(香港科技大学(广州));Alibaba Tongyi Lab(阿里云联合实验室);camera group of DJI(大疆创新相机团队);The University of Hong Kong(香港大学);Sun Yat-Sen University(中山大学)