Despite the success of deep learning in video understanding tasks, processing every frame in a video is computationally expensive and often unnecessary in real-time applications. Frame selection aims to extract the most informative and representative frames to help a model better understand video content. Existing frame selection methods either individually sample frames based on per-frame importance prediction, without considering interaction among frames, or adopt reinforcement learning agents to find representative frames in succession, which are costly to train and may lead to potential stability issues. To overcome the limitations of existing methods, we propose a Search-Map-Search learning paradigm which combines the advantages of heuristic search and supervised learning to select the best combination of frames from a video as one entity. By combining search with learning, the proposed method can better capture frame interactions while incurring a low inference overhead. Specifically, we first propose a hierarchical search method conducted on each training video to search for the optimal combination of frames with the lowest error on the downstream task. A feature mapping function is then learned to map the frames of a video to the representation of its target optimal frame combination. During inference, another search is performed on an unseen video to select a combination of frames whose feature representation is close to the projected feature representation. Extensive experiments based on several action recognition benchmarks demonstrate that our frame selection method effectively improves performance of action recognition models, and significantly outperforms a number of competitive baselines.
SurgPETL: Parameter-Efficient Image-to-Surgical-Video Transfer Learning for Surgical Phase Recognition
Hong Kong University of Science and Technology(香港科技大学);Tongji University(同济大学);Tongji Hospital(同济医院);Tongji Institute of Digestive Disease(同济消化病研究所);HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute(香港科技大学深港协同创新研究院)
TransRank: Self-supervised Video Representation Learning via Ranking-based Transformation Recognition
The Chinese University of HongKong(香港中文大学);Centre of Perceptual and Interactive Intelligence(感知与交互智能中心);University of Bath(巴斯大学);Shanghai AI Laboratory(上海人工智能实验室);SenseTime Research(商汤科技研究院)
Rethinking the Learning Paradigm for Facial Expression Recognition
University of Trento \& Fondazione Bruno Kessler Trento Trento Italy;University of Trento Trento Trento Italy;University of Trento \& Fondazione Bruno Kessler;University of Trento
Noise-Tolerant Paradigm for Training Face Recognition CNNs
Beijing University of Chemical Technology(北京化工大学);Yunshitu Corporation(云视途公司)
得益于大规模训练数据集,深度卷积神经网络(CNN)在人脸识别(FR)方面取得了令人印象深刻的结果。然而,巨大规模的数据集不可避免地导致噪声数据,这明显降低了训练的CNN模型的性能。从大规模FR数据集中踢出错误的标签仍然非常昂贵,尽管提出了一些清理方法。通过对基于角间隔损失(angular margin based loss,AM-Loss)函数监督下的CNN模型训练全过程的分析,发现训练样本的$θ$分布隐含地反映了其干净的概率。因此,我们提出了一种新的训练范式,采用基于上述概率的加权样本的想法。在没有任何噪声先验知识的情况下,我们可以用大规模FR数据集训练高性能的CNN模型。实验证明了我们的训练范式的有效性。这些代码可在https://github.com/huangyangyu/NoiseFace上获得。
A Pseudo Global Fusion Paradigm-Based Cross-View Network for LiDAR-Based Place Recognition
School of Electronics and Information Engineering, and Xingzhi College, South China Normal University(电子信息工程学院和星智学院,华南师范大学);School of Data Science and Engineering, and Xingzhi College, South China Normal University(数据科学与工程学院和星智学院,华南师范大学);College of Intelligence Science and Technology, National University of Defense Technology(智能科学与技术学院,国防科技大学);College of Electronics & Information Engineering, Shanghai Research Institute for Intelligent Autonomous Systems, the State Key Laboratory of Intelligent Autonomous Systems, and Frontiers Science Center for Intelligent Autonomous Systems, Tongji University(电子与信息工程学院,上海智能自主系统研究所,智能自主系统国家重点实验室,智能自主系统前沿科学中心,同济大学);Department of Electronic and Computer Engineering, Hong Kong University of Science and Technology(电子与计算机工程系,香港科技大学)
Privacy-Preserving Human Activity Recognition from Extreme Low Resolution
Indiana University(印第安纳大学);EgoVid Inc.(EgoVid公司);Jet Propulsion Laboratory, California Institute of Technology(加州理工学院喷气推进实验室);Xi’an Jiaotong-Liverpool University(西交利物浦大学);Ulsan National Institute of Science and Technology(蔚山科学技术院)
Efficient Pyramid Channel Attention Network for Pathological Myopia Recognition
Southern University of Science and Technology(南方科技大学);Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);Wenzhou Medical University(温州医科大学);Eye Hospital, Wenzhou Medical University(温州医科大学附属眼视光医院);National Clinical Research Center for Ocular Diseases(国家眼科疾病临床研究中心);State Key Laboratory of Ophthalmology, Optometry and Visual Science(眼视光学与视觉科学国家重点实验室)
UGNA-VPR: A Novel Training Paradigm for Visual Place Recognition Based on Uncertainty-Guided NeRF Augmentation
College of Intelligence Science and Technology, National University of Defense Technology(国防科技大学智能科学学院);Faculty of Robot Science and Engineering, Northeastern University(东北大学机器人科学与工程学院);School of Electronic and Information Engineering, Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳)电子与信息工程学院);Institute of Cyber-Systems and Control, Zhejiang University(浙江大学网络系统与控制研究所);SIASUN Robot& Automation Co., Ltd(新松机器人自动化股份有限公司);School of Computer Science, Fudan University(复旦大学计算机科学技术学院)
On the Element-Wise Representation and Reasoning in Zero-Shot Image Recognition: A Systematic Survey
The Hong Kong Polytechnic University(香港理工大学);The University of Queensland(昆士兰大学);Hong Kong University of Science and Technology(香港科技大学);Alibaba Group(阿里巴巴集团);Nanyang Technological University(南洋理工大学)
TransFace++: Rethinking the Face Recognition Paradigm with a Focus on Accuracy, Efficiency, and Security
Zhejiang University(浙江大学);Wolf 1069 b Lab, Sany Group(沃尔夫1069b实验室,三一集团);Department of Computing, Imperial College London(计算系,帝国理工学院伦敦分校);Department of Engineering, King’s College London(工程系,国王学院伦敦分校)
SequencePAR: Understanding Pedestrian Attributes via A Sequence Generation Paradigm
State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology(光电信息采集与防护技术重点实验室);School of Computer Science and Technology(计算机科学与技术学院);Anhui University(安徽大学);the School of Artificial Intelligence(人工智能学院);Anhui Provincial Key Laboratory of Multimodal Cognitive Computation(安徽省多模态认知计算重点实验室);iFLYTEK AI Research Institute(iFLYTEK人工智能研究院)
DialogueVPR: Towards Conversational Visual Place Recognition
School of Artificial Intelligence, Beijing University of Posts and Telecommunications(北京邮电大学人工智能学院);Shandong Computer Science Center, Qilu University of Technology(山东省计算中心(国家超级计算济南中心),齐鲁工业大学);Macquarie University(麦考瑞大学);Spatialtemporal AI(时空人工智能公司);University of Macau(澳门大学);Aerospace Information Research Institute(航天信息研究所)