在同步和校准的多视图设置中定位3D人体关节的常见方法由两个步骤组成：（1）在每个视图上单独应用2D检测器以定位2D中的关节，以及（2）对来自每个视图的2D检测执行鲁棒三角测量以获取3D关节位置。然而，在步骤1中，2D检测器仅限于解决可能在3D中更好地解决的挑战性情况，例如遮挡和倾斜视角，纯粹在2D中，而不利用任何3D信息。因此，我们提出了可微分的“核线Transformer”，这使得2D检测器能够利用3D感知特征来改善2D姿态估计。直觉是：给定当前视图中的2D位置p，我们希望首先在相邻视图中找到其对应点p'，然后将p'处的特征与p处的特征组合，从而导致p处的3D感知特征。受立体匹配的启发，核线Transformer利用核线约束和特征匹配来近似p '处的特征。在InterHand和Human3.6M上的实验表明，我们的方法在基线上有一致的改进。具体来说，在不使用外部数据的情况下，我们使用ResNet-50主干和图像大小256 x 256训练的Human3.6M模型比最先进的模型性能高出4.23 mm，并达到MPJPE 26.9 mm。
英文摘要：
A common approach to localize 3D human joints in a synchronized and calibrated multi-view setup consists of two-steps: (1) apply a 2D detector separately on each view to localize joints in 2D, and (2) perform robust triangulation on 2D detections from each view to acquire the 3D joint locations. However, in step 1, the 2D detector is limited to solving challenging cases which could potentially be better resolved in 3D, such as occlusions and oblique viewing angles, purely in 2D without leveraging any 3D information. Therefore, we propose the differentiable "epipolar transformer", which enables the 2D detector to leverage 3D-aware features to improve 2D pose estimation. The intuition is: given a 2D location p in the current view, we would like to first find its corresponding point p' in a neighboring view, and then combine the features at p' with the features at p, thus leading to a 3D-aware feature at p. Inspired by stereo matching, the epipolar transformer leverages epipolar constraints and feature matching to approximate the features at p'. Experiments on InterHand and Human3.6M show that our approach has consistent improvements over the baselines. Specifically, in the condition where no external data is used, our Human3.6M model trained with ResNet-50 backbone and image size 256 x 256 outperforms state-of-the-art by 4.23 mm and achieves MPJPE 26.9 mm.
TransFusion: Cross-view Fusion with Transformer for 3D Human Pose Estimation
University of California, Irvine（加州大学欧文分校）；Facebook（脸书）；Amazon（亚马逊）
估计每个视图中的2D人体姿态通常是校准的多视图3D姿态估计中的第一步。但是2D姿态检测器的性能受到诸如遮挡和倾斜视角等具有挑战性的情况的影响。为了解决这些挑战，以前的工作从对极几何中导出不同视图之间的点对点对应关系，并利用这些对应关系来合并预测热图或特征表示。而不是预测后的合并/校准，在这里，我们介绍了一个Transformer框架的多视图3D姿态估计，旨在通过整合来自不同视图的信息，直接提高个人的2D预测。受以前的多模态Transformers的启发，我们设计了一个统一的Transformer架构，名为TransFusion，融合来自当前视图和相邻视图的线索。此外，我们提出了核线场的概念，编码到Transformer模型的三维位置信息。由对极场引导的3D位置编码提供了对不同视图的像素之间的对应性进行编码的有效方式。在Human 3.6M和Ski-Pose上的实验表明，与其他融合方法相比，该方法具有更高的效率和一致的改进。具体来说，我们在Human 3.6M上实现了25.8 mm MPJPE，仅使用256 x 256分辨率的5 M参数。
Exploring Non-Local Spatial-Angular Correlations with a Hybrid Mamba-Transformer Framework for Light Field Super-Resolution
School of Information Science and Engineering, Huaqiao University（华侨大学信息科学与工程学院）；School of Mechanical Engineering and Automation, Huaqiao University（华侨大学机械工程与自动化学院）；School of Engineering, Huaqiao University（华侨大学工程学院）；School of Optoelectronic and Communication Engineering, Xiamen University of Technology（厦门理工大学光电子与通信工程学院）；Artificial Intelligence Institute, Hangzhou Dianzi University（杭州电子科技大学人工智能学院）
TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation
Johns Hopkins University（约翰斯·霍普金斯大学）；University of Electronic Science and Technology of China（电子科技大学）；Stanford University（斯坦福大学）；East China Normal University（华东师范大学）；PAII Inc.（PAII公司）
Severity Quantification and Lesion Localization of COVID-19 on CXR using Vision Transformer
Korea Advanced Institute of Science and Technology（韩国科学技术院）；Asan Medical Center（峨山医院）；University of Ulsan College of Medicine（蔚山大学医学院）；College of Medicine, Chungnam National Univerity（忠南国立大学医学院）；College of Medicine, Yeungnam University（岭南大学医学院）；School of Medicine, Kyungpook National University（庆北国立大学医学院）
Geometric Consistency Protocol for Foundation Model Features in Multi-View Satellite Imagery
Hubei Province Key Research and Development Program（湖北省重点研发计划）；LIESMARS Special Research Funding（测绘遥感信息工程国家重点实验室专项研究基金）；National Science Fund for Distinguished Young Scholars（国家杰出青年科学基金）
CTP: A hybrid CNN-Transformer-PINN model for ocean front forecasting
School of Oceanography, Shanghai Jiao Tong University, China（上海交通大学海洋学院）；State Key Laboratory of Satellite Ocean Environment Dynamics, Second Institute of Oceanography, MNR, China（国家卫星海洋环境动力学重点实验室，第二海洋研究所）；College of Marine Science and Technology, China University of Geosciences, China（中国地质大学（北京）海洋科学与技术学院）
Integrating AI for Human-Centric Breast Cancer Diagnostics: A Multi-Scale and Multi-View Swin Transformer Framework
Concordia Institute for Information Systems Engineering (CIISE)（康科迪亚信息系统工程研究所）；Concordia University（康科迪亚大学）；Thomas Jefferson University Hospital（泰勒斯·杰弗里斯大学医院）；College of Computing & Informatics（计算与信息学院）；Drexel University（德雷塞尔大学）
