Our research investigates the challenges Deep Reinforcement Learning (DRL) faces in complex, Partially Observable Markov Decision Processes (POMDP) such as autonomous driving (AD), and proposes a solution for vision-based navigation in these environments. Partial observability reduces RL performance significantly, and this can be mitigated by augmenting sensor information and data fusion to reflect a more Markovian environment. However, this necessitates an increasingly complex perception module, whose training via RL is complicated due to inherent limitations. As the neural network architecture becomes more complex, the reward function's effectiveness as an error signal diminishes since the only source of supervision is the reward, which is often noisy, sparse, and delayed. Task-irrelevant elements in images, such as the sky or certain objects, pose additional complexities. Our research adopts an offline-trained encoder to leverage large video datasets through self-supervised learning to learn generalizable representations. Then, we train a head network on top of these representations through DRL to learn to control an ego vehicle in the CARLA AD simulator. This study presents a broad investigation of the impact of different learning schemes for offline-training of encoders on the performance of DRL agents in challenging AD tasks. Furthermore, we show that the features learned by watching BDD100K driving videos can be directly transferred to achieve lane following and collision avoidance in CARLA simulator, in a zero-shot learning fashion. Finally, we explore the impact of various architectural decisions for the RL networks to utilize the transferred representations efficiently. Therefore, in this work, we introduce and validate an optimal way for obtaining suitable representations of the environment, and transferring them to RL networks.
Offline Reinforcement Learning with Discrete Diffusion Skills
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
A2PO: Towards Effective Offline Reinforcement Learning from an Advantage-aware Perspective
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；State Key Laboratory of Blockchain and Data Security, Zhejiang University（浙江大学区块链与数据安全国家重点实验室）；Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security（杭州高新区（滨江）区块链与数据安全研究院）
Handwriting Trajectory Recovery using End-to-End Deep Encoder-Decoder Network
Institute of Engineering & Management（工程与管理学院）；Jadavpur University（贾达普大学）；Indian Institute of Technology Roorkee（印度理工学院鲁尔基分校）；Indian Statistical Institute（印度统计研究所）
Robust Encoder-Decoder Learning Framework towards Offline Handwritten Mathematical Expression Recognition Based on Multi-Scale Deep Neural Network
School of Instrumentation Science and Optoelectronics Engineering, Beihang University（北京航空航天大学仪器科学与光电工程学院）；Institute of Electronics, Chinese Academy of Sciences（中国科学院电子学研究所）；University of Chinese Academy of Sciences（中国科学院大学）
Stroke Constrained Attention Network for Online Handwritten Mathematical Expression Recognition
University of Science and Technology of China（中国科学技术大学）；National Engineering Laboratory for Speech and Language Information Processing（国家语音与语言信息处理工程实验室）
Double Encoder-Decoder Networks for Gastrointestinal Polyp Segmentation
Bournemouth University（伯恩茅斯大学）；Australian Institute for Machine Learning, University of Adelaide（阿德莱德大学澳大利亚机器学习研究所）；BCN Medtech（巴塞罗那医疗科技公司）；Universitat Pompeu Fabra（庞培法布拉大学）；ICREA（加泰罗尼亚高等研究院）
Healthy versus pathological learning transferability in shoulder muscle MRI segmentation using deep convolutional encoder-decoders
IMT Atlantique（大西洋国立高等矿业电信学院）；Inserm（法国国家健康与医学研究院）；University Hospital of Brest（布雷斯特大学医院）；Fondation ILDYS（ILDYS基金会）；NIH（美国国立卫生研究院）
由于肌肉形状、大小、位置、纹理和损伤的巨大差异，肌肉骨骼疾病患者的病理性肩部肌肉的自动分割是一项具有挑战性的任务。一种可靠的全自动磁共振图像分割方法可以极大地帮助临床医生规划治疗干预和预测干预结果，同时消除耗时的手动分割工作。这项工作的目的有三个方面。首先，我们研究了使用深度学习技术进行病理性肩部肌肉分割的可行性，给出了非常有限的可用注释儿科数据。其次，我们解决了从健康到病理数据的学习可移植性，通过比较不同的学习方案的模型泛化。第三，提出了使用在非医疗数据上预训练的编码器的深度卷积编码器-解码器架构的扩展版本，以提高分割准确性。方法学方面的评价在留一出的方式对数据集的24个肩关节检查，从产科臂丛神经麻痹的患者，并集中在4个不同的肌肉，包括三角肌以及冈下肌，冈上肌和肩胛下肌从肩袖。最相关的分割模型在ImageNet上进行了部分预训练，并联合利用了患者间的健康和病理注释数据。三角肌、冈下肌、冈上肌和肩胛下肌的Dice评分分别为82.4%、82.0%、71.0%和82.8%。除冈上肌外，绝对表面估计误差均小于83 mm $^2$，$^2$为134.6 mm。这些贡献提供了新的视角，在肌肉骨骼疾病管理的背景下，力推理。
X-Distill: Cross-Architecture Vision Distillation for Visuomotor Learning
Tsinghua University（清华大学）；Institute for Interdisciplinary Information Sciences（交叉信息研究院）；Shanghai Qi Zhi Institute（上海启智研究院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Huazhong University of Science and Technology（华中科技大学）
D'OH: Decoder-Only Random Hypernetworks for Implicit Neural Representations
Australian Institute for Machine Learning, University of Adelaide（阿德莱德大学澳大利亚机器学习研究所）；Mathematical Institute for Data Science, John Hopkins University（约翰霍普金斯大学数据科学数学研究所）
Segmentation of Cellular Patterns in Confocal Images of Melanocytic Lesions in vivo via a Multiscale Encoder-Decoder Network (MED-Net)
Memorial Sloan Kettering Cancer Center（纪念斯隆凯特琳癌症中心）；Northeastern University（东北大学）；Caliber Imaging and Diagnostics（Caliber影像与诊断公司）；SUNY Downstate Medical Center（纽约州立大学下州医学中心）；Skin Medical Research Diagnostics, P.L.L.C.（皮肤医学研究诊断有限责任公司）；University of Modena and Reggio Emilia（摩德纳-雷焦艾米利亚大学）
