AI 中文总结
针对低光条件下视频字幕生成质量下降的问题，提出一种不确定性感知的表示校正框架，通过轻量级稀疏高斯过程模块修正VideoChat2的中间表示，仅用44个样本训练，显著提升字幕准确性。
AI 中文摘要
低光条件会显著降低视觉语言模型（VLMs）准确描述视频中人类动作的能力。在本工作中，我提出了一种高效的不确定性感知表示校正框架，用于改进VideoChat2在真实低光条件下生成的字幕。该框架不是对底层VLM进行微调，而是在投影层和语言模型之间引入一个轻量级的稀疏高斯过程误差估计模块，以校正中间表示。校正模块学习估计原始投影表示与经过验证的目标表示之间的残差，然后使用新制定的不确定性感知系数进行自适应缩放，并添加到原始表示中。为了进一步改进残差估计，我引入了分区组合核设计。校正模型仅使用ARID数据集中的44个样本进行单独训练，并且在推理过程中仅需少量额外计算开销。通过定量残差预测和生成字幕的定性分析来评估所提出校正的有效性。尽管我的实验使用了VideoChat2，但所提出的框架设计为适用于其他兼容的VLM架构。代码可用性：本工作附带的实现公开可用，网址为：https://this https URL。
英文摘要
Low-light conditions can significantly degrade the ability of vision-language models (VLMs) to accurately describe human actions in videos. In this work, I propose an efficient uncertainty-aware representation correction framework for improving captions generated by VideoChat2 under real-world low-light conditions. Instead of fine-tuning the underlying VLM, the proposed framework introduces a lightweight sparse Gaussian process-based error estimation module between the projection layer and the language model to correct the intermediate representation. The correction module learns to estimate the residual between the original projected representation and a verified target representation, which is then adaptively scaled using a newly formulated uncertainty-aware coefficient and added to the original representation. To further improve residual estimation, I introduce a partitioned combined-kernel design. The correction model is trained separately using only 44 samples from the ARID dataset and requires only a small additional computational overhead during inference. The effectiveness of the proposed correction is evaluated through quantitative residual prediction and qualitative analysis of the generated captions. Although VideoChat2 is used in my experiments, the proposed framework is designed to be applicable to other compatible VLM architectures. \textbf{Code Availability: The implementation accompanying this work is publicly available at}:\href{https://github.com/areferezaee/Low-rank-SVGP-NP-update}{https://github.com/areferezaee/Low-rank-SVGP-NP-update}
