DarkQA：在低光室内场景中对视觉-原始问题进行问答的视觉语言模型基准测试
DarkQA: Benchmarking Vision-Language Models on Visual-Primitive Question Answering in Low-Light Indoor Scenes
AI总结：
DarkQA针对低光环境下视觉语言模型的感知基本能力进行基准测试，通过多级低光条件评估，揭示模型在复杂任务中的局限性。
摘要：
视觉语言模型（VLMs）越来越多地被用作具身代理的核心推理模块。现有基准测试评估其在理想、光照良好的条件下的能力，但要求24/7可靠运行需要在广泛的视觉退化条件下表现，包括夜间或黑暗环境中的低光条件，这是被忽视的核心需求。为解决这一未被充分探索的挑战，我们提出了DarkQA，一个开放源代码的基准测试，用于评估具身场景中多级低光条件下的感知基本能力。DarkQA评估单视角自体中心观察，跨越受控退化级别，隔离低光感知失败，使其在与其他复杂具身任务纠缠之前进行评估。该基准测试包含9.4K确定性生成并可验证的问题-图像对，覆盖五个视觉基本家族。DarkQA的关键设计特征是其物理真实性：视觉退化在线性RAW空间中建模，模拟基于物理的照明下降和传感器噪声，随后通过受启发的ISP渲染管线；我们进一步验证合成结果与真实低光相机数据配对。我们评估了代表性的VLMs和低光图像增强（LLIE）预处理方法。结果表明，在低光照和传感器噪声下，VLMs的退化表现一致，而LLIE提供严重性依赖但不稳定的恢复。通过评估一系列最先进的VLMs和LLIE模型，我们系统地揭示了VLMs在这些挑战性视觉条件下操作的局限性。我们的代码和基准测试数据集将在接受后发布。项目网站：https://darkqa-benchmark.github.io
英文摘要：
Vision Language Models (VLMs) are increasingly adopted as central reasoning modules for embodied agents. Existing benchmarks evaluate their capabilities under ideal, well-lit conditions, yet robust 24/7 operation demands performance under a wide range of visual degradations, including low-light conditions at night or in dark environments, a core necessity that has been largely overlooked. To address this underexplored challenge, we present DarkQA, an open-source benchmark for evaluating perceptual primitives under multi-level low-light conditions in embodied scenarios. DarkQA evaluates single-view egocentric observations across controlled degradation levels, isolating low-light perceptual failures before they are entangled with complex embodied tasks. The benchmark contains 9.4K deterministically generated and verifiable question-image pairs spanning five visual-primitive families. A key design feature of DarkQA is its physical fidelity: visual degradations are modeled in linear RAW space, simulating physics-based illumination drop and sensor noise followed by an ISP-inspired rendering pipeline; we further validate the synthesis against real paired low-light camera data. We evaluate representative VLMs and Low-Light Image Enhancement (LLIE) preprocessing methods. Results show consistent VLM degradation under low illumination and sensor noise, while LLIE provides severity-dependent but unstable recovery. We demonstrate the utility of DarkQA by evaluating a wide range of state-of-the-art VLMs and Low-Light Image Enhancement (LLIE) models, and systematically reveal VLMs' limitations when operating under these challenging visual conditions. Our code and benchmark dataset will be released upon acceptance. Project website: https://darkqa-benchmark.github.io
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
