神经光适应：基于凝视的心理图像重建
Neural Photofit: Gaze-based Mental Image Reconstruction
摘要：
我们提出了一种新的方法，利用人类的固定视觉解码的图像一个人心目中的photofit（面部合成）。我们的方法结合了三个神经网络：编码器，评分网络和解码器。编码器提取图像特征，并预测人类观察者看到的每张脸的神经激活图。神经评分网络比较人类和神经的注意力，并预测每个提取的图像特征的相关性得分。最后，图像特征被聚合成一个单一的特征向量，作为一个线性组合的所有功能加权的相关性，解码器解码成最终的photofit。我们在一个新的数据集上训练神经评分网络，该数据集包含19名参与者观看合成面孔拼贴画的凝视数据。我们表明，我们的方法显着优于平均基线预测和报告的人类研究表明，我们可以解码的photofits是视觉上合理的，接近观察者的心理形象。
英文摘要：
We propose a novel method that leverages human fixations to visually decode the image a person has in mind into a photofit (facial composite). Our method combines three neural networks: An encoder, a scoring network, and a decoder. The encoder extracts image features and predicts a neural activation map for each face looked at by a human observer. A neural scoring network compares the human and neural attention and predicts a relevance score for each extracted image feature. Finally, image features are aggregated into a single feature vector as a linear combination of all features weighted by relevance which a decoder decodes into the final photofit. We train the neural scoring network on a novel dataset containing gaze data of 19 participants looking at collages of synthetic faces. We show that our method significantly outperforms a mean baseline predictor and report on a human study that shows that we can decode photofits that are visually plausible and close to the observer's mental image.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
