InteraRec：使用多模态大语言模型的基于截图的推荐
InteraRec: Screenshot Based Recommendations Using Multimodal Large Language Models
AI总结：
本文提出 InteraRec 框架，利用网页高频截图和 MLLM 提取用户偏好，并结合 LLM 优化生成个性化推荐，同时构建 Amazon 截图数据集验证其效果。
摘要：
网络日志由详细记录用户在任何网站上活动的记录组成，可提供有关用户偏好、行为和兴趣的宝贵洞察。众多推荐算法采用协同过滤、基于内容的过滤以及混合方法等策略，利用通过这些网络日志挖掘的数据，向用户提供个性化推荐。尽管这些网络日志中存在丰富的信息，但从中识别和提取相关信息及关键特征需要大量的工程工作。数据的复杂性质也给解释带来了挑战，尤其是对非专家而言。在本研究中，我们提出了一个先进的交互式推荐框架，称为 InteraRec，它不同于仅依赖网络日志生成推荐的传统方法。InteraRec 框架在用户浏览网站时捕获网页的高频截图。借助最先进的多模态大语言模型（MLLM），它通过基于预定义关键词生成文本摘要，从这些截图中提取有关用户偏好的宝贵洞察。随后，一个集成 LLM 的优化设置利用该摘要生成定制推荐。通过实验，我们证明了 InteraRec 在向用户提供有价值且个性化的内容方面的有效性。此外，我们还探索了将会话型推荐系统集成到 InteraRec 框架中，旨在提升其整体性能。最后，我们构建了一个新数据集，其中包含来自 Amazon 网站产品网页的截图，用于验证 InteraRec 框架。详细实验表明，InteraRec 框架能够有效提供契合个人用户偏好的有价值且个性化的推荐。
英文摘要：
Weblogs, comprised of records detailing user activities on any website, offer valuable insights into user preferences, behavior, and interests. Numerous recommendation algorithms, employing strategies such as collaborative filtering, content-based filtering, and hybrid methods, leverage the data mined through these weblogs to provide personalized recommendations to users. Despite the abundance of information available in these weblogs, identifying and extracting pertinent information and key features from them necessitate extensive engineering endeavors. The intricate nature of the data also poses a challenge for interpretation, especially for non-experts. In this study, we introduce a sophisticated and interactive recommendation framework denoted as InteraRec, which diverges from conventional approaches that exclusively depend on weblogs for recommendation generation. InteraRec framework captures high-frequency screenshots of web pages as users navigate through a website. Leveraging state-of-the-art multimodal large language models (MLLMs), it extracts valuable insights into user preferences from these screenshots by generating a textual summary based on predefined keywords. Subsequently, an LLM-integrated optimization setup utilizes this summary to generate tailored recommendations. Through our experiments, we demonstrate the effectiveness of InteraRec in providing users with valuable and personalized offerings. Furthermore, we explore the integration of session-based recommendation systems into the InteraRec framework, aiming to enhance its overall performance. Finally, we curate a new dataset comprising of screenshots from product web pages on the Amazon website for the validation of the InteraRec framework. Detailed experiments demonstrate the efficacy of the InteraRec framework in delivering valuable and personalized recommendations tailored to individual user preferences.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
