Large Language Models (LLMs) are increasingly applied in recommendation scenarios due to their strong natural language understanding and generation capabilities. However, they are trained on vast corpora whose contents are not publicly disclosed, raising concerns about data leakage. Recent work has shown that the MovieLens-1M dataset is memorized by both the LLaMA and OpenAI model families, but the extraction of such memorized data has so far relied exclusively on manual prompt engineering. In this paper, we pose three main questions: Is it possible to enhance manual prompting? Can LLM memorization be detected through methods beyond manual prompting? And can the detection of data leakage be automated? To address these questions, we evaluate three approaches: (i) jailbreak prompt engineering; (ii) unsupervised latent knowledge discovery, probing internal activations via Contrast-Consistent Search (CCS) and Cluster-Norm; and (iii) Automatic Prompt Engineering (APE), which frames prompt discovery as a meta-learning process that iteratively refines candidate instructions. Experiments on MovieLens-1M using LLaMA models show that jailbreak prompting does not improve the retrieval of memorized items and remains inconsistent; CCS reliably distinguishes genuine from fabricated movie titles but fails on numerical user and rating data; and APE retrieves item-level information with moderate success yet struggles to recover numerical interactions. These findings suggest that automatically optimizing prompts is the most promising strategy for extracting memorized samples.
Exploring the Escalation of Source Bias in User, Data, and Recommender System Feedback Loop
Gaoling School of Artificial Intelligence\ University of China Beijing China；CAS Key Laboratory of AI Safety\ of Computing Technology Chinese Academy of Sciences Beijing China；Huawei Noah’s Ark Lab Shenzhen China；Gaoling School of Artificial Intelligence\ University of China；CAS Key Laboratory of AI Safety\ of Computing Technology Chinese Academy of Sciences；Huawei Noah’s Ark Lab
EventChat: Implementation and user-centric evaluation of a large language model-driven conversational recommender system for exploring leisure events in an SME context
Swiss Federal Institute of Technology, Zurich（苏黎世联邦理工学院）
Exploring the Impact of Personality Traits on Conversational Recommender Systems: A Simulation with Large Language Models
Chinese University of Hong Kong（香港中文大学）；National University of Singapore（新加坡国立大学）；Singapore Management University（新加坡管理大学）；Hong Kong Baptist University（香港浸会大学）；Huazhong University of Science and Technology（华中科技大学）
Illusion or Algorithm? Investigating Memorization, Emergence, and Symbolic Processing in In-Context Learning
Technical University of Darmstadt（德累斯顿技术大学）；UKP Lab（UKP实验室）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；The University of Bath（巴斯大学）
Exploring Applications of State Space Models and Advanced Training Techniques in Sequential Recommendations: A Comparative Study on Efficiency and Performance
Kempelen Institute of Intelligent Technologies（肯佩伦智能技术研究所）；Slovak University of Technology in Bratislava（布拉迪斯拉发斯洛伐克工业大学）；Exponea
Most of the research in the recommender systems domain is focused on the optimization of the metrics based on historical data such as Mean Average Precision (MAP) or Recall. However, there is a gap between the research and industry since the leading Key Performance Indicators (KPIs) for businesses are revenue and profit. In this paper, we explore the impact of manipulating the profit awareness of a recommender system. An average e-commerce business does not usually use a complicated recommender algorithm. We propose an adjustment of a predicted ranking for score-based recommender systems and e…
Algorithmic fairness for artificial intelligence has become increasingly relevant as these systems become more pervasive in society. One realm of AI, recommender systems, presents unique challenges for fairness due to trade offs between optimizing accuracy for users and fairness to providers. But what is fair in the context of recommendation--particularly when there are multiple stakeholders? In an initial exploration of this problem, we ask users what their ideas of fair treatment in recommendation might be, and why. We analyze what might cause discrepancies or changes between user's opinions…
Media recommender systems aim to capture users' preferences and provide precise personalized recommendation of media content. There are two critical components in the common paradigm of modern recommender models: (1) representation learning, which generates an embedding for each user and item; and (2) interaction modeling, which fits user preferences towards items based on their representations. In spite of great success, when a great amount of users and items exist, it usually needs to create, store, and optimize a huge embedding table, where the scale of model parameters easily reach million…
Recommender systems are indispensable because they influence our day-to-day behavior and decisions by giving us personalized suggestions. Services like Kindle, Youtube, and Netflix depend heavily on the performance of their recommender systems to ensure that their users have a good experience and to increase revenues. Despite their popularity, it has been shown that recommender systems reproduce and amplify the bias present in the real world. The resulting feedback creates a self-perpetuating loop that deteriorates the user experience and results in homogenizing recommendations over time. Furt…
University of Washington（华盛顿大学）；Allen Institute for Artificial Intelligence（艾伦人工智能研究所）；Northwestern University（西北大学）；University of Washington & Allen Institute for Artificial Intelligence（华盛顿大学和艾伦人工智能研究所）
Explanations are well-known to improve recommender systems' transparency. These explanations may be local, explaining an individual recommendation, or global, explaining the recommender model in general. Despite their widespread use, there has been little investigation into the relative benefits of these two approaches. Do they provide the same benefits to users, or do they serve different purposes? We conducted a 30-participant exploratory study and a 30-participant controlled user study with a research-paper recommender system to analyze how providing participants local, global, or both expl…
Effective methodologies for evaluating recommender systems are critical, so that such systems can be compared in a sound manner. A commonly overlooked aspect of recommender system evaluation is the selection of the data splitting strategy. In this paper, we both show that there is no standard splitting strategy and that the selection of splitting strategy can have a strong impact on the ranking of recommender systems. In particular, we perform experiments comparing three common splitting strategies, examining their impact over seven state-of-the-art recommendation models for two datasets. Our …
Towards Recommender Systems LLMs Playground (RecSysLLMsP): Exploring Polarization and Engagement in Simulated Social Networks
Institute for Artificial Intelligence Research and Development of Serbia（塞尔维亚人工智能研究与发展研究所）；University of Belgrade（贝尔格莱德大学）；Polytechnic University of Bari（巴里理工大学）；University of Kragujevac（克拉古耶瓦茨大学）
Related Pins at Pinterest: The Evolution of a Real-World Recommender System
Related Pins是一个网络规模的推荐系统，为Pinterest上超过40%的用户参与提供支持。本文是一个纵向研究的三年的发展，探索系统及其组件的演变，从原型到现在的状态。每个组件最初都是在工程工作和计算资源方面受到许多限制的情况下构建的，因此我们优先考虑最简单和最高利用率的解决方案。我们展示了有机增长如何导致一个复杂的系统，以及我们如何管理这种复杂性。在构建这个系统的过程中出现了许多挑战，例如避免反馈循环，评估性能，激活内容以及消除遗留的错误。最后，我们提供了解决这些挑战时，工程Web规模的推荐系统的建议。
Time to Split: Exploring Data Splitting Strategies for Offline Evaluation of Sequential Recommenders
AIRI, Skoltech Moscow Russian Federation（AIRI，斯克里普钦斯基因工学院莫斯科俄罗斯联邦）；Sber AI Lab, Skoltech Moscow Russian Federation（Sber AI Lab，斯克里普钦斯基因工学院莫斯科俄罗斯联邦）；Sber AI Lab Moscow Russian Federation（Sber AI Lab，莫斯科斯克里普钦斯基因工学院俄罗斯联邦）；Sber AI Lab, HSE University Moscow Russian Federation（Sber AI Lab，莫斯科高等经济大学俄罗斯联邦）
Popularity bias is the idea that a recommender system will unduly favor popular artists when recommending artists to users. As such, they may contribute to a winner-take-all marketplace in which a small number of artists receive nearly all of the attention, while similarly meritorious artists are unlikely to be discovered. In this paper, we attempt to measure popularity bias in three state-of-art recommender system models (e.g., SLIM, Multi-VAE, WRMF) and on three commercial music streaming services (Spotify, Amazon Music, YouTube). We find that the most accurate model (SLIM) also has the most…
University of Hong Kong（香港大学）；Brandeis University（布兰迪斯大学）
Graph Neural Networks (GNNs) have been shown as promising solutions for collaborative filtering (CF) with the modeling of user-item interaction graphs. The key idea of existing GNN-based recommender systems is to recursively perform the message passing along the user-item interaction edge for refining the encoded embeddings. Despite their effectiveness, however, most of the current recommendation models rely on sufficient and high-quality training data, such that the learned representations can well capture accurate user preference. User behavior data in many practical recommendation scenarios…
University of Texas at Austin（得克萨斯大学奥斯汀分校）；Facebook Inc（脸书公司）
Session-based Recurrent Neural Networks (RNNs) are gaining increasing popularity for recommendation task, due to the high autocorrelation of user's behavior on the latest session and the effectiveness of RNN to capture the sequence order information. However, most existing session-based RNN recommender systems still solely focus on the short-term interactions within a single session and completely discard all the other long-term data across different sessions. While traditional Collaborative Filtering (CF) methods have many advanced research works on exploring long-term dependency, which show …
Collaborative filtering algorithms find useful patterns in rating and consumption data and exploit these patterns to guide users to good items. Many of the patterns in rating datasets reflect important real-world differences between the various users and items in the data; other patterns may be irrelevant or possibly undesirable for social or ethical reasons, particularly if they reflect undesired discrimination, such as discrimination in publishing or purchasing against authors who are women or ethnic minorities. In this work, we examine the response of collaborative filtering recommender alg…
