MemSim: A Bayesian Simulator for Evaluating Memory of LLM-based Personal Assistants
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；Huawei Technologies Ltd.（华为技术有限公司）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）
LLM-based agents have been widely applied as personal assistants, capable of memorizing information from user messages and responding to personal queries. However, there still lacks an objective and automatic evaluation on their memory capability, largely due to the challenges in constructing reliable questions and answers (QAs) according to user messages. In this paper, we propose MemSim, a Bayesian simulator designed to automatically construct reliable QAs from generated user messages, simultaneously keeping their diversity and scalability. Specifically, we introduce the Bayesian Relation Network (BRNet) and a causal generation mechanism to mitigate the impact of LLM hallucinations on factual information, facilitating the automatic creation of an evaluation dataset. Based on MemSim, we generate a dataset in the daily-life scenario, named MemDaily, and conduct extensive experiments to assess the effectiveness of our approach. We also provide a benchmark for evaluating different memory mechanisms in LLM-based agents with the MemDaily dataset. To benefit the research community, we have released our project at https://github.com/nuster1128/MemSim.
Personalized Large Language Model Assistant with Evolving Conditional Memory
The Hong Kong Polytechnic University（香港理工大学）；INF Technology (Shanghai) co., Ltd.（INF科技（上海）有限公司）；Institute of Artificial Intelligence, Soochow University, China（苏州大学人工智能研究院）
Exploring the Potential of LLMs as Personalized Assistants: Dataset, Evaluation, and Analysis
Department of Electrical and Computer Engineering, Seoul National University（电子与计算机工程系，首尔国立大学）；Interdisciplinary Program in Artificial Intelligence, Seoul National University（人工智能跨学科计划，首尔国立大学）；AIIS, ASRI, INMC, and ISRC, Seoul National University（人工智能研究所、人工智能研究机构、智能网络中心和智能系统研究组，首尔国立大学）
A Prompt-Engineering Approach to Develop Scalable, Flexible, and Real-Time Hybrid Micro-Level Personalization in a General Purpose AI Teaching Assistant
Modern urbanization is demanding smarter technologies to improve a variety of applications in intelligent transportation systems to relieve the increasing amount of vehicular traffic congestion and incidents. Existing incident detection techniques are limited to the use of sensors in the transportation network and hang on human-inputs. Despite of its data abundance, social media is not well-exploited in such context. In this paper, we develop an automated traffic alert system based on Natural Language Processing (NLP) that filters this flood of information and extract important traffic-related…
ReGraP-LLaVA: Reasoning enabled Graph-based Personalized Large Language and Vision Assistant
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究所）；School of Engineering, Westlake University（西湖大学工程学院）；University of Minnesota – Twin Cities（明尼苏达大学双城分校）；University of Toronto（多伦多大学）
$π$-Bench: Evaluating Proactive Personal Assistant Agents in Long-Horizon Workflows
Shanghai Jiao Tong University（上海交通大学）；Shanghai AI Laboratory（上海人工智能实验室）；Fudan University（复旦大学）；University of Science and Technology of China（中国科学技术大学）；Peking University（北京大学）；Nanjing University（南京大学）；Zhejiang University（浙江大学）；Tongji University（同济大学）；Soochow University（苏州大学）；The Chinese University of Hong Kong（香港中文大学）
Claw-Anything: Benchmarking Always-On Personal Assistants with Broader Access to User's Digital World
Beijing Institute of Technology（北京理工大学）；Huawei Technologies Co., Ltd（华为技术有限公司）；Peking University（北京大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
Intelligent writing assistants powered by large language models (LLMs) are more popular today than ever before, but their further widespread adoption is precluded by sub-optimal performance. In this position paper, we argue that a major reason for this sub-optimal performance and adoption is a singular focus on the information content of language while ignoring its social aspects. We analyze the different dimensions of these social factors in the context of writing assistants and propose their incorporation into building smarter, more effective, and truly personalized writing assistants that w…
Open TutorAI: An Open-source Platform for Personalized and Immersive Learning with Generative AI
IRF-SIC Laboratory, Ibnou Zohr University（IRF-SIC实验室，伊本·扎赫尔大学）；Regional Center for Education（教育与培训专业地区中心）；Polydisciplinary Faculty of Taroudant, Ibnou Zohr University（塔鲁旦多学科学院，伊本·扎赫尔大学）；Higher School of Technology of Guelmim, Ibnou Zohr University（盖尔米姆技术高等学校，伊本·扎赫尔大学）；Paragraphe laboratory, Paris 8 and CY Cergy Paris Universities（Paragraphe实验室，巴黎8大学和CY塞克巴黎大学）
Many software systems, such as online social networks enable users to share information about themselves. While the action of sharing is simple, it requires an elaborate thought process on privacy: what to share, with whom to share, and for what purposes. Thinking about these for each piece of content to be shared is tedious. Recent approaches to tackle this problem build personal assistants that can help users by learning what is private over time and recommending privacy labels such as private or public to individual content that a user considers sharing. However, privacy is inherently ambig…
We present a new approach to dialogue specification for Virtual Personal Assistants (VPAs) based on so-called dialogue workflow graphs, with several demonstrated advantages over current ontology-based methods. Our new dialogue specification language (DSL) enables customers to more easily participate in the VPA modeling process due to a user-friendly modeling framework. Resulting models are also significantly more compact. VPAs can be developed much more rapidly. The DSL is a new modeling layer on top of our ontology-based Dialogue Management (DM) framework OntoVPA. We explain the rationale and…
College of Information Technologies and Computer Sciences NUST MISIS（莫斯科国立钢铁合金学院信息技术与计算机科学学院）
This article describes creating algorithmic support for the functioning of a personal virtual assistant, which allows automating the processing of customer requests. The study aims to reduce errors and processing time for a client request in business systems - text chats or voice channels using a text transcription system. The results of the development of algorithmic support and an assessment of the quality of work on synthetic data presented.
