Beamforming-LLM：我错过了什么、在哪里以及何时错过的？
Beamforming-LLM: What, Where and When Did I Miss?
AI总结：
本文提出Beamforming-LLM系统，结合波束形成、Whisper转录和RAG，实现多说话人环境中错失对话的语义回忆，提供对比摘要和时空上下文，奠定智能听觉记忆基础。
摘要：
我们提出了Beamforming-LLM，一个使用户能够在多说话人环境中语义回忆可能错过的对话的系统。该系统结合了使用麦克风阵列的空间音频采集与检索增强生成（RAG），以支持诸如“当我正在关注关于狗的对话时，我错过了什么？”之类的自然语言查询。方向性音频流通过波束形成进行分离，使用Whisper进行转录，并使用句子编码器嵌入到向量数据库中。在收到用户查询后，检索语义相关的片段，与未关注的片段进行时间对齐，并使用轻量级大语言模型（GPT-4o-mini）进行总结。结果是一个用户友好的界面，提供对比性摘要、空间上下文和时间戳音频回放。这项工作为智能听觉记忆系统奠定了基础，并在辅助技术、会议总结和上下文感知的个人空间计算中具有广泛的应用。
英文摘要：
We present Beamforming-LLM, a system that enables users to semantically recall conversations they may have missed in multi-speaker environments. The system combines spatial audio capture using a microphone array with retrieval-augmented generation (RAG) to support natural language queries such as, "What did I miss when I was following the conversation on dogs?" Directional audio streams are separated using beamforming, transcribed with Whisper, and embedded into a vector database using sentence encoders. Upon receiving a user query, semantically relevant segments are retrieved, temporally aligned with non-attended segments, and summarized using a lightweight large language model (GPT-4o-mini). The result is a user-friendly interface that provides contrastive summaries, spatial context, and timestamped audio playback. This work lays the foundation for intelligent auditory memory systems and has broad applications in assistive technology, meeting summarization, and context-aware personal spatial computing.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
