Long-term memory is essential for multimodal agents to build coherent experience, accumulate world knowledge, and achieve continual learning. However, constructing effective memory goes beyond memory module design and basic requirements such as accuracy and fidelity; the key challenge lies in determining what to memorize. Multimodal agents, such as embodied agents, continuously perceive, reason, and act in real or virtual environments, receiving an unbounded stream of multimodal observations. From this combinatorial explosion of information, an agent must selectively retain content that is relevant to its role in the environment and valuable for future tasks. To bridge this gap, we frame memory generation as a learnable memorization policy and introduce TaskMem (Task-focused Memorization Policy Learning), a reinforcement-learning-based framework that enables the policy to dynamically adjust its focus to the demands of real tasks encountered in the environment. TaskMem adopts a two-phase training paradigm: Phase One learns how to memorize by optimizing memory quality under fundamental fidelity requirements; Phase Two occurs after deployment, where the agent learns what to memorize by tuning an adapter on its base MLLM, using recent environment tasks to define a reward model that guides the memorization policy toward task-relevant content. To evaluate our approach, we reformulate VideoMME, EgoLife, and EgoTempo into streaming benchmarks that simulate a realistic setting in which an agent processes streaming observations and handles tasks arriving online. To isolate memory assessment, the questions must be answered using only the agent's memory, without access to raw video. Built on Qwen3-VL-30B-A3B, TaskMem improves VQA accuracy by 6.3%, 7.0%, and 5.3% on these benchmarks, respectively.
Do Multimodal Large Language Models Understand Welding?
University of Notre Dame(圣母大学);Keough School of Global Affairs, University of Notre Dame(圣母大学基全球事务学院);Industry Labs, University of Notre Dame(圣母大学产业实验室)
VisualWebInstruct: Scaling up Multimodal Instruction Data through Web Search
University of Waterloo(滑铁卢大学);University of Toronto(多伦多大学);University of California, Santa Barbara(加利福尼亚大学圣巴巴拉分校);Carnegie Mellon University(卡内基梅隆大学);Nanyang Technological University(南洋理工大学)
FlowVQA: Mapping Multimodal Logic in Visual Question Answering with Flowcharts
Mercer Mettl(默瑟梅特尔);IGDTUW(英迪拉·甘地德里技术大学(女子));Google Research(谷歌研究院);Indian Institute of Technology Guwahati(古瓦哈蒂印度理工学院);University of Pennsylvania(宾夕法尼亚大学)
MOVE: A Mixture-of-Vision-Encoders Approach for Domain-Focused Vision-Language Processing
Skolkovo Institute of Science and Technology(斯科尔科沃科学技术研究院);Artificial Intelligence Research Institute (AIRI)(人工智能研究院);Higher School of Economics(高等经济学院)