Memory-augmented LLM agents store and retrieve information from prior interactions, yet the relative importance of how memories are written versus how they are retrieved remains unclear. We introduce a diagnostic framework that analyzes how performance differences manifest across write strategies, retrieval methods, and memory utilization behavior, and apply it to a 3x3 study crossing three write strategies (raw chunks, Mem0-style fact extraction, MemGPT-style summarization) with three retrieval methods (cosine, BM25, hybrid reranking). On LoCoMo, retrieval method is the dominant factor: average accuracy spans 20 points across retrieval methods (57.1% to 77.2%) but only 3-8 points across write strategies. Raw chunked storage, which requires zero LLM calls, matches or outperforms expensive lossy alternatives, suggesting that current memory pipelines may discard useful context that downstream retrieval mechanisms fail to compensate for. Failure analysis shows that performance breakdowns most often manifest at the retrieval stage rather than at utilization. We argue that, under current retrieval practices, improving retrieval quality yields larger gains than increasing write-time sophistication. Code is publicly available at https://github.com/boqiny/memory-probe.
When Do LLMs Apply the Wrong Law? Diagnosing LLM Failures in Temporal Legal Reasoning
Institute of Science Tokyo（东京科学大学）；Osaka University（大阪大学）；NII LLMC（日本信息学研究所语言、语言学与媒体中心）；Nara Institute of Science and Technology（奈良科学技术研究所）；Center of Juris-Informatics, ROIS-DS（日本学术研究推进机构跨学科科学研究中心法学信息学中心）
iSING Lab, Hong Kong University of Science and Technology（香港科技大学 iSING 实验室）；Harvard University（哈佛大学）；Intel, Barefoot Switch Division（英特尔 Barefoot 交换机部门）；Peking University（北京大学）
As the scale of distributed training grows, communication becomes a bottleneck. To accelerate the communication, recent works introduce In-Network Aggregation (INA), which moves the gradients summation into network middle-boxes, e.g., programmable switches to reduce the traffic volume. However, switch memory is scarce compared to the volume of gradients transmitted in distributed training. Although literature applies methods like pool-based streaming or dynamic sharing to tackle the mismatch, switch memory is still a potential performance bottleneck. Furthermore, we observe the under-utilizati…
Before Humans Join the Team: Diagnosing Coordination Failures in Healthcare Robot Team Simulation
Cornell Tech（康奈尔科技校区）；Department of Information Science, Cornell University（康奈尔大学信息科学系）；University of Maryland, College Park（马里兰大学帕克分校）；Imperial College London（伦敦帝国学院）
AgentCollabBench: Diagnosing When Good Agents Make Bad Collaborators
University of Utah（犹他大学）；University of Maryland, Baltimore County（马里兰大学巴尔的摩县分校）；University of Dhaka（达卡大学）；Vellore Institute of Technology（韦洛雷理工学院）；University of Virginia（弗吉尼亚大学）；Rajshahi University of Engineering and Technology（拉贾加赫尔工程与技术大学）；Shahjalal University of Science and Technology（沙赫jalal科学与技术大学）；BRAC University（BRAC大学）；Islamic University of Technology（伊斯兰技术大学）；Comilla University（科摩拉大学）
