Saarthi is an agentic AI framework that uses multi-agent collaboration to perform end-to-end formal verification. Even though the framework provides a complete flow from specification to coverage closure, with around 40% efficacy, there are several challenges that need to be addressed to make it more robust and reliable. Artificial General Intelligence (AGI) is still a distant goal, and current Large Language Model (LLM)-based agents are prone to hallucinations and making mistakes, especially when dealing with complex tasks such as formal verification. However, with the right enhancements and improvements, we believe that Saarthi can be a significant step towards achieving domain-specific general intelligence for formal verification. Especially for problems that require Short Term, Short Context (STSC) capabilities, such as formal verification, Saarthi can be a powerful tool to assist verification engineers in their work. In this paper, we present two key enhancements to the Saarthi framework: (1) a structured rulebook and specification grammar to improve the accuracy and controllability of SystemVerilog Assertion (SVA) generation, and (2) integration of advanced Retrieval Augmented Generation (RAG) techniques, such as GraphRAG, to provide agents with access to technical knowledge and best practices for iterative refinement and improvement of outputs. We also benchmark these enhancements for the overall Saarthi framework using challenging test cases from NVIDIA's CVDP benchmark targeting formal verification. Our benchmark results stand out with a 70% improvement in the accuracy of generated assertions, and a 50% reduction in the number of iterations required to achieve coverage closure.
Why We Need World Models for AGI: Where LLMs Fail and How World Models May Outperform
Department of Computing Technologies(计算技术系);SRM Institute of Science and Technology(SRM科学与技术学院);Bio-Sensing and Bio-Sensors Group(生物传感与生物传感器组);Smart Automation and Communication Technologies Research Institute of Sciences and Engineering(科学与工程智能自动化与通信技术研究所);University of Sharjah, UAE(阿联酋沙迦大学);Department of Computer Engineering(计算机工程系);College of Computing and Informatics(计算与信息学院)
MechMath Agent Team: LLM Driven Agents for Mathematical Research
State Key Laboratory of Mathematical Sciences, Academy of Mathematics and Systems Science, CAS(中国科学院数学与系统科学研究院数学科学国家重点实验室);School of Mathematical Sciences, University of Chinese Academy of Sciences(中国科学院大学数学科学学院);School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences(中国科学院大学高等研究院)
Optimising for Flourishing: Flourishing Metrics and Return on Flourishing as Success Criteria for Artificial Intelligence and Post-AGI Economic Systems