We introduce the AutoGRAMS framework for programming multi-step interactions with language models. AutoGRAMS represents AI agents as a graph, where each node can execute either a language modeling instruction or traditional code. Likewise, transitions in the graph can be governed by either language modeling decisions or traditional branch logic. AutoGRAMS supports using variables as memory and allows nodes to call other AutoGRAMS graphs as functions. We show how AutoGRAMS can be used to design highly sophisticated agents, including self-referential agents that can modify their own graph. AutoGRAMS's graph-centric approach aids interpretability, controllability, and safety during the design, development, and deployment of AI agents. We provide our framework as open source at https://github.com/autograms/autograms .
LPO: Towards Accurate GUI Agent Interaction via Location Preference Optimization
The Hong Kong University of Science and Technology(香港科学与技术大学);Alibaba Group(阿里巴巴集团);The Chinese University of Hong Kong(香港中文大学);Nanjing University of Science and Technology(南京理工大学);The Hong Kong University of Science and Technology (Guangzhou)(广州香港科学与技术大学)
CoCo-Agent: A Comprehensive Cognitive MLLM Agent for Smartphone GUI Automation
School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University(上海交通大学电子信息与电气工程学院);Department of Computer Science and Engineering, Shanghai Jiao Tong University(上海交通大学计算机科学与工程系);Shanghai Key Laboratory of Trusted Data Circulation and Governance in Web3(上海市Web3可信数据流通与治理重点实验室)