LLM agents now perform strongly in software engineering, deep research, GUI automation, and various other applications, while recent agent scaffolds and models are increasingly integrating these capabilities into unified systems. Yet, most evaluations still test these capabilities in isolation, which leaves a gap for more diverse use cases that require agents to combine different capabilities. We introduce CocoaBench, a benchmark for unified digital agents built from human-designed, long-horizon tasks that require flexible composition of vision, search, and coding. Tasks are specified only by an instruction and an automatic evaluation function over the final output, enabling reliable and scalable evaluation across diverse agent infrastructures. We also present CocoaAgent, a lightweight shared scaffold for controlled comparison across model backbones. Experiments show that current agents remain far from reliable on CocoaBench, with the best evaluated system achieving only 45.1% success rate. Our analysis further points to substantial room for improvement in reasoning and planning, tool use and execution, and visual grounding.
HelloBench: Evaluating Long Text Generation Capabilities of Large Language Models
Beihang University(北京航空航天大学);Shanghai AI Laboratory(上海人工智能实验室);Peking University(北京大学);M-A-P;University of Chinese Academy of Sciences(中国科学院大学);Nanjing University(南京大学)
Measuring the Behavioral Fidelity of Long-Horizon Human Activity Simulations
Human-Computer Interaction Institute, School of Computer Science, Carnegie Mellon University(卡内基梅隆大学计算机科学学院人机交互研究所);Fujitsu Ltd.(富士通有限公司);Fujitsu Research of America(富士通美国研究院)
SpatiaLab: Can Vision-Language Models Perform Spatial Reasoning in the Wild?
Computational Intelligence and Operations Laboratory(计算智能与运筹实验室);Shahjalal University of Science and Technology(沙赫jalal科技大学);BRAC University(BRAC大学);North South University(北南大学);Monash University(墨尔本大学);Qatar Computing Research Institute(卡塔尔计算研究院)