GeMA: Learning Latent Manifold Frontiers for Benchmarking Complex Systems
Jia Ming Li, Anupriya, Daniel J. Graham
备注:
Latent manifold frontiers for benchmarking complex production systems, and applications to national rail operators, wind farms, and macroeconomic productivity are presented
复杂系统性能评估对交通规划、监管和宏观经济分析至关重要。传统前沿方法如DEA和SFA估计输入输出空间中的有效前沿,但假设限制性强且无法直接处理异质性和规模效应。本文提出Geometric Manifold Analysis(GeMA),通过生产力-流形变分自编码器(ProMan-VAE)实现潜在流形前沿框架。GeMA将生产集表示为嵌入在联合输入输出空间中的低维流形边界。Split-head编码器学习捕捉技术结构和运营低效的潜在变量。效率相对于学习的流形评估,内生同伴组作为潜在技术空间中的聚类出现,商构造支持规模不变的基准测试,局部认证半径从解码器雅可比和Lipschitz界推导,量化效率分数的几何鲁棒性。GeMA在合成数据和四个现实案例研究中验证:全球城市铁路系统(COMET)、英国铁路运营商(ORR)、国家经济(Penn World Table)和高频风力发电机数据集。在这些领域,GeMA在经典假设下表现与传统方法相当,并在异质性、非凸性或规模偏差显著时提供额外见解。
英文摘要:
Benchmarking the performance of complex systems such as rail networks, renewable generation assets and national economies is central to transport planning, regulation and macroeconomic analysis. Classical frontier methods, notably Data Envelopment Analysis (DEA) and Stochastic Frontier Analysis (SFA), estimate an efficient frontier in the observed input-output space and define efficiency as distance to this frontier, but rely on restrictive assumptions on the production set and only indirectly address heterogeneity and scale effects.
We propose Geometric Manifold Analysis (GeMA), a latent manifold frontier framework implemented via a productivity-manifold variational autoencoder (ProMan-VAE). Instead of specifying a frontier function in the observed space, GeMA represents the production set as the boundary of a low-dimensional manifold embedded in the joint input-output space. A split-head encoder learns latent variables that capture technological structure and operational inefficiency. Efficiency is evaluated with respect to the learned manifold, endogenous peer groups arise as clusters in latent technology space, a quotient construction supports scale-invariant benchmarking, and a local certification radius, derived from the decoder Jacobian and a Lipschitz bound, quantifies the geometric robustness of efficiency scores.
We validate GeMA on synthetic data with non-convex frontiers, heterogeneous technologies and scale bias, and on four real-world case studies: global urban rail systems (COMET), British rail operators (ORR), national economies (Penn World Table) and a high-frequency wind-farm dataset. Across these domains GeMA behaves comparably to established methods when classical assumptions hold, and provides additional insight in settings with pronounced heterogeneity, non-convexity or size-related bias.
CIAware-Bench: Benchmarking Control Intervention Awareness Across Frontier LLMs
MATS;Mila – Quebec AI Institute(Mila – 魁北克人工智能研究所);Université de Montréal(蒙特利尔大学);Astra Fellowship;ELLIS Institute Tübingen, MPI for Intelligent Systems & Tübingen AI Center;LawZero;Google DeepMind(谷歌DeepMind)
Radiology's Last Exam (RadLE): Benchmarking Frontier Multimodal AI Against Human Experts and a Taxonomy of Visual Reasoning Errors in Radiology
Centre for Responsible Autonomous Systems in Healthcare (CRASH) Lab, Koita Centre for Digital Health(负责任的自主医疗系统中心(CRASH)实验室、Koita数字健康中心);Ashoka University(阿什oka大学);Independent Researcher(独立研究者);Koita Centre for Digital Health(Koita数字健康中心)
DiscoverPhysics: Benchmarking LLMs for Out-of-the-Box Scientific Thinking
Princeton University(普林斯顿大学);Boston University(波士顿大学);New York University(纽约大学);Flatiron Institute(Flatiron研究所);Institute for Advanced Studies(高级研究研究所)
Benchmarking Multi-Agent LLM Architectures for Financial Document Processing: A Comparative Study of Orchestration Patterns, Cost-Accuracy Tradeoffs and Production Scaling Strategies
Department of Computer Science and Engineering(计算机科学与工程系);New York University(纽约大学);New York, NY 10012(纽约市NY 10012)
A Closer Look at Agentic BBO: Benchmarking LLM Agents for Black-Box Optimization
State Key Laboratory of Novel Software Technology, Nanjing University(南京大学现代软件工程国家重点实验室);School of Artificial Intelligence, Nanjing University(南京大学人工智能学院)
AutoLab: Can Frontier Models Solve Long-Horizon Auto Research and Engineering Tasks?
MIT(麻省理工学院);Stanford University(斯坦福大学);University of California, Berkeley(加州大学伯克利分校);University of California, Los Angeles(加州大学洛杉矶分校);University of California, San Diego(加州大学圣地亚哥分校);University of Washington(华盛顿大学);University of Toronto(多伦多大学);University of Michigan(密歇根大学);National University of Singapore(新加坡国立大学);University of Tokyo(东京大学)
TerraIncognita: A Dynamic Benchmark for Species Discovery Using Frontier Models
Iowa State University(爱荷华州立大学);University of Arizona(亚利桑那大学);University of Nevada(内华达大学);Yanayacu Biological Station and Center for Creative Studies(亚纳亚库生物站和创意研究中心);New York University(纽约大学)
Benchmarking Time Series Generation Methods for Privacy-Preserving Forecasting
University of Minho(米尼奥大学);Universidade do Porto(波尔图大学);Laboratory for Artificial Intelligence and Computer Science (LIACC)(人工智能与计算机科学实验室(LIACC));Fraunhofer Portugal AICOS(弗劳恩霍夫葡萄牙AICOS机构);INESCTEC;University of Coimbra(科英布拉大学)