EconCausal: A Context-Aware Economic Reasoning Benchmark for Large Language Models
Graduate School of Data Science, KAIST(韩国科学技术院数据科学研究生院);College of Business, KAIST(韩国科学技术院商学院);Data Science for Humanity Group, MPI-SP(马克斯·普朗克所际数据科学为人类集团);School of Computing, KAIST(韩国科学技术院计算学院);Division of Social Science, HKUST(香港科技大学社会科学系)
Donggyu Lee, Hyeok Yun, Meeyoung Cha, Sungwon Park, Sangyoon Park, Jihee Kim
备注:
29 pages. Revised version with updated evaluation results, robustness analyses, expert validation, and references
Socio-economic causal effects depend heavily on their institutional and environmental contexts. The same intervention can produce different, even opposite, effects across regulatory regimes, market conditions, time periods, or populations. This poses a challenge for large language models (LLMs) in decision-support roles: can they infer the direction of a causal effect under a specified context, and revise that judgment when the context changes?
To address this, we introduce EconCausal, a large-scale benchmark of 10,490 context-annotated causal triplets extracted from 2,595 high-quality empirical studies in top-tier economics and finance journals, constructed through a rigorous four-stage pipeline with multi-run consensus, context refinement, and multi-critic filtering.
Across 19 models, we observe source-sign over-transfer in cross-context prediction. While top models reach ~84% accuracy in single-context prediction, mean accuracy is 74.75% overall but drops to 44.27% when source and target signs disagree. On these cases, accuracy is 11.54 percentage points (pp) lower with the signed source example than with the target input alone. Mean accuracy is below 60% under corrupted source signs. Models also struggle to predict study-reported nonsignificant effects (19.16% accuracy).
The dataset and benchmark are publicly available at https://anonymous.4open.science/r/econcausal-benchmark-6F12.
Ideological Bias in LLMs' Economic Causal Reasoning
Graduate School of Data Science, KAIST(韩国科学技术院数据科学研究生院);College of Business, KAIST(韩国科学技术院商学院);School of Computing, KAIST(韩国科学技术院计算机学院);Division of Social Science, HKUST(香港科技大学社会科学系)
CityRiSE: Reasoning Urban Socio-Economic Status in Large Vision-Language Models via Reinforcement Learning
Information Hub, The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州)信息中心);Department of Electronic Engineering, BNRist, Tsinghua University(清华大学电子工程系)
Reasoning Like an Economist: Post-Training on Economic Problems Induces Strategic Generalization in LLMs
Duke University(杜克大学);EPIC Lab, Shanghai Jiao Tong University(上海交通大学EPIC实验室);Qwen Team, Alibaba Group(阿里集团Qwen团队);University of Pennsylvania(宾夕法尼亚大学);The University of Chicago(芝加哥大学)
EcoAct: Economic Agent Determines When to Register What Action
Pennsylvania State University(宾夕法尼亚州立大学);Microsoft(微软公司);The University of Washington(华盛顿大学);University of British Columbia(不列颠哥伦比亚大学);Google DeepMind(谷歌DeepMind)
Thinking Economically: A Hierarchical Framework for Adaptive-Complexity Reasoning in LLMs
The Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));The Hong Kong University of Science and Technology(香港科学与技术大学);Nanyang Technological University(南洋理工大学);Singapore Institute of Manufacturing Technology, A*STAR(新加坡制造技术研究所,A*STAR)
A Multi-LLM-Agent-Based Framework for Economic and Public Policy Analysis
Tsinghua University(清华大学);Academy of Mathematics and System Science of the Chinese Academy of Sciences(中国科学院数学与系统科学研究院);University of Chinese Academy of Sciences(中国科学院大学);Xiamen University(厦门大学);Xiamen University–Chinese Academy of Sciences(厦门大学-中国科学院);The Hong Kong University of Science and Technology(香港科技大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
LLM-based Agentic Reasoning Frameworks: A Survey from Methods to Scenarios
Beijing Jiaotong University(北京交通大学);Lancaster University(兰卡斯特大学);Max Planck Institute for Informatics, Saarland Informatics Campus(马克斯·普朗克研究所(信息学)萨尔兰州信息学校区);University of Electronic Science and Technology of China(电子科技大学)
ERA-IT: Aligning Semantic Models with Revealed Economic Preference for Real-Time and Explainable Patent Valuation
School of Computing(计算学院);Macquarie University(麦考瑞大学);School of Computer Science(计算机科学学院);University of Technology Sydney(悉尼技术大学);School of Management(管理学院);Zhejiang University(浙江大学)
Corrupted by Reasoning: Reasoning Language Models Become Free-Riders in Public Goods Games
University of Zurich(苏黎世大学);KAIST AI(韩国科学技术院人工智能);ETH Zürich(苏黎世联邦理工学院);MPI for Intelligent Systems(马克斯·普朗克智能系统研究所);University of Toronto(多伦多大学);Vector Institute(矢量研究所)
MineAgent: Towards Remote-Sensing Mineral Exploration with Multimodal Large Language Models
Australian Artificial Intelligence Institute, University of Technology Sydney(悉尼科技大学澳大利亚人工智能研究所);Faculty of Science and Engineering, Curtin University(科廷大学科学与工程学院)
A Survey of Sustainability in Large Language Models: Applications, Economics, and Challenges
Cleveland State University(克利夫兰州立大学);The Davey Tree Expert Company(戴维树专家公司);The MathWorks(美国MathWorks公司);Khoury College of Computer Science, Roux Institute at Northeastern University(东北大学鲁克斯研究所库尔计算机科学学院)
EffiReason-Bench: A Unified Benchmark for Evaluating and Advancing Efficient Reasoning in Large Language Models
Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Nanyang Technological University(南洋理工大学);The Hong Kong University of Science and Technology(香港科技大学);Singapore Institute of Manufacturing Technology, A*STAR(新加坡制造技术研究所,A*STAR)
An Agentic System for Rare Disease Diagnosis with Traceable Reasoning
Shanghai Jiao Tong University(上海交通大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Xinhua Hospital affiliated to Shanghai Jiao Tong University School of Medicine(上海交通大学医学院附属新华医院);Shanghai Institute for Pediatric Research(上海市儿科医学研究所);Harvard Medical School(哈佛医学院);The Affiliated Children’s Hospital of Xiangya School of Medicine(湘雅医学院附属儿童医院)