arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.30028cs.CY
收藏

商业大语言模型能成为议会政治伙伴吗?比较大语言模型推理与罗马尼亚立法解释

Can Commercial LLMs Be Parliamentary Political Companions? Comparing LLM Reasoning Against Romanian Legislative Expuneri de Motive

GuidingAI

Iulian Lucău, Adelin-George Voicu

AI总结:

本文通过比较六种大语言模型在罗马尼亚议会法案解释上的表现,探讨其作为政治顾问工具的可靠性,发现前沿模型在语义接近度上表现优异,但所有模型在任务依赖性上均存在编造问题。

摘要:

本文评估商业大语言模型(LLMs)是否能作为可靠的政治理论顾问工具,通过比较其输出与官方立法推理。使用15个罗马尼亚参议院法案及其官方解释备忘录的数据集,测试了六个LLM,涵盖三个供应商家族和多个能力层级:GPT-5-mini、GPT-5-chat(OpenAI)、Claude Haiku 4.5(Anthropic)以及Llama 4 Maverick、Llama 3.3 70B和Llama 3.1 8B(Meta)。每个模型生成的预测理由通过结合LLM-as-Judge语义评分和程序化文本相似性度量的双框架评估。我们通过委托-代理理论和有限理性概念,将立法者视为委托人,将其委托任务给受有限理性约束的代理人,同时存在结构性信息不对称。结果揭示出明显的双层结构:前沿模型(Claude Haiku 4.5、GPT-5-chat、GPT-5-mini)在语义接近度评分上达到4.6分以上(满分5分),而开放式模型则集中在一层以下(Cohen's d大于1.4)。然而,所有模型在任务依赖性上均表现出编造问题,对标准化立法模板(如欧盟指令转译)表现良好,但对政治上独特提案生成合理但无根据的推理。我们引入“级联有限理性”概念,描述在受有限理性约束的委托人、代理人和评估者之间失败如何累积,并认为立法者的操作风险不是稳定的意识形态偏见,而是由训练数据覆盖所塑造的上下文无知。

英文摘要:

This paper evaluates whether commercial large language models (LLMs) can function as reliable political advisory tools by comparing their outputs against official legislative reasoning. Using a dataset of 15 Romanian Senate law proposals paired with their official explanatory memoranda (expuneri de motive), we test six LLMs spanning three provider families and multiple capability tiers: GPT-5-mini, GPT-5-chat (OpenAI), Claude Haiku 4.5 (Anthropic), and Llama 4 Maverick, Llama 3.3 70B, and Llama 3.1 8B (Meta). Each model generates predicted rationales evaluated through a dual framework combining LLM-as-Judge semantic scoring and programmatic text similarity metrics. We frame the LLM-politician relationship through principal-agent theory and bounded rationality, conceptualizing the legislator as a principal delegating advisory tasks to a boundedly rational agent under structural information asymmetry. Results reveal a sharp two-tier structure: frontier models (Claude Haiku 4.5, GPT-5-chat, GPT-5-mini) achieve statistically indistinguishable semantic closeness scores above 4.6 out of 5.0, while open-weight models cluster a full tier below (Cohen's d larger than 1.4). However, all models exhibit task-dependent confabulation, performing well on standardized legislative templates (e.g., EU directive transpositions) but generating plausible yet unfounded reasoning for politically idiosyncratic proposals. We introduce the concept of cascading bounded rationality to describe how failures compound across bounded principals, agents, and evaluators, and argue that the operative risk for legislators is not stable ideological bias but contextual ignorance shaped by training data coverage.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑