FregeLogic在SemEval 2026任务11中的应用：一种混合神经符号架构用于内容鲁棒的三段论有效性预测
FregeLogic at SemEval 2026 Task 11: A Hybrid Neuro-Symbolic Architecture for Content-Robust Syllogistic Validity Prediction
AI总结：
本文提出FregeLogic系统，结合多个LLM分类器与Z3 SMT求解器，通过形式逻辑验证减少内容偏差，提升三段论有效性预测的准确率和鲁棒性。
摘要：
我们介绍了FregeLogic，一种混合神经符号系统，用于SemEval-2026任务11（子任务1），旨在解决三段论有效性预测问题，同时减少预测中的内容效应。我们的方法结合了五个LLM分类器的集成，涵盖三个开源权重模型（Llama 4 Maverick、Llama 4 Scout和Qwen3-32B）以及不同的提示策略，与Z3 SMT求解器结合，后者作为形式逻辑的决定者。核心假设是LLM在集成中的分歧信号可能的基于内容的错误，其中现实世界的可信度影响逻辑判断。通过将这些争议案例交由Z3的结构化验证，我们的系统在数据集（N=960）上的嵌套5折交叉验证中实现了94.3%的准确率，内容效应为2.85，综合得分为41.88。这比纯集成（39.12）的综合得分提高了2.76点，准确率提高了0.9%，由内容效应的减少（3.39到2.85）驱动。采用结构化输出API调用进行Z3提取将失败率从约22%降至接近零，且通过存在公理的亚里士多德编码验证了任务注释。我们的结果表明，针对神经符号整合，在集成共识最低的区域应用形式方法，可以提高本任务使用的综合准确率加内容效应指标。
英文摘要：
We present FregeLogic, a hybrid neuro-symbolic system for SemEval-2026 Task 11 (Subtask 1), which addresses syllogistic validity prediction while reducing content effects on predictions. Our approach combines an ensemble of five LLM classifiers, spanning three open-weights models (Llama 4 Maverick, Llama 4 Scout, and Qwen3-32B) paired with varied prompting strategies, with a Z3 SMT solver that serves as a formal logic tiebreaker. The central hypothesis is that LLM disagreement within the ensemble signals likely content-biased errors, where real-world believability interferes with logical judgment. By deferring to Z3's structurally-grounded formal verification on these disputed cases, our system achieves 94.3% accuracy with a content effect of 2.85 and a combined score of 41.88 in nested 5-fold cross-validation on the dataset (N=960). This represents a 2.76-point improvement in combined score over the pure ensemble (39.12), with a 0.9% accuracy gain, driven by a 16% reduction in content effect (3.39 to 2.85). Adopting structured-output API calls for Z3 extraction reduced failure rates from ~22% to near zero, and an Aristotelian encoding with existence axioms was validated against task annotations. Our results suggest that targeted neuro-symbolic integration, applying formal methods precisely where ensemble consensus is lowest, can improve the combined accuracy-plus-content-effect metric used by this task.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
