LongEval-Sci的全文时间检索的提交与诊断分析
Submitted and Diagnostic Analysis of Full-Text Temporal Retrieval for LongEval-Sci
AI总结:
研究LongEval-Sci科学检索,比较多种方法与基线。全文检索是基础,时间整合可提升官方纵向有效性,引用证据待优化。还报告基于摄取速度和陈旧覆盖漂移的定性每周IR系统更新监测分析。
摘要:
LongEval-Sci在集合变化下评估科学检索,系统需在当前语料库有效且随时间文档积累仍可用。本文报告2026年官方任务1结果及开发诊断。比较多种方法与基线,得出全文检索是最强基础,时间整合可提升官方纵向有效性,引用证据需优化等结论。还报告了基于摄取速度和陈旧覆盖漂移的定性每周IR系统更新监测分析。
英文摘要:
LongEval-Sci evaluates scientific retrieval under collection change, where a system should be effective on the current corpus and remain usable as documents accumulate over time. This paper reports both official Task 1 results and development diagnostics for LongEval-Sci 2026. We compare the official PyTerrier BM25 and Qwen3 dense baselines with full-text BM25, additive and router variants, temporal full-text retrieval, temporal+citation retrieval, RM3 query expansion, cross-encoder reranking, and reciprocal rank fusion (RRF). In the official DCTR evaluation, the temporalized full-text runs are our strongest submissions: FT BM25+temporal and FT BM25+temporal+citation obtain the best ARP on all three snapshots (0.285, 0.267, and 0.180 nDCG@10) and reduce snapshot-3 relative change from 0.481 for the BM25 pivot to 0.368. Citation features match the temporal-only variant but do not provide a measurable additional gain in the official summary. Our internal snapshot-1 diagnostics show a complementary pattern: full-text BM25 is the strongest single development retriever (DCTR nDCG@10 = 0.3302, MAP = 0.2853), RRF gives the best deep recall (Recall@1000 = 0.9667), and some uncalibrated overlays can sharply degrade top-rank quality. We therefore conclude that full-text retrieval is the strongest foundation, temporal integration can improve official longitudinal effectiveness when applied to that foundation, and citation evidence still requires cleaner ablation and calibration. Beyond ranking, we also report a qualitative weekly IR-system update-monitoring analysis based on ingestion velocity and stale-coverage drift.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。