Sift or Get Off the PoC:将信息检索应用于漏洞研究的SiftRank
Sift or Get Off the PoC: Applying Information Retrieval to Vulnerability Research with SiftRank
AI总结:
SiftRank利用LLM进行文档排序,通过高效算法实现大规模漏洞分析,可在99秒内识别漏洞修复函数。
摘要:
安全研究本质上是一个资源受限和由此导致优先级排序的问题。实际上,攻击面太大,而时间和精力却不足以分析所有内容。最有效的安全研究人员往往是那些最擅长直观决定应调查攻击面哪一部分的人。我们证明,选择众多可能性中最有可能的选项的问题可以重新表述为一个信息检索问题,并通过使用文档排名技术来解决,其中LLM执行重工作作为通用排名器。我们提出了SiftRank,一种通过三种关键机制实现O(n)复杂度的排名算法:使用LLM对文档进行小批量排序(每次约10个项目)的列表排序;基于转折点的收敛检测,当分数分布稳定时自适应终止排名;以及迭代细化,逐步将排名努力集中在最相关的文档上。与现有重排序方法不同,后者需要一个单独的第一阶段检索步骤来缩小数据集到约100个候选,SiftRank直接在数千个项目上操作,每个文档在多个随机批次中被评估,以缓解LLM不一致判断的影响。我们在N-day漏洞分析中展示了实际有效性,成功在99秒内以0.82美元的推理成本,从2197个修改的函数中识别出一个漏洞修复函数。我们的方法使通常受手动分析限制的问题能够实现可扩展的安全优先级排序,仅需标准的LLM API访问,无需专门的基础设施、嵌入或领域特定的微调。SiftRank的开源实现可在https://github.com/noperator/siftrank找到。
英文摘要:
Security research is fundamentally a problem of resource constraint and consequent prioritization. There is simply too much attack surface and too little time and energy to spend analyzing it all. The most effective security researchers are often those who are most skilled at intuitively deciding which part of an expansive attack surface to investigate. We demonstrate that this problem of selecting the most promising option from among many possibilities can be reframed as an information retrieval problem, and solved using document ranking techniques with LLMs performing the heavy lifting as general-purpose rankers. We present SiftRank, a ranking algorithm achieving O(n) complexity through three key mechanisms: listwise ranking using an LLM to order documents in small batches of approximately 10 items at a time; inflection-based convergence detection that adaptively terminates ranking when score distributions have stabilized; and iterative refinement that progressively focuses ranking effort on the most relevant documents. Unlike existing reranking approaches that require a separate first-stage retrieval step to narrow datasets to approximately 100 candidates, SiftRank operates directly on thousands of items, with each document evaluated across multiple randomized batches to mitigate inconsistent judgments by an LLM. We demonstrate practical effectiveness on N-day vulnerability analysis, successfully identifying a vulnerability-fixing function among 2,197 changed functions in a stripped binary firmware patch within 99 seconds at an inference cost of $0.82. Our approach enables scalable security prioritization for problems that are generally constrained by manual analysis, requiring only standard LLM API access without specialized infrastructure, embedding, or domain-specific fine-tuning. An open-source implementation of SiftRank may be found at https://github.com/noperator/siftrank.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。