arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.22706cs.AI
收藏

通过多任务风险融合的LLM生成驾驶员干预信息的安全感知评估

Safety-Aware Evaluation of LLM-Generated Driver Intervention Messages through Multi-Task Risk Fusion

Faculty of Business and Commerce, Kansai University(关西大学商学部);Data Science and AI Innovation Research Promotion Center, Shiga University(滋贺大学数据科学与人工智能创新研究推进中心)

Keito Inoshita

AI总结:

提出DSAIS指标,结合规则与LLM评估五个维度,通过多任务风险融合框架提升干预信息质量,实验表明多任务集成使上下文相关性提升9.1%。

摘要:

现有的驾驶员干预系统依赖听觉警报和固定模板,未能利用多任务识别输出。通用指标如BLEU和BERTScore无法捕捉干预特定的质量维度,包括风险-紧迫性对齐、认知负荷和驾驶员可接受性。本文提出驾驶员安全感知干预分数(DSAIS),这是一种领域特定指标,通过结合轻量级规则计算与LLM评估的混合架构评估五个维度,并提供一个端到端框架,通过风险融合、状态历史管理和动态提示构建将四任务识别输出集成到LLM中。在AIDE数据集上使用五个模型和七个条件的实验表明,DSAIS在三个架构不同的评估者中实现了ICC 0.798-0.840,在所有控制条件下Cohen's d > 1.5。多维子分数分析量化了基于规则和基于LLM系统之间的上下文适应性差距,揭示多任务集成比基于规则的基线提高了9.1%的上下文相关性。消融实验表明,每个框架组件都对上下文相关性有贡献,子分数分解揭示了聚合评分掩盖的增益。驾驶员情绪识别被确定为最关键的上游因素,紧凑的本地LLM(7B-9B参数)实现了优于基于API的模型的质量,为车载部署提供了实用设计指南。

英文摘要:

Existing driver intervention systems rely on auditory alerts and fixed templates, failing to leverage multi-task recognition outputs. General-purpose metrics such as BLEU and BERTScore cannot capture intervention-specific quality dimensions including risk-urgency alignment, cognitive load, and driver acceptability. In this paper, we propose the Driver Safety-Aware Intervention Score (DSAIS), a domain-specific metric evaluating five dimensions through a hybrid architecture combining lightweight rule-based computation with LLM Judge evaluation, together with an end-to-end framework integrating four-task recognition outputs into an LLM through risk fusion, state history management, and dynamic prompt construction. Experiments on the AIDE dataset with five models and seven conditions demonstrate that DSAIS achieves ICC 0.798-0.840 across three architecturally distinct judges and Cohen's d > 1.5 across all control conditions. Multi-dimensional sub-score analysis quantifies the contextual adaptability gap between rule-based and LLM-based systems, revealing that multi-task integration improves contextual relevance by 9.1% over rule-based baselines. Ablation experiments demonstrate that each framework component contributes to contextual relevance, with sub-score decomposition revealing gains that aggregate scoring masks. Driver emotion recognition is identified as the most critical upstream factor, and compact local LLMs (7B--9B parameters) achieve quality superior to API-based models, providing practical design guidelines for in-vehicle deployment.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑