AI 中文总结
本文通过提出HIVE工具集，研究语音与键盘输入扰动对LLM智能体性能的影响，发现语音转录扰动损害更大、两种扰动影响源于标记留存数量等七项结论。
AI 中文摘要
人类通过打字或说话向语言模型输入信息，每种输入渠道会留下独特特征：键盘输入会产生正字法噪声；语音输入则会因常规转录产生不流畅内容，以及AI语音听写工具带来的重构问题。这些扰动会如何影响大语言模型（LLM）的性能？本文提出HIVE（Human Input-Variation Engine，人类输入变化引擎），这是一套包含语音转录扰动和QWERTY键盘扰动的工具集，我们利用HIVE评估模型对这些扰动的鲁棒性，共得出七项发现：（i）语音转录扰动会降低所有被测指令调优模型的准确率，且代价来自转录的结构而非其中的填充词；（ii）QWERTY键盘扰动的代价更低，模型可承受大量此类扰动后准确率才会下降；（iii）两种扰动的影响均源于同一原因：问题的标记（token）在扰动后留存的数量，破坏标记会造成损害，而在标记旁添加新标记的代价很小；（iv）两种输入渠道的差异仅出现在答案需构建或推导的场景中，在多选题场景中无差异；（v）这种损害并非仅由测试集污染导致；（vi）无法通过轻量适配训练消除；（vii）增加思考预算可几乎完全恢复键盘输入渠道的性能，但对语音输入渠道无改善，且压缩语音在增加思考预算后表现更差。
英文摘要
Human input reaches language models by typing or speaking, and each channel leaves a distinct signature: orthographic noise for keyboards; for voice, disfluency from conventional transcription and restructuring from AI-backed dictation tools. How do they impact an LLM's performance? In this paper we present HIVE (Human Input-Variation Engine), a suite of voice transcription perturbations and QWERTY keyboard perturbations. We use HIVE to evaluate how robust models are to these perturbations. We present seven findings. (i) Voice transcription perturbations lower accuracy across every instruction-tuned model we test, and it is the structure of the transcription rather than its fillers that carries the cost. (ii) QWERTY keyboard perturbations cost less, and a model absorbs a lot of them before accuracy falls away. (iii) Both trace back to one cause, how many of the question's tokens survive the perturbation: destroying a token is what hurts, while adding new ones alongside it costs little. (iv) The gap between the two channels appears only where the answer must be constructed or deduced; on multiple choice there is none. (v) The harm does not solely come from test-set contamination. (vi) It cannot be trained away with lightweight adaptation. (vii) A thinking budget recovers the keyboard channel almost entirely but leaves the spoken registers untouched, and compressed speech is worse with it.
